Add a GetForeignUpperPaths callback function for FDWs.

This is basically like the just-added create_upper_paths_hook, but
control is funneled only to the FDW responsible for all the baserels
of the current query; so providing such a callback is much less likely
to add useless overhead than using the hook function is.

The documentation is a bit sketchy.  We'll likely want to improve it,
and/or adjust the call conventions, when we get some experience with
actually using this callback.  Hopefully somebody will find time to
experiment with it before 9.6 feature freeze.

Author: tglsfdc

doc/src/sgml/fdwhandler.sgml

@@ -176,7 +176,8 @@ GetForeignPlan (PlannerInfo *root,
      access path.  This is called at the end of query planning.
      The parameters are as for <function>GetForeignRelSize</>, plus
      the selected <structname>ForeignPath</> (previously produced by
-     <function>GetForeignPaths</> or <function>GetForeignJoinPaths</>),
+     <function>GetForeignPaths</>, <function>GetForeignJoinPaths</>,
+     or <function>GetForeignUpperPaths</>),
      the target list to be emitted by the plan node,
      the restriction clauses to be enforced by the plan node,
      and the outer subplan of the <structname>ForeignScan</>,
@@ -344,6 +345,38 @@ GetForeignJoinPaths (PlannerInfo *root,
     </para>
    </sect2>
 
+   <sect2 id="fdw-callbacks-upper-planning">
+    <title>FDW Routines For Planning Post-Scan/Join Processing</title>
+
+    <para>
+     If an FDW supports performing remote post-scan/join processing, such as
+     remote aggregation, it should provide this callback function:
+    </para>
+
+    <para>
+<programlisting>
+void
+GetForeignUpperPaths (PlannerInfo *root,
+                      RelOptInfo *scan_join_rel);
+</programlisting>
+     Create possible access paths for <firstterm>upper relation</> processing,
+     which is the planner's term for all post-scan/join query processing, such
+     as aggregation, window functions, sorting, and table updates.  This
+     optional function is called during query planning.  Currently, it is
+     called only if all base relation(s) involved in the query belong to the
+     same FDW.  This function should generate <structname>ForeignPath</>
+     path(s) for the steps that the FDW knows how to perform remotely, and
+     call <function>add_path</> to add these paths to the appropriate upper
+     relation.  As with <function>GetForeignJoinPaths</>, it is not necessary
+     that this function succeed in creating any paths, since paths involving
+     local processing are always possible.
+    </para>
+
+    <para>
+     See <xref linkend="fdw-planning"> for additional information.
+    </para>
+   </sect2>
+
    <sect2 id="fdw-callbacks-update">
     <title>FDW Routines For Updating Foreign Tables</title>
 
@@ -1160,7 +1193,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
     <para>
      The FDW callback functions <function>GetForeignRelSize</>,
      <function>GetForeignPaths</>, <function>GetForeignPlan</>,
-     <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
+     <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>,
+     and <function>GetForeignUpperPaths</>
      must fit into the workings of the <productname>PostgreSQL</> planner.
      Here are some notes about what they must do.
     </para>
@@ -1322,7 +1356,7 @@ GetForeignServerByName(const char *name, bool missing_ok);
      An FDW might additionally support direct execution of some plan actions
      that are above the level of scans and joins, such as grouping or
      aggregation.  To offer such options, the FDW should generate paths
-     (probably ForeignPaths or CustomPaths) and insert them into the
+     and insert them into the
      appropriate <firstterm>upper relation</>.  For example, a path
      representing remote aggregation should be inserted into the relation
      obtained from <literal>fetch_upper_rel(root, UPPERREL_GROUP_AGG,
@@ -1332,6 +1366,9 @@ GetForeignServerByName(const char *name, bool missing_ok);
      else there will be an error at plan time).  If the remote-aggregation
      path wins, which it usually would, it will be converted into a plan in
      the usual way, by calling <function>GetForeignPlan</>.
+     Usually the most convenient place to generate such paths is in
+     the <function>GetForeignUpperPaths</> callback function, although
+     it can be done earlier if that seems appropriate.
     </para>
 
     <para>

src/backend/optimizer/README

@@ -916,7 +916,10 @@ remote aggregation into the UPPERREL_GROUP_AGG upperrel, if it notices
 that the query represents an aggregation that could be done entirely on
 the foreign server.  That Path will then compete with Paths representing
 local aggregation on a regular scan of the foreign table, once the core
-planner reaches the point of considering aggregation.
+planner reaches the point of considering aggregation.  (In practice,
+it will usually be more convenient for FDWs to detect such cases in a
+GetForeignUpperPaths callback; but that still represents injecting a
+Path before the core code has touched the corresponding upperrel.)
 
 
 Parallel Query and Partial Paths

src/backend/optimizer/plan/planner.c

@@ -1752,13 +1752,17 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
 		root->upper_targets[UPPERREL_GROUP_AGG] = grouping_target;
 
 		/*
-		 * Let extensions, particularly CustomScan providers, consider
-		 * injecting extension Paths into the query's upperrels, where they
-		 * will compete with the Paths we create below.  We pass the final
-		 * scan/join rel because that's not so easily findable from the
-		 * PlannerInfo struct; anything else the hook wants to know should be
-		 * obtainable via "root".
+		 * Let extensions, particularly FDWs and CustomScan providers,
+		 * consider injecting extension Paths into the query's upperrels,
+		 * where they will compete with the Paths we create below.  We pass
+		 * the final scan/join rel because that's not so easily findable from
+		 * the PlannerInfo struct; anything else the hooks want to know should
+		 * be obtainable via "root".
 		 */
+		if (current_rel->fdwroutine &&
+			current_rel->fdwroutine->GetForeignUpperPaths)
+			current_rel->fdwroutine->GetForeignUpperPaths(root, current_rel);
+
 		if (create_upper_paths_hook)
 			(*create_upper_paths_hook) (root, current_rel);
 

src/backend/optimizer/util/pathnode.c

@@ -1813,15 +1813,15 @@ create_worktablescan_path(PlannerInfo *root, RelOptInfo *rel,
 
 /*
  * create_foreignscan_path
- *	  Creates a path corresponding to a scan of a foreign table or
- *	  a foreign join, returning the pathnode.
+ *	  Creates a path corresponding to a scan of a foreign table, foreign join,
+ *	  or foreign upper-relation processing, returning the pathnode.
  *
  * This function is never called from core Postgres; rather, it's expected
- * to be called by the GetForeignPaths or GetForeignJoinPaths function of
- * a foreign data wrapper.  We make the FDW supply all fields of the path,
- * since we do not have any way to calculate them in core.  However, there
- * is a sane default for the pathtarget (rel->reltarget), so we let a NULL
- * for "target" select that.
+ * to be called by the GetForeignPaths, GetForeignJoinPaths, or
+ * GetForeignUpperPaths function of a foreign data wrapper.  We make the FDW
+ * supply all fields of the path, since we do not have any way to calculate
+ * them in core.  However, there is a usually-sane default for the pathtarget
+ * (rel->reltarget), so we let a NULL for "target" select that.
  */
 ForeignPath *
 create_foreignscan_path(PlannerInfo *root, RelOptInfo *rel,

src/include/foreign/fdwapi.h

@@ -59,6 +59,9 @@ typedef void (*GetForeignJoinPaths_function) (PlannerInfo *root,
 														  JoinType jointype,
 												   JoinPathExtraData *extra);
 
+typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
+											   RelOptInfo *scan_join_rel);
+
 typedef void (*AddForeignUpdateTargets_function) (Query *parsetree,
 												   RangeTblEntry *target_rte,
 												   Relation target_relation);
@@ -166,6 +169,9 @@ typedef struct FdwRoutine
 	/* Functions for remote-join planning */
 	GetForeignJoinPaths_function GetForeignJoinPaths;
 
+	/* Functions for remote upper-relation (post scan/join) planning */
+	GetForeignUpperPaths_function GetForeignUpperPaths;
+
 	/* Functions for updating foreign tables */
 	AddForeignUpdateTargets_function AddForeignUpdateTargets;
 	PlanForeignModify_function PlanForeignModify;