Skip to content

Commit 7c4f524

Browse files
peterepetere
committed
Logical replication support for initial data copy
Add functionality for a new subscription to copy the initial data in the tables and then sync with the ongoing apply process. For the copying, add a new internal COPY option to have the COPY source data provided by a callback function. The initial data copy works on the subscriber by receiving COPY data from the publisher and then providing it locally into a COPY that writes to the destination table. A WAL receiver can now execute full SQL commands. This is used here to obtain information about tables and publications. Several new options were added to CREATE and ALTER SUBSCRIPTION to control whether and when initial table syncing happens. Change pg_dump option --no-create-subscription-slots to --no-subscription-connect and use the new CREATE SUBSCRIPTION ... NOCONNECT option for that. Author: Petr Jelinek <petr.jelinek@2ndquadrant.com> Tested-by: Erik Rijkers <er@xs4all.nl>
1 parent 707576b commit 7c4f524

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+2966
-341
lines changed

‎contrib/file_fdw/file_fdw.c‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -662,6 +662,7 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
662662
node->ss.ss_currentRelation,
663663
filename,
664664
is_program,
665+
NULL,
665666
NIL,
666667
options);
667668

@@ -737,6 +738,7 @@ fileReScanForeignScan(ForeignScanState *node)
737738
node->ss.ss_currentRelation,
738739
festate->filename,
739740
festate->is_program,
741+
NULL,
740742
NIL,
741743
festate->options);
742744
}
@@ -1100,7 +1102,8 @@ file_acquire_sample_rows(Relation onerel, int elevel,
11001102
/*
11011103
* Create CopyState from FDW options.
11021104
*/
1103-
cstate = BeginCopyFrom(NULL, onerel, filename, is_program, NIL, options);
1105+
cstate = BeginCopyFrom(NULL, onerel, filename, is_program, NULL, NIL,
1106+
options);
11041107

11051108
/*
11061109
* Use per-tuple memory context to prevent leak of memory used to read

‎doc/src/sgml/catalogs.sgml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,11 @@
300300
<entry>logical replication subscriptions</entry>
301301
</row>
302302

303+
<row>
304+
<entry><link linkend="catalog-pg-subscription-rel"><structname>pg_subscription_rel</structname></link></entry>
305+
<entry>relation state for subscriptions</entry>
306+
</row>
307+
303308
<row>
304309
<entry><link linkend="catalog-pg-tablespace"><structname>pg_tablespace</structname></link></entry>
305310
<entry>tablespaces within this database cluster</entry>
@@ -6418,6 +6423,79 @@
64186423
</table>
64196424
</sect1>
64206425

6426+
<sect1 id="catalog-pg-subscription-rel">
6427+
<title><structname>pg_subscription_rel</structname></title>
6428+
6429+
<indexterm zone="catalog-pg-subscription-rel">
6430+
<primary>pg_subscription_rel</primary>
6431+
</indexterm>
6432+
6433+
<para>
6434+
The catalog <structname>pg_subscription_rel</structname> contains the
6435+
state for each replicated relation in each subscription. This is a
6436+
many-to-many mapping.
6437+
</para>
6438+
6439+
<para>
6440+
This catalog only contains tables known to the subscription after running
6441+
either <command>CREATE SUBSCRIPTION</command> or
6442+
<command>ALTER SUBSCRIPTION ... REFRESH</command>.
6443+
</para>
6444+
6445+
<table>
6446+
<title><structname>pg_subscription_rel</structname> Columns</title>
6447+
6448+
<tgroup cols="4">
6449+
<thead>
6450+
<row>
6451+
<entry>Name</entry>
6452+
<entry>Type</entry>
6453+
<entry>References</entry>
6454+
<entry>Description</entry>
6455+
</row>
6456+
</thead>
6457+
6458+
<tbody>
6459+
<row>
6460+
<entry><structfield>srsubid</structfield></entry>
6461+
<entry><type>oid</type></entry>
6462+
<entry><literal><link linkend="catalog-pg-subscription"><structname>pg_subscription</structname></link>.oid</literal></entry>
6463+
<entry>Reference to subscription</entry>
6464+
</row>
6465+
6466+
<row>
6467+
<entry><structfield>srrelid</structfield></entry>
6468+
<entry><type>oid</type></entry>
6469+
<entry><literal><link linkend="catalog-pg-class"><structname>pg_class</structname></link>.oid</literal></entry>
6470+
<entry>Reference to relation</entry>
6471+
</row>
6472+
6473+
<row>
6474+
<entry><structfield>srsubstate</structfield></entry>
6475+
<entry><type>char</type></entry>
6476+
<entry></entry>
6477+
<entry>
6478+
State code:
6479+
<literal>i</> = initialize,
6480+
<literal>d</> = data is being copied,
6481+
<literal>s</> = synchronized,
6482+
<literal>r</> = ready (normal replication)
6483+
</entry>
6484+
</row>
6485+
6486+
<row>
6487+
<entry><structfield>srsublsn</structfield></entry>
6488+
<entry><type>pg_lsn</type></entry>
6489+
<entry></entry>
6490+
<entry>
6491+
End LSN for <literal>s</> and <literal>r</> states.
6492+
</entry>
6493+
</row>
6494+
</tbody>
6495+
</tgroup>
6496+
</table>
6497+
</sect1>
6498+
64216499
<sect1 id="catalog-pg-tablespace">
64226500
<title><structname>pg_tablespace</structname></title>
64236501

‎doc/src/sgml/config.sgml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3449,6 +3449,31 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
34493449
</listitem>
34503450
</varlistentry>
34513451

3452+
<varlistentry id="guc-max-sync-workers-per-subscription" xreflabel="max_sync_workers_per_subscription">
3453+
<term><varname>max_sync_workers_per_subscription</varname> (<type>integer</type>)
3454+
<indexterm>
3455+
<primary><varname>max_sync_workers_per_subscription</> configuration parameter</primary>
3456+
</indexterm>
3457+
</term>
3458+
<listitem>
3459+
<para>
3460+
Maximum number of synchronization workers per subscription. This
3461+
parameter controls the amount of paralelism of the initial data copy
3462+
during the subscription initialization or when new tables are added.
3463+
</para>
3464+
<para>
3465+
Currently, there can be only one synchronization worker per table.
3466+
</para>
3467+
<para>
3468+
The synchronization workers are taken from the pool defined by
3469+
<varname>max_logical_replication_workers</varname>.
3470+
</para>
3471+
<para>
3472+
The default value is 2.
3473+
</para>
3474+
</listitem>
3475+
</varlistentry>
3476+
34523477
</variablelist>
34533478
</sect2>
34543479

‎doc/src/sgml/logical-replication.sgml‎

Lines changed: 40 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,11 @@
2424
</para>
2525

2626
<para>
27-
Logical replication sends changes on the publisher to the subscriber as
28-
they occur in real-time. The subscriber applies the data in the same order
29-
as the publisher so that transactional consistency is guaranteed for
27+
Logical replication of a table typically starts with a taking a snapshot
28+
of the data on the publisher database and copying that to the subscriber.
29+
Once that is done, the changes on the publisher are sent to the subscriber
30+
as they occur in real-time. The subscriber applies the data in the same
31+
order as the publisher so that transactional consistency is guaranteed for
3032
publications within a single subscription. This method of data replication
3133
is sometimes referred to as transactional replication.
3234
</para>
@@ -159,7 +161,9 @@
159161

160162
<para>
161163
Each subscription will receive changes via one replication slot (see
162-
<xref linkend="streaming-replication-slots">).
164+
<xref linkend="streaming-replication-slots">). Additional temporary
165+
replication slots may be required for the initial data synchronization
166+
of pre-existing table data.
163167
</para>
164168

165169
<para>
@@ -264,9 +268,25 @@
264268
to <literal>replica</literal>, which produces the usual effects on triggers
265269
and constraints.
266270
</para>
271+
272+
<sect2 id="logical-replication-snapshot">
273+
<title>Initial Snapshot</title>
274+
<para>
275+
The initial data in existing subscribed tables are snapshotted and
276+
copied in a parallel instance of a special kind of apply process.
277+
This process will create its own temporary replication slot and
278+
copy the existing data. Once existing data is copied, the worker
279+
enters synchronization mode, which ensures that the table is brought
280+
up to a synchronized state with the main apply process by streaming
281+
any changes that happened during the initial data copy using standard
282+
logical replication. Once the synchronization is done, the control
283+
of the replication of the table is given back to the main apply
284+
process where the replication continues as normal.
285+
</para>
286+
</sect2>
267287
</sect1>
268288

269-
<sect1 id="logical-replication-monitoring">
289+
<sect1 id="logical-replication-monitoring">
270290
<title>Monitoring</title>
271291

272292
<para>
@@ -287,7 +307,9 @@
287307
<para>
288308
Normally, there is a single apply process running for an enabled
289309
subscription. A disabled subscription or a crashed subscription will have
290-
zero rows in this view.
310+
zero rows in this view. If the initial data synchronization of any
311+
table is in progress, there will be additional workers for the tables
312+
being synchronized.
291313
</para>
292314
</sect1>
293315

@@ -337,20 +359,21 @@
337359
<para>
338360
On the publisher side, <varname>wal_level</varname> must be set to
339361
<literal>logical</literal>, and <varname>max_replication_slots</varname>
340-
must be set to at least the number of subscriptions expected to connect.
341-
And <varname>max_wal_senders</varname> should be set to at least the same
342-
as <varname>max_replication_slots</varname> plus the number of physical replicas
343-
that are connected at the same time.
362+
must be set to at least the number of subscriptions expected to connect,
363+
plus some reserve for table synchronization. And
364+
<varname>max_wal_senders</varname> should be set to at least the same as
365+
<varname>max_replication_slots</varname> plus the number of physical
366+
replicas that are connected at the same time.
344367
</para>
345368

346369
<para>
347370
The subscriber also requires the <varname>max_replication_slots</varname>
348371
to be set. In this case it should be set to at least the number of
349372
subscriptions that will be added to the subscriber.
350373
<varname>max_logical_replication_workers</varname> must be set to at
351-
least the number of subscriptions. Additionally the
352-
<varname>max_worker_processes</varname> may need to be adjusted to
353-
accommodate for replication workers, at least
374+
least the number of subscriptions, again plus some reserve for the table
375+
synchronization. Additionally the <varname>max_worker_processes</varname>
376+
may need to be adjusted to accommodate for replication workers, at least
354377
(<varname>max_logical_replication_workers</varname>
355378
+ <literal>1</literal>). Note that some extensions and parallel queries
356379
also take worker slots from <varname>max_worker_processes</varname>.
@@ -393,8 +416,10 @@ CREATE SUBSCRIPTION mysub CONNECTION 'dbname=foo host=bar user=repuser' PUBLICAT
393416
</para>
394417

395418
<para>
396-
The above will start the replication process of changes to
397-
<literal>users</literal> and <literal>departments</literal> tables.
419+
The above will start the replication process, which synchronizes the
420+
initial table contents of the tables <literal>users</literal> and
421+
<literal>departments</literal> and then starts replicating
422+
incremental changes to those tables.
398423
</para>
399424
</sect1>
400425
</chapter>

‎doc/src/sgml/monitoring.sgml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1863,6 +1863,12 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
18631863
<entry><type>integer</></entry>
18641864
<entry>Process ID of the subscription worker process</entry>
18651865
</row>
1866+
<row>
1867+
<entry><structfield>relid</></entry>
1868+
<entry><type>Oid</></entry>
1869+
<entry>OID of the relation that the worker is synchronizing; null for the
1870+
main apply worker</entry>
1871+
</row>
18661872
<row>
18671873
<entry><structfield>received_lsn</></entry>
18681874
<entry><type>pg_lsn</></entry>
@@ -1899,7 +1905,8 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
18991905
<para>
19001906
The <structname>pg_stat_subscription</structname> view will contain one
19011907
row per subscription for main worker (with null PID if the worker is
1902-
not running).
1908+
not running), and additional rows for workers handling the initial data
1909+
copy of the subscribed tables.
19031910
</para>
19041911

19051912
<table id="pg-stat-ssl-view" xreflabel="pg_stat_ssl">

‎doc/src/sgml/protocol.sgml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1487,7 +1487,7 @@ The commands accepted in walsender mode are:
14871487
</varlistentry>
14881488

14891489
<varlistentry id="protocol-replication-create-slot" xreflabel="CREATE_REPLICATION_SLOT">
1490-
<term><literal>CREATE_REPLICATION_SLOT</literal> <replaceable class="parameter">slot_name</> [ <literal>TEMPORARY</> ] { <literal>PHYSICAL</> [ <literal>RESERVE_WAL</> ] | <literal>LOGICAL</> <replaceable class="parameter">output_plugin</> [ <literal>EXPORT_SNAPSHOT</> | <literal>NOEXPORT_SNAPSHOT</> ] }
1490+
<term><literal>CREATE_REPLICATION_SLOT</literal> <replaceable class="parameter">slot_name</> [ <literal>TEMPORARY</> ] { <literal>PHYSICAL</> [ <literal>RESERVE_WAL</> ] | <literal>LOGICAL</> <replaceable class="parameter">output_plugin</> [ <literal>EXPORT_SNAPSHOT</> | <literal>NOEXPORT_SNAPSHOT</> | <literal>USE_SNAPSHOT</> ] }
14911491
<indexterm><primary>CREATE_REPLICATION_SLOT</primary></indexterm>
14921492
</term>
14931493
<listitem>
@@ -1542,12 +1542,17 @@ The commands accepted in walsender mode are:
15421542
<varlistentry>
15431543
<term><literal>EXPORT_SNAPSHOT</></term>
15441544
<term><literal>NOEXPORT_SNAPSHOT</></term>
1545+
<term><literal>USE_SNAPSHOT</></term>
15451546
<listitem>
15461547
<para>
15471548
Decides what to do with the snapshot created during logical slot
15481549
initialization. <literal>EXPORT_SNAPSHOT</>, which is the default,
15491550
will export the snapshot for use in other sessions. This option can't
1550-
be used inside a transaction. <literal>NOEXPORT_SNAPSHOT</> will
1551+
be used inside a transaction. <literal>USE_SNAPSHOT</> will use the
1552+
snapshot for the current transaction executing the command. This
1553+
option must be used in a transaction, and
1554+
<literal>CREATE_REPLICATION_SLOT</literal> must be the first command
1555+
run in that transaction. Finally, <literal>NOEXPORT_SNAPSHOT</> will
15511556
just use the snapshot for logical decoding as normal but won't do
15521557
anything else with it.
15531558
</para>

‎doc/src/sgml/ref/alter_subscription.sgml‎

Lines changed: 45 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,15 +21,21 @@ PostgreSQL documentation
2121

2222
<refsynopsisdiv>
2323
<synopsis>
24-
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> WITH ( <replaceable class="PARAMETER">option</replaceable> [, ... ] ) ]
24+
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> WITH ( <replaceable class="PARAMETER">suboption</replaceable> [, ... ] ) ]
2525

26-
<phrase>where <replaceable class="PARAMETER">option</replaceable> can be:</phrase>
26+
<phrase>where <replaceable class="PARAMETER">suboption</replaceable> can be:</phrase>
2727

28-
SLOT NAME = <replaceable class="PARAMETER">slot_name</replaceable>
28+
SLOT NAME = <replaceable class="PARAMETER">slot_name</replaceable>
29+
30+
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> SET PUBLICATION <replaceable class="PARAMETER">publication_name</replaceable> [, ...] { REFRESH WITH ( <replaceable class="PARAMETER">puboption</replaceable> [, ... ] ) | NOREFRESH }
31+
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> REFRESH PUBLICATION WITH ( <replaceable class="PARAMETER">puboption</replaceable> [, ... ] )
32+
33+
<phrase>where <replaceable class="PARAMETER">puboption</replaceable> can be:</phrase>
34+
35+
COPY DATA | NOCOPY DATA
2936

3037
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
3138
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> CONNECTION '<replaceable>conninfo</replaceable>'
32-
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> SET PUBLICATION <replaceable>publication_name</replaceable> [, ...]
3339
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> ENABLE
3440
ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> DISABLE
3541
</synopsis>
@@ -65,7 +71,6 @@ ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> DISABLE
6571

6672
<varlistentry>
6773
<term><literal>CONNECTION '<replaceable class="parameter">conninfo</replaceable>'</literal></term>
68-
<term><literal>SET PUBLICATION <replaceable class="parameter">publication_name</replaceable></literal></term>
6974
<term><literal>SLOT NAME = <replaceable class="parameter">slot_name</replaceable></literal></term>
7075
<listitem>
7176
<para>
@@ -76,6 +81,40 @@ ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> DISABLE
7681
</listitem>
7782
</varlistentry>
7883

84+
<varlistentry>
85+
<term><literal>SET PUBLICATION <replaceable class="parameter">publication_name</replaceable></literal></term>
86+
<listitem>
87+
<para>
88+
Changes list of subscribed publications. See
89+
<xref linkend="SQL-CREATESUBSCRIPTION"> for more information.
90+
</para>
91+
<para>
92+
When <literal>REFRESH</literal> is specified, this command will also
93+
act like <literal>REFRESH PUBLICATION</literal>. When
94+
<literal>NOREFRESH</literal> is specified, the comamnd will not try to
95+
refresh table information.
96+
</para>
97+
</listitem>
98+
</varlistentry>
99+
100+
<varlistentry>
101+
<term>REFRESH PUBLICATION</term>
102+
<listitem>
103+
<para>
104+
Fetch missing table information from publisher. This will start
105+
replication of tables that were added to the subscribed-to publications
106+
since the last invocation of <command>REFRESH PUBLICATION</command> or
107+
since <command>CREATE SUBSCRIPTION</command>.
108+
</para>
109+
<para>
110+
The <literal>COPY DATA</literal> and <literal>NOCOPY DATA</literal>
111+
options specify if the existing data in the publications that are being
112+
subscribed to should be copied. <literal>COPY DATA</literal> is the
113+
default.
114+
</para>
115+
</listitem>
116+
</varlistentry>
117+
79118
<varlistentry>
80119
<term><literal>ENABLE</literal></term>
81120
<listitem>
@@ -95,6 +134,7 @@ ALTER SUBSCRIPTION <replaceable class="PARAMETER">name</replaceable> DISABLE
95134
</para>
96135
</listitem>
97136
</varlistentry>
137+
98138
</variablelist>
99139
</refsect1>
100140

0 commit comments

Comments
 (0)