Skip to content

Commit 858ec11

Browse files
robertmhaasrobertmhaas
committed
Introduce replication slots.
Replication slots are a crash-safe data structure which can be created on either a master or a standby to prevent premature removal of write-ahead log segments needed by a standby, as well as (with hot_standby_feedback=on) pruning of tuples whose removal would cause replication conflicts. Slots have some advantages over existing techniques, as explained in the documentation. In a few places, we refer to the type of replication slots introduced by this patch as "physical" slots, because forthcoming patches for logical decoding will also have slots, but with somewhat different properties. Andres Freund and Robert Haas
1 parent 5bdef38 commit 858ec11

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+2356
-62
lines changed

‎doc/src/sgml/catalogs.sgml‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,11 @@
233233
<entry>query rewrite rules</entry>
234234
</row>
235235

236+
<row>
237+
<entry><link linkend="catalog-pg-replication-slots"><structname>pg_replication_slots</structname></link></entry>
238+
<entry>replication slot information</entry>
239+
</row>
240+
236241
<row>
237242
<entry><link linkend="catalog-pg-seclabel"><structname>pg_seclabel</structname></link></entry>
238243
<entry>security labels on database objects</entry>
@@ -5157,6 +5162,100 @@
51575162

51585163
</sect1>
51595164

5165+
<sect1 id="catalog-pg-replication-slots">
5166+
<title><structname>pg_replication_slots</structname></title>
5167+
5168+
<indexterm zone="catalog-pg-replication-slots">
5169+
<primary>pg_replication_slots</primary>
5170+
</indexterm>
5171+
5172+
<para>
5173+
The <structname>pg_replication_slots</structname> view provides a listing
5174+
of all replication slots that currently exist on the database cluster,
5175+
along with their current state.
5176+
</para>
5177+
5178+
<para>
5179+
For more on replication slots,
5180+
see <xref linkend="streaming-replication-slots">.
5181+
</para>
5182+
5183+
<table>
5184+
5185+
<title><structname>pg_replication_slots</structname> Columns</title>
5186+
5187+
<tgroup cols="4">
5188+
<thead>
5189+
<row>
5190+
<entry>Name</entry>
5191+
<entry>Type</entry>
5192+
<entry>References</entry>
5193+
<entry>Description</entry>
5194+
</row>
5195+
</thead>
5196+
5197+
<tbody>
5198+
<row>
5199+
<entry><structfield>slot_name</structfield></entry>
5200+
<entry><type>text</type></entry>
5201+
<entry></entry>
5202+
<entry>A unique, cluster-wide identifier for the replication slot</entry>
5203+
</row>
5204+
5205+
<row>
5206+
<entry><structfield>slot_type</structfield></entry>
5207+
<entry><type>text</type></entry>
5208+
<entry></entry>
5209+
<entry>The slot type - <literal>physical</> or <literal>logical</></entry>
5210+
</row>
5211+
5212+
<row>
5213+
<entry><structfield>datoid</structfield></entry>
5214+
<entry><type>oid</type></entry>
5215+
<entry><literal><link linkend="catalog-pg-database"><structname>pg_database</structname></link>.oid</literal></entry>
5216+
<entry>The oid of the database this slot is associated with, or
5217+
null. Only logical slots have an associated database.</entry>
5218+
</row>
5219+
5220+
<row>
5221+
<entry><structfield>database</structfield></entry>
5222+
<entry><type>text</type></entry>
5223+
<entry><literal><link linkend="catalog-pg-database"><structname>pg_database</structname></link>.datname</literal></entry>
5224+
<entry>The name of the database this slot is associated with, or
5225+
null. Only logical slots have an associated database.</entry>
5226+
</row>
5227+
5228+
<row>
5229+
<entry><structfield>active</structfield></entry>
5230+
<entry><type>boolean</type></entry>
5231+
<entry></entry>
5232+
<entry>True if this slot is currently actively being used</entry>
5233+
</row>
5234+
5235+
<row>
5236+
<entry><structfield>xmin</structfield></entry>
5237+
<entry><type>xid</type></entry>
5238+
<entry></entry>
5239+
<entry>The oldest transaction that this slot needs the database to
5240+
retain. <literal>VACUUM</literal> cannot remove tuples deleted
5241+
by any later transaction.
5242+
</entry>
5243+
</row>
5244+
5245+
<row>
5246+
<entry><structfield>restart_lsn</structfield></entry>
5247+
<entry><type>text</type></entry>
5248+
<entry></entry>
5249+
<entry>The address (<literal>LSN</literal>) of oldest WAL which still
5250+
might be required by the consumer of this slot and thus won't be
5251+
automatically removed during checkpoints.
5252+
</entry>
5253+
</row>
5254+
</tbody>
5255+
</tgroup>
5256+
</table>
5257+
</sect1>
5258+
51605259

51615260
<sect1 id="catalog-pg-seclabel">
51625261
<title><structname>pg_seclabel</structname></title>

‎doc/src/sgml/config.sgml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2348,6 +2348,25 @@ include 'filename'
23482348
</listitem>
23492349
</varlistentry>
23502350

2351+
<varlistentry id="guc-max-replication-slots" xreflabel="max_replication_slots">
2352+
<term><varname>max_replication_slots</varname> (<type>integer</type>)</term>
2353+
<indexterm>
2354+
<primary><varname>max_replication_slots</> configuration parameter</primary>
2355+
</indexterm>
2356+
<listitem>
2357+
<para>
2358+
Specifies the maximum number of replication slots
2359+
(see <xref linkend="streaming-replication-slots"> that the server
2360+
can support. The default is zero. This parameter can only be set at
2361+
server start.
2362+
<varname>wal_level</varname> must be set
2363+
to <literal>archive</literal> or higher to allow replication slots to
2364+
be used. Setting it to a lower value than the number of currently
2365+
existing replication slots will prevent the server from starting.
2366+
</para>
2367+
</listitem>
2368+
</varlistentry>
2369+
23512370
<varlistentry id="guc-wal-keep-segments" xreflabel="wal_keep_segments">
23522371
<term><varname>wal_keep_segments</varname> (<type>integer</type>)</term>
23532372
<indexterm>

‎doc/src/sgml/func.sgml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16290,6 +16290,76 @@ postgres=# SELECT * FROM pg_xlogfile_name_offset(pg_stop_backup());
1629016290
</para>
1629116291
</sect2>
1629216292

16293+
<sect2 id="functions-replication">
16294+
<title>Replication Functions</title>
16295+
16296+
<para>
16297+
PostgreSQL exposes a number of functions for controlling and interacting
16298+
with replication features. See <xref linkend="streaming-replication">
16299+
and <xref linkend="streaming-replication-slots">.
16300+
</para>
16301+
16302+
<para>
16303+
Many of these functions have equivalent commands in the replication
16304+
protocol; see <xref linkend="protocol-replication">.
16305+
</para>
16306+
16307+
<para>
16308+
The sections <xref linkend="functions-snapshot-synchronization">, <xref
16309+
linkend="functions-recovery-control"> and <xref
16310+
linkend="functions-admin-backup"> are also relevant for replication.
16311+
</para>
16312+
16313+
<table id="functions-replication-table">
16314+
<title>Replication <acronym>SQL</acronym> Functions</title>
16315+
<tgroup cols="3">
16316+
<thead>
16317+
<row>
16318+
<entry>Function</entry>
16319+
<entry>Return Type</entry>
16320+
<entry>Description</entry>
16321+
</row>
16322+
</thead>
16323+
<tbody>
16324+
<row>
16325+
<entry>
16326+
<indexterm>
16327+
<primary>pg_create_physical_replication_slot</primary>
16328+
</indexterm>
16329+
<literal><function>pg_create_physical_replication_slot(<parameter>slotname</parameter> <type>text</type>, <parameter>plugin</parameter> <type>text</type>)</function></literal>
16330+
</entry>
16331+
<entry>
16332+
(<parameter>slotname</parameter> <type>text</type>, <parameter>xlog_position</parameter> <type>text</type>)
16333+
</entry>
16334+
<entry>
16335+
Creates a new physical replication slot named
16336+
<parameter>slotname</parameter>. Streaming changes from a physical slot
16337+
is only possible with the walsender protocol - see <xref
16338+
linkend="protocol-replication">. Corresponds to the walsender protocol
16339+
command <literal>CREATE_REPLICATION_SLOT ... PHYSICAL</literal>.
16340+
</entry>
16341+
</row>
16342+
<row>
16343+
<entry>
16344+
<indexterm>
16345+
<primary>pg_drop_replication_slot</primary>
16346+
</indexterm>
16347+
<literal><function>pg_drop_replication_slot(<parameter>slotname</parameter> <type>text</type>)</function></literal>
16348+
</entry>
16349+
<entry>
16350+
(<parameter>slotname</parameter> <type>text</type>)
16351+
</entry>
16352+
<entry>
16353+
Drops the physical or logical replication slot
16354+
named <parameter>slotname</parameter>. Same as walsender protocol
16355+
command <literal>DROP_REPLICATION_SLOT</>.
16356+
</entry>
16357+
</row>
16358+
</tbody>
16359+
</tgroup>
16360+
</table>
16361+
</sect2>
16362+
1629316363
<sect2 id="functions-admin-dbobject">
1629416364
<title>Database Object Management Functions</title>
1629516365

‎doc/src/sgml/high-availability.sgml‎

Lines changed: 86 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -643,7 +643,9 @@ protocol to make nodes agree on a serializable transactional order.
643643
entries in <filename>pg_hba.conf</> with the database field set to
644644
<literal>replication</>. Also ensure <varname>max_wal_senders</> is set
645645
to a sufficiently large value in the configuration file of the primary
646-
server.
646+
server. If replication slots will be used,
647+
ensure that <varname>max_replication_slots</varname> is set sufficiently
648+
high as well.
647649
</para>
648650

649651
<para>
@@ -750,13 +752,14 @@ archive_cleanup_command = 'pg_archivecleanup /path/to/archive %r'
750752

751753
<para>
752754
If you use streaming replication without file-based continuous
753-
archiving, you have to set <varname>wal_keep_segments</> in the master
754-
to a value high enough to ensure that old WAL segments are not recycled
755-
too early, while the standby might still need them to catch up. If the
756-
standby falls behind too much, it needs to be reinitialized from a new
757-
base backup. If you set up a WAL archive that's accessible from the
758-
standby, <varname>wal_keep_segments</> is not required as the standby can always
759-
use the archive to catch up.
755+
archiving, the server might recycle old WAL segments before the standby
756+
has received them. If this occurs, the standby will need to be
757+
reinitialized from a new base backup. You can avoid this by setting
758+
<varname>wal_keep_segments</> to a value large enough to ensure that
759+
WAL segments are not recycled too early, or by configuration a replication
760+
slot for the standby. If you set up a WAL archive that's accessible from
761+
the standby, these solutions are not required, since the standby can
762+
always use the archive to catch up provided it retains enough segments.
760763
</para>
761764

762765
<para>
@@ -871,6 +874,81 @@ primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
871874
</sect3>
872875
</sect2>
873876

877+
<sect2 id="streaming-replication-slots">
878+
<title>Replication Slots</title>
879+
<indexterm zone="high-availability">
880+
<primary>Replication Slots</primary>
881+
</indexterm>
882+
<para>
883+
Replication slots provide an automated way to ensure that the master does
884+
not remove WAL segments until they have been received by all standbys,
885+
and that the master does not remove rows which could cause a
886+
<link linkend="hot-standby-conflict">recovery conflict</> even when the
887+
standby is disconnected.
888+
</para>
889+
<para>
890+
In lieu of using replication slots, it is possible to prevent the removal
891+
of old WAL segments using <xref linkend="guc-wal-keep-segments">, or by
892+
storing the segments in an archive using <xref linkend="restore-command">.
893+
However, these methods often result in retaining more WAL segments than
894+
required, whereas replication slots retain only the number of segments
895+
known to be needed. An advantage of these methods is that they bound
896+
the space requirement for <literal>pg_xlog</>; there is currently no way
897+
to do this using replication slots.
898+
</para>
899+
<para>
900+
Similarly, <varname>hot_standby_feedback</varname>
901+
and <varname>vacuum_defer_cleanup_age</varname> provide protection against
902+
relevant rows being removed by vacuum, but the former provides no
903+
protection during any time period when the standby is not connected,
904+
and the latter often needs to be set to a high value to provide adequate
905+
protection. Replication slots overcome these disadvantages.
906+
</para>
907+
<sect3 id="streaming-replication-slots-manipulation">
908+
<title>Querying and manipulating replication slots</title>
909+
<para>
910+
Each replication slot has a name, which can contain lower-case letters,
911+
numbers, and the underscore character.
912+
</para>
913+
<para>
914+
Existing replication slots and their state can be seen in the
915+
<link linkend="catalog-pg-replication-slots"><structname>pg_replication_slots</structname></link>
916+
view.
917+
</para>
918+
<para>
919+
Slots can be created and dropped either via the streaming replication
920+
protocol (see <xref linkend="protocol-replication">) or via SQL
921+
functions (see <xref linkend="functions-replication">).
922+
</para>
923+
</sect3>
924+
<sect3 id="streaming-replication-slots-config">
925+
<title>Configuration Example</title>
926+
<para>
927+
You can create a replication slot like this:
928+
<programlisting>
929+
postgres=# SELECT * FROM pg_create_physical_replication_slot('node_a_slot');
930+
slotname | xlog_position
931+
-------------+---------------
932+
node_a_slot |
933+
934+
postgres=# SELECT * FROM pg_replication_slots;
935+
slot_name | slot_type | datoid | database | active | xmin | restart_lsn
936+
-------------+-----------+--------+----------+--------+------+-------------
937+
node_a_slot | physical | 0 | | f | |
938+
(1 row)
939+
</programlisting>
940+
To configure the standby to use this slot, <varname>primary_slotname</>
941+
should be configured in the standby's <filename>recovery.conf</>.
942+
Here is a simple example:
943+
<programlisting>
944+
standby_mode = 'on'
945+
primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
946+
primary_slotname = 'node_a_slot'
947+
</programlisting>
948+
</para>
949+
</sect3>
950+
</sect2>
951+
874952
<sect2 id="cascading-replication">
875953
<title>Cascading Replication</title>
876954

0 commit comments

Comments
 (0)