Skip to content

Commit 7b504eb

Browse files
alvherrealvherre
committed
Implement multivariate n-distinct coefficients
Add support for explicitly declared statistic objects (CREATE STATISTICS), allowing collection of statistics on more complex combinations that individual table columns. Companion commands DROP STATISTICS and ALTER STATISTICS ... OWNER TO / SET SCHEMA / RENAME are added too. All this DDL has been designed so that more statistic types can be added later on, such as multivariate most-common-values and multivariate histograms between columns of a single table, leaving room for permitting columns on multiple tables, too, as well as expressions. This commit only adds support for collection of n-distinct coefficient on user-specified sets of columns in a single table. This is useful to estimate number of distinct groups in GROUP BY and DISTINCT clauses; estimation errors there can cause over-allocation of memory in hashed aggregates, for instance, so it's a worthwhile problem to solve. A new special pseudo-type pg_ndistinct is used. (num-distinct estimation was deemed sufficiently useful by itself that this is worthwhile even if no further statistic types are added immediately; so much so that another version of essentially the same functionality was submitted by Kyotaro Horiguchi: https://postgr.es/m/20150828.173334.114731693.horiguchi.kyotaro@lab.ntt.co.jp though this commit does not use that code.) Author: Tomas Vondra. Some code rework by Álvaro. Reviewed-by: Dean Rasheed, David Rowley, Kyotaro Horiguchi, Jeff Janes, Ideriha Takeshi Discussion: https://postgr.es/m/543AFA15.4080608@fuzzy.cz https://postgr.es/m/20170320190220.ixlaueanxegqd5gr@alvherre.pgsql
1 parent f120b61 commit 7b504eb

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+3599
-63
lines changed

‎doc/src/sgml/catalogs.sgml‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -295,6 +295,11 @@
295295
<entry>planner statistics</entry>
296296
</row>
297297

298+
<row>
299+
<entry><link linkend="catalog-pg-statistic-ext"><structname>pg_statistic_ext</structname></link></entry>
300+
<entry>extended planner statistics</entry>
301+
</row>
302+
298303
<row>
299304
<entry><link linkend="catalog-pg-subscription"><structname>pg_subscription</structname></link></entry>
300305
<entry>logical replication subscriptions</entry>
@@ -4247,6 +4252,98 @@
42474252
</table>
42484253
</sect1>
42494254

4255+
<sect1 id="catalog-pg-statistic-ext">
4256+
<title><structname>pg_statistic_ext</structname></title>
4257+
4258+
<indexterm zone="catalog-pg-statistic-ext">
4259+
<primary>pg_statistic_ext</primary>
4260+
</indexterm>
4261+
4262+
<para>
4263+
The catalog <structname>pg_statistic_ext</structname>
4264+
holds extended planner statistics.
4265+
</para>
4266+
4267+
<table>
4268+
<title><structname>pg_statistic_ext</> Columns</title>
4269+
4270+
<tgroup cols="4">
4271+
<thead>
4272+
<row>
4273+
<entry>Name</entry>
4274+
<entry>Type</entry>
4275+
<entry>References</entry>
4276+
<entry>Description</entry>
4277+
</row>
4278+
</thead>
4279+
4280+
<tbody>
4281+
4282+
<row>
4283+
<entry><structfield>starelid</structfield></entry>
4284+
<entry><type>oid</type></entry>
4285+
<entry><literal><link linkend="catalog-pg-class"><structname>pg_class</structname></link>.oid</literal></entry>
4286+
<entry>The table that the described columns belongs to</entry>
4287+
</row>
4288+
4289+
<row>
4290+
<entry><structfield>staname</structfield></entry>
4291+
<entry><type>name</type></entry>
4292+
<entry></entry>
4293+
<entry>Name of the statistic.</entry>
4294+
</row>
4295+
4296+
<row>
4297+
<entry><structfield>stanamespace</structfield></entry>
4298+
<entry><type>oid</type></entry>
4299+
<entry><literal><link linkend="catalog-pg-namespace"><structname>pg_namespace</structname></link>.oid</literal></entry>
4300+
<entry>
4301+
The OID of the namespace that contains this statistic
4302+
</entry>
4303+
</row>
4304+
4305+
<row>
4306+
<entry><structfield>staowner</structfield></entry>
4307+
<entry><type>oid</type></entry>
4308+
<entry><literal><link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.oid</literal></entry>
4309+
<entry>Owner of the statistic</entry>
4310+
</row>
4311+
4312+
<row>
4313+
<entry><structfield>staenabled</structfield></entry>
4314+
<entry><type>char[]</type></entry>
4315+
<entry></entry>
4316+
<entry>
4317+
An array with the modes of the enabled statistic types, encoded as
4318+
<literal>d</literal> for ndistinct coefficients.
4319+
</entry>
4320+
</row>
4321+
4322+
<row>
4323+
<entry><structfield>stakeys</structfield></entry>
4324+
<entry><type>int2vector</type></entry>
4325+
<entry><literal><link linkend="catalog-pg-attribute"><structname>pg_attribute</structname></link>.attnum</literal></entry>
4326+
<entry>
4327+
This is an array of values that indicate which table columns this
4328+
statistic covers. For example a value of <literal>1 3</literal> would
4329+
mean that the first and the third table columns make up the statistic key.
4330+
</entry>
4331+
</row>
4332+
4333+
<row>
4334+
<entry><structfield>standistinct</structfield></entry>
4335+
<entry><type>pg_ndistinct</type></entry>
4336+
<entry></entry>
4337+
<entry>
4338+
N-distinct coefficients, serialized as <structname>pg_ndistinct</> type.
4339+
</entry>
4340+
</row>
4341+
4342+
</tbody>
4343+
</tgroup>
4344+
</table>
4345+
</sect1>
4346+
42504347
<sect1 id="catalog-pg-namespace">
42514348
<title><structname>pg_namespace</structname></title>
42524349

‎doc/src/sgml/func.sgml‎

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -16720,6 +16720,10 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
1672016720
<primary>pg_get_serial_sequence</primary>
1672116721
</indexterm>
1672216722

16723+
<indexterm>
16724+
<primary>pg_get_statisticsextdef</primary>
16725+
</indexterm>
16726+
1672316727
<indexterm>
1672416728
<primary>pg_get_triggerdef</primary>
1672516729
</indexterm>
@@ -16889,6 +16893,11 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
1688916893
<entry>get name of the sequence that a <type>serial</type>, <type>smallserial</type> or <type>bigserial</type> column
1689016894
uses</entry>
1689116895
</row>
16896+
<row>
16897+
<entry><literal><function>pg_get_statisticsextdef(<parameter>statext_oid</parameter>)</function></literal></entry>
16898+
<entry><type>text</type></entry>
16899+
<entry>get <command>CREATE STATISTICS</> command for extended statistics objects</entry>
16900+
</row>
1689216901
<row>
1689316902
<entry><function>pg_get_triggerdef</function>(<parameter>trigger_oid</parameter>)</entry>
1689416903
<entry><type>text</type></entry>
@@ -17034,19 +17043,20 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
1703417043
<para>
1703517044
<function>pg_get_constraintdef</function>,
1703617045
<function>pg_get_indexdef</function>, <function>pg_get_ruledef</function>,
17037-
and <function>pg_get_triggerdef</function>, respectively reconstruct the
17038-
creating command for a constraint, index, rule, or trigger. (Note that this
17039-
is a decompiled reconstruction, not the original text of the command.)
17040-
<function>pg_get_expr</function> decompiles the internal form of an
17041-
individual expression, such as the default value for a column. It can be
17042-
useful when examining the contents of system catalogs. If the expression
17043-
might contain Vars, specify the OID of the relation they refer to as the
17044-
second parameter; if no Vars are expected, zero is sufficient.
17045-
<function>pg_get_viewdef</function> reconstructs the <command>SELECT</>
17046-
query that defines a view. Most of these functions come in two variants,
17047-
one of which can optionally <quote>pretty-print</> the result. The
17048-
pretty-printed format is more readable, but the default format is more
17049-
likely to be interpreted the same way by future versions of
17046+
<function>pg_get_statisticsextdef</function>, and
17047+
<function>pg_get_triggerdef</function>, respectively reconstruct the
17048+
creating command for a constraint, index, rule, extended statistics object,
17049+
or trigger. (Note that this is a decompiled reconstruction, not the
17050+
original text of the command.) <function>pg_get_expr</function> decompiles
17051+
the internal form of an individual expression, such as the default value
17052+
for a column. It can be useful when examining the contents of system
17053+
catalogs. If the expression might contain Vars, specify the OID of the
17054+
relation they refer to as the second parameter; if no Vars are expected,
17055+
zero is sufficient. <function>pg_get_viewdef</function> reconstructs the
17056+
<command>SELECT</> query that defines a view. Most of these functions come
17057+
in two variants, one of which can optionally <quote>pretty-print</> the
17058+
result. The pretty-printed format is more readable, but the default format
17059+
is more likely to be interpreted the same way by future versions of
1705017060
<productname>PostgreSQL</>; avoid using pretty-printed output for dump
1705117061
purposes. Passing <literal>false</> for the pretty-print parameter yields
1705217062
the same result as the variant that does not have the parameter at all.

‎doc/src/sgml/ref/allfiles.sgml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ Complete list of usable sgml source files in this directory.
3434
<!ENTITY alterSequence SYSTEM "alter_sequence.sgml">
3535
<!ENTITY alterSubscription SYSTEM "alter_subscription.sgml">
3636
<!ENTITY alterSystem SYSTEM "alter_system.sgml">
37+
<!ENTITY alterStatistics SYSTEM "alter_statistics.sgml">
3738
<!ENTITY alterTable SYSTEM "alter_table.sgml">
3839
<!ENTITY alterTableSpace SYSTEM "alter_tablespace.sgml">
3940
<!ENTITY alterTSConfig SYSTEM "alter_tsconfig.sgml">
@@ -80,6 +81,7 @@ Complete list of usable sgml source files in this directory.
8081
<!ENTITY createSchema SYSTEM "create_schema.sgml">
8182
<!ENTITY createSequence SYSTEM "create_sequence.sgml">
8283
<!ENTITY createServer SYSTEM "create_server.sgml">
84+
<!ENTITY createStatistics SYSTEM "create_statistics.sgml">
8385
<!ENTITY createSubscription SYSTEM "create_subscription.sgml">
8486
<!ENTITY createTable SYSTEM "create_table.sgml">
8587
<!ENTITY createTableAs SYSTEM "create_table_as.sgml">
@@ -126,6 +128,7 @@ Complete list of usable sgml source files in this directory.
126128
<!ENTITY dropSchema SYSTEM "drop_schema.sgml">
127129
<!ENTITY dropSequence SYSTEM "drop_sequence.sgml">
128130
<!ENTITY dropServer SYSTEM "drop_server.sgml">
131+
<!ENTITY dropStatistics SYSTEM "drop_statistics.sgml">
129132
<!ENTITY dropSubscription SYSTEM "drop_subscription.sgml">
130133
<!ENTITY dropTable SYSTEM "drop_table.sgml">
131134
<!ENTITY dropTableSpace SYSTEM "drop_tablespace.sgml">

‎doc/src/sgml/ref/alter_statistics.sgml‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
<!--
2+
doc/src/sgml/ref/alter_statistics.sgml
3+
PostgreSQL documentation
4+
-->
5+
6+
<refentry id="SQL-ALTERSTATISTICS">
7+
<indexterm zone="sql-alterstatistics">
8+
<primary>ALTER STATISTICS</primary>
9+
</indexterm>
10+
11+
<refmeta>
12+
<refentrytitle>ALTER STATISTICS</refentrytitle>
13+
<manvolnum>7</manvolnum>
14+
<refmiscinfo>SQL - Language Statements</refmiscinfo>
15+
</refmeta>
16+
17+
<refnamediv>
18+
<refname>ALTER STATISTICS</refname>
19+
<refpurpose>
20+
change the definition of a extended statistics
21+
</refpurpose>
22+
</refnamediv>
23+
24+
<refsynopsisdiv>
25+
<synopsis>
26+
ALTER STATISTICS <replaceable class="parameter">name</replaceable> OWNER TO { <replaceable class="PARAMETER">new_owner</replaceable> | CURRENT_USER | SESSION_USER }
27+
ALTER STATISTICS <replaceable class="parameter">name</replaceable> RENAME TO <replaceable class="parameter">new_name</replaceable>
28+
ALTER STATISTICS <replaceable class="parameter">name</replaceable> SET SCHEMA <replaceable class="parameter">new_schema</replaceable>
29+
</synopsis>
30+
</refsynopsisdiv>
31+
32+
<refsect1>
33+
<title>Description</title>
34+
35+
<para>
36+
<command>ALTER STATISTICS</command> changes the parameters of an existing
37+
extended statistics. Any parameters not specifically set in the
38+
<command>ALTER STATISTICS</command> command retain their prior settings.
39+
</para>
40+
41+
<para>
42+
You must own the statistics to use <command>ALTER STATISTICS</>.
43+
To change a statistics' schema, you must also have <literal>CREATE</>
44+
privilege on the new schema.
45+
To alter the owner, you must also be a direct or indirect member of the new
46+
owning role, and that role must have <literal>CREATE</literal> privilege on
47+
the statistics' schema. (These restrictions enforce that altering the owner
48+
doesn't do anything you couldn't do by dropping and recreating the statistics.
49+
However, a superuser can alter ownership of any statistics anyway.)
50+
</para>
51+
</refsect1>
52+
53+
<refsect1>
54+
<title>Parameters</title>
55+
56+
<para>
57+
<variablelist>
58+
<varlistentry>
59+
<term><replaceable class="parameter">name</replaceable></term>
60+
<listitem>
61+
<para>
62+
The name (optionally schema-qualified) of the statistics to be altered.
63+
</para>
64+
</listitem>
65+
</varlistentry>
66+
67+
<varlistentry>
68+
<term><replaceable class="PARAMETER">new_owner</replaceable></term>
69+
<listitem>
70+
<para>
71+
The user name of the new owner of the statistics.
72+
</para>
73+
</listitem>
74+
</varlistentry>
75+
76+
<varlistentry>
77+
<term><replaceable class="parameter">new_name</replaceable></term>
78+
<listitem>
79+
<para>
80+
The new name for the statistics.
81+
</para>
82+
</listitem>
83+
</varlistentry>
84+
85+
<varlistentry>
86+
<term><replaceable class="parameter">new_schema</replaceable></term>
87+
<listitem>
88+
<para>
89+
The new schema for the statistics.
90+
</para>
91+
</listitem>
92+
</varlistentry>
93+
94+
</variablelist>
95+
</para>
96+
</refsect1>
97+
98+
<refsect1>
99+
<title>Compatibility</title>
100+
101+
<para>
102+
There's no <command>ALTER STATISTICS</command> command in the SQL standard.
103+
</para>
104+
</refsect1>
105+
106+
<refsect1>
107+
<title>See Also</title>
108+
109+
<simplelist type="inline">
110+
<member><xref linkend="sql-createstatistics"></member>
111+
<member><xref linkend="sql-dropstatistics"></member>
112+
</simplelist>
113+
</refsect1>
114+
115+
</refentry>

‎doc/src/sgml/ref/alter_table.sgml‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,12 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="PARAMETER">name</replaceable>
119119
<para>
120120
This form drops a column from a table. Indexes and
121121
table constraints involving the column will be automatically
122-
dropped as well. You will need to say <literal>CASCADE</> if
123-
anything outside the table depends on the column, for example,
124-
foreign key references or views.
122+
dropped as well.
123+
Multivariate statistics referencing the dropped column will also be
124+
removed if the removal of the column would cause the statistics to
125+
contain data for only a single column.
126+
You will need to say <literal>CASCADE</> if anything outside the table
127+
depends on the column, for example, foreign key references or views.
125128
If <literal>IF EXISTS</literal> is specified and the column
126129
does not exist, no error is thrown. In this case a notice
127130
is issued instead.

‎doc/src/sgml/ref/comment.sgml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ COMMENT ON
5151
SCHEMA <replaceable class="PARAMETER">object_name</replaceable> |
5252
SEQUENCE <replaceable class="PARAMETER">object_name</replaceable> |
5353
SERVER <replaceable class="PARAMETER">object_name</replaceable> |
54+
STATISTICS <replaceable class="PARAMETER">object_name</replaceable> |
5455
TABLE <replaceable class="PARAMETER">object_name</replaceable> |
5556
TABLESPACE <replaceable class="PARAMETER">object_name</replaceable> |
5657
TEXT SEARCH CONFIGURATION <replaceable class="PARAMETER">object_name</replaceable> |
@@ -125,8 +126,8 @@ COMMENT ON
125126
The name of the object to be commented. Names of tables,
126127
aggregates, collations, conversions, domains, foreign tables, functions,
127128
indexes, operators, operator classes, operator families, sequences,
128-
text search objects, types, and views can be schema-qualified.
129-
When commenting on a column,
129+
statistics, text search objects, types, and views can be
130+
schema-qualified. When commenting on a column,
130131
<replaceable class="parameter">relation_name</replaceable> must refer
131132
to a table, view, composite type, or foreign table.
132133
</para>
@@ -327,6 +328,7 @@ COMMENT ON RULE my_rule ON my_table IS 'Logs updates of employee records';
327328
COMMENT ON SCHEMA my_schema IS 'Departmental data';
328329
COMMENT ON SEQUENCE my_sequence IS 'Used to generate primary keys';
329330
COMMENT ON SERVER myserver IS 'my foreign server';
331+
COMMENT ON STATISTICS my_statistics IS 'Improves planner row estimations';
330332
COMMENT ON TABLE my_schema.my_table IS 'Employee Information';
331333
COMMENT ON TABLESPACE my_tablespace IS 'Tablespace for indexes';
332334
COMMENT ON TEXT SEARCH CONFIGURATION my_config IS 'Special word filtering';

0 commit comments

Comments
 (0)