Skip to content

Commit dc68668

Browse files
michaelpqmichaelpq
committed
Introduce log_destination=jsonlog
"jsonlog" is a new value that can be added to log_destination to provide logs in the JSON format, with its output written to a file, making it the third type of destination of this kind, after "stderr" and "csvlog". The format is convenient to feed logs to other applications. There is also a plugin external to core that provided this feature using the hook in elog.c, but this had to overwrite the output of "stderr" to work, so being able to do both at the same time was not possible. The files generated by this log format are suffixed with ".json", and use the same rotation policies as the other two formats depending on the backend configuration. This takes advantage of the refactoring work done previously in ac7c807, bed6ed3, 8b76f89 and 2d77d83 for the backend parts, and 72b76f7 for the TAP tests, making the addition of any new file-based format rather straight-forward. The documentation is updated to list all the keys and the values that can exist in this new format. pg_current_logfile() also required a refresh for the new option. Author: Sehrope Sarkuni, Michael Paquier Reviewed-by: Nathan Bossart, Justin Pryzby Discussion: https://postgr.es/m/CAH7T-aqswBM6JWe4pDehi1uOiufqe06DJWaU5=X7dDLyqUExHg@mail.gmail.com
1 parent 6478896 commit dc68668

File tree

12 files changed

+660
-48
lines changed

12 files changed

+660
-48
lines changed

‎doc/src/sgml/config.sgml‎

Lines changed: 211 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5931,7 +5931,8 @@ SELECT * FROM parent WHERE key = 2400;
59315931
<para>
59325932
<productname>PostgreSQL</productname> supports several methods
59335933
for logging server messages, including
5934-
<systemitem>stderr</systemitem>, <systemitem>csvlog</systemitem> and
5934+
<systemitem>stderr</systemitem>, <systemitem>csvlog</systemitem>,
5935+
<systemitem>jsonlog</systemitem>, and
59355936
<systemitem>syslog</systemitem>. On Windows,
59365937
<systemitem>eventlog</systemitem> is also supported. Set this
59375938
parameter to a list of desired log destinations separated by
@@ -5950,25 +5951,35 @@ SELECT * FROM parent WHERE key = 2400;
59505951
CSV-format log output.
59515952
</para>
59525953
<para>
5953-
When either <systemitem>stderr</systemitem> or
5954-
<systemitem>csvlog</systemitem> are included, the file
5955-
<filename>current_logfiles</filename> is created to record the location
5956-
of the log file(s) currently in use by the logging collector and the
5957-
associated logging destination. This provides a convenient way to
5958-
find the logs currently in use by the instance. Here is an example of
5959-
this file's content:
5954+
If <systemitem>jsonlog</systemitem> is included in
5955+
<varname>log_destination</varname>, log entries are output in
5956+
<acronym>JSON</acronym> format, which is convenient for loading logs
5957+
into programs.
5958+
See <xref linkend="runtime-config-logging-jsonlog"/> for details.
5959+
<xref linkend="guc-logging-collector"/> must be enabled to generate
5960+
JSON-format log output.
5961+
</para>
5962+
<para>
5963+
When either <systemitem>stderr</systemitem>,
5964+
<systemitem>csvlog</systemitem> or <systemitem>jsonlog</systemitem> are
5965+
included, the file <filename>current_logfiles</filename> is created to
5966+
record the location of the log file(s) currently in use by the logging
5967+
collector and the associated logging destination. This provides a
5968+
convenient way to find the logs currently in use by the instance. Here
5969+
is an example of this file's content:
59605970
<programlisting>
59615971
stderr log/postgresql.log
59625972
csvlog log/postgresql.csv
5973+
jsonlog log/postgresql.json
59635974
</programlisting>
59645975

59655976
<filename>current_logfiles</filename> is recreated when a new log file
59665977
is created as an effect of rotation, and
59675978
when <varname>log_destination</varname> is reloaded. It is removed when
5968-
neither <systemitem>stderr</systemitem>
5969-
nor <systemitem>csvlog</systemitem> are included
5970-
in <varname>log_destination</varname>, and when the logging collector is
5971-
disabled.
5979+
none of <systemitem>stderr</systemitem>,
5980+
<systemitem>csvlog</systemitem> or <systemitem>jsonlog</systemitem> are
5981+
included in <varname>log_destination</varname>, and when the logging
5982+
collector is disabled.
59725983
</para>
59735984

59745985
<note>
@@ -6106,6 +6117,13 @@ local0.* /var/log/postgresql
61066117
(If <varname>log_filename</varname> ends in <literal>.log</literal>, the suffix is
61076118
replaced instead.)
61086119
</para>
6120+
<para>
6121+
If JSON-format output is enabled in <varname>log_destination</varname>,
6122+
<literal>.json</literal> will be appended to the timestamped
6123+
log file name to create the file name for JSON-format output.
6124+
(If <varname>log_filename</varname> ends in <literal>.log</literal>, the suffix is
6125+
replaced instead.)
6126+
</para>
61096127
<para>
61106128
This parameter can only be set in the <filename>postgresql.conf</filename>
61116129
file or on the server command line.
@@ -7467,6 +7485,187 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
74677485
</orderedlist>
74687486
</para>
74697487
</sect2>
7488+
<sect2 id="runtime-config-logging-jsonlog">
7489+
<title>Using JSON-Format Log Output</title>
7490+
7491+
<para>
7492+
Including <literal>jsonlog</literal> in the
7493+
<varname>log_destination</varname> list provides a convenient way to
7494+
import log files into many different programs. This option emits log
7495+
lines in (<acronym>JSON</acronym>) format.
7496+
</para>
7497+
7498+
<para>
7499+
String fields with null values are excluded from output.
7500+
Additional fields may be added in the future. User applications that
7501+
process <literal>jsonlog</literal> output should ignore unknown fields.
7502+
</para>
7503+
7504+
<para>
7505+
Each log line is serialized as a JSON object as of the following
7506+
set of keys with their values.
7507+
</para>
7508+
7509+
<table>
7510+
<title>Keys and values of JSON log entries</title>
7511+
<tgroup cols="3">
7512+
<thead>
7513+
<row>
7514+
<entry>Key name</entry>
7515+
<entry>Type</entry>
7516+
<entry>Description</entry>
7517+
</row>
7518+
</thead>
7519+
<tbody>
7520+
<row>
7521+
<entry><literal>timestamp</literal></entry>
7522+
<entry>string</entry>
7523+
<entry>Time stamp with milliseconds</entry>
7524+
</row>
7525+
<row>
7526+
<entry><literal>user</literal></entry>
7527+
<entry>string</entry>
7528+
<entry>User name</entry>
7529+
</row>
7530+
<row>
7531+
<entry><literal>dbname</literal></entry>
7532+
<entry>string</entry>
7533+
<entry>Database name</entry>
7534+
</row>
7535+
<row>
7536+
<entry><literal>pid</literal></entry>
7537+
<entry>number</entry>
7538+
<entry>Process ID</entry>
7539+
</row>
7540+
<row>
7541+
<entry><literal>remote_host</literal></entry>
7542+
<entry>string</entry>
7543+
<entry>Client host</entry>
7544+
</row>
7545+
<row>
7546+
<entry><literal>remote_port</literal></entry>
7547+
<entry>number</entry>
7548+
<entry>Client port</entry>
7549+
</row>
7550+
<row>
7551+
<entry><literal>session_id</literal></entry>
7552+
<entry>string</entry>
7553+
<entry>Session ID</entry>
7554+
</row>
7555+
<row>
7556+
<entry><literal>line_num</literal></entry>
7557+
<entry>number</entry>
7558+
<entry>Per-session line number</entry>
7559+
</row>
7560+
<row>
7561+
<entry><literal>ps</literal></entry>
7562+
<entry>string</entry>
7563+
<entry>Current ps display</entry>
7564+
</row>
7565+
<row>
7566+
<entry><literal>session_start</literal></entry>
7567+
<entry>string</entry>
7568+
<entry>Session start time</entry>
7569+
</row>
7570+
<row>
7571+
<entry><literal>vxid</literal></entry>
7572+
<entry>string</entry>
7573+
<entry>Virtual transaction ID</entry>
7574+
</row>
7575+
<row>
7576+
<entry><literal>txid</literal></entry>
7577+
<entry>string</entry>
7578+
<entry>Regular transaction ID</entry>
7579+
</row>
7580+
<row>
7581+
<entry><literal>error_severity</literal></entry>
7582+
<entry>string</entry>
7583+
<entry>Error severity</entry>
7584+
</row>
7585+
<row>
7586+
<entry><literal>state_code</literal></entry>
7587+
<entry>string</entry>
7588+
<entry>SQLSTATE code</entry>
7589+
</row>
7590+
<row>
7591+
<entry><literal>message</literal></entry>
7592+
<entry>string</entry>
7593+
<entry>Error message</entry>
7594+
</row>
7595+
<row>
7596+
<entry><literal>detail</literal></entry>
7597+
<entry>string</entry>
7598+
<entry>Error message detail</entry>
7599+
</row>
7600+
<row>
7601+
<entry><literal>hint</literal></entry>
7602+
<entry>string</entry>
7603+
<entry>Error message hint</entry>
7604+
</row>
7605+
<row>
7606+
<entry><literal>internal_query</literal></entry>
7607+
<entry>string</entry>
7608+
<entry>Internal query that led to the error</entry>
7609+
</row>
7610+
<row>
7611+
<entry><literal>internal_position</literal></entry>
7612+
<entry>number</entry>
7613+
<entry>Cursor index into internal query</entry>
7614+
</row>
7615+
<row>
7616+
<entry><literal>context</literal></entry>
7617+
<entry>string</entry>
7618+
<entry>Error context</entry>
7619+
</row>
7620+
<row>
7621+
<entry><literal>statement</literal></entry>
7622+
<entry>string</entry>
7623+
<entry>Client-supplied query string</entry>
7624+
</row>
7625+
<row>
7626+
<entry><literal>cursor_position</literal></entry>
7627+
<entry>string</entry>
7628+
<entry>Cursor index into query string</entry>
7629+
</row>
7630+
<row>
7631+
<entry><literal>func_name</literal></entry>
7632+
<entry>string</entry>
7633+
<entry>Error location function name</entry>
7634+
</row>
7635+
<row>
7636+
<entry><literal>file_name</literal></entry>
7637+
<entry>string</entry>
7638+
<entry>File name of error location</entry>
7639+
</row>
7640+
<row>
7641+
<entry><literal>file_line_num</literal></entry>
7642+
<entry>number</entry>
7643+
<entry>File line number of the error location</entry>
7644+
</row>
7645+
<row>
7646+
<entry><literal>application_name</literal></entry>
7647+
<entry>string</entry>
7648+
<entry>Client application name</entry>
7649+
</row>
7650+
<row>
7651+
<entry><literal>backend_type</literal></entry>
7652+
<entry>string</entry>
7653+
<entry>Type of backend</entry>
7654+
</row>
7655+
<row>
7656+
<entry><literal>leader_pid</literal></entry>
7657+
<entry>number</entry>
7658+
<entry>Process ID of leader for active parallel workers</entry>
7659+
</row>
7660+
<row>
7661+
<entry><literal>query_id</literal></entry>
7662+
<entry>number</entry>
7663+
<entry>Query ID</entry>
7664+
</row>
7665+
</tbody>
7666+
</tgroup>
7667+
</table>
7668+
</sect2>
74707669

74717670
<sect2>
74727671
<title>Process Title</title>

‎doc/src/sgml/func.sgml‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22446,10 +22446,12 @@ SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n);
2244622446
format, <function>pg_current_logfile</function> without an argument
2244722447
returns the path of the file having the first format found in the
2244822448
ordered list: <literal>stderr</literal>,
22449-
<literal>csvlog</literal>. <literal>NULL</literal> is returned
22450-
if no log file has any of these formats.
22449+
<literal>csvlog</literal>, <literal>jsonlog</literal>.
22450+
<literal>NULL</literal> is returned if no log file has any of these
22451+
formats.
2245122452
To request information about a specific log file format, supply
22452-
either <literal>csvlog</literal> or <literal>stderr</literal> as the
22453+
either <literal>csvlog</literal>, <literal>jsonlog</literal> or
22454+
<literal>stderr</literal> as the
2245322455
value of the optional parameter. The result is <literal>NULL</literal>
2245422456
if the log format requested is not configured in
2245522457
<xref linkend="guc-log-destination"/>.

0 commit comments

Comments
 (0)