Postgresql function executed much longer than the same query

Question

I'm using PostgreSQL 9.2.9 and have the following problem.

There are function:

CREATE OR REPLACE FUNCTION report_children_without_place(text, date, date, integer)
RETURNS TABLE (department_name character varying, kindergarten_name character varying, a1 bigint) AS $BODY$
BEGIN
    RETURN QUERY WITH rh AS (
        SELECT (array_agg(status ORDER BY date DESC))[1] AS status, request
        FROM requeststatushistory
        WHERE date <= $3
        GROUP BY request
    )
    SELECT
        w.name,
        kgn.name,
        COUNT(*)
    FROM kindergarten_request_table_materialized kr
    JOIN rh ON rh.request = kr.id
    JOIN requeststatuses s ON s.id = rh.status AND s.sysname IN ('confirmed', 'need_meet_completion', 'kindergarten_need_meet')
    JOIN workareas kgn ON kr.kindergarten = kgn.id AND kgn.tree <@ CAST($1 AS LTREE) AND kgn.active
    JOIN organizationforms of ON of.id = kgn.organizationform AND of.sysname IN  ('state','municipal','departmental')
    JOIN workareas w ON w.tree @> kgn.tree AND w.active
    JOIN workareatypes mt ON mt.id = w.type AND mt.sysname = 'management'
    WHERE kr.requestyear = $4
    GROUP BY kgn.name, w.name
    ORDER BY w.name, kgn.name;
END
$BODY$ LANGUAGE PLPGSQL STABLE;

EXPLAIN ANALYZE SELECT * FROM report_children_without_place('83.86443.86445', '14-04-2015', '14-04-2015', 2014);

Total runtime: 242805.085 ms. But query from function's body executes much faster:

EXPLAIN ANALYZE WITH rh AS (
SELECT (array_agg(status ORDER BY date DESC))[1] AS status, request
FROM requeststatushistory
WHERE date <= '14-04-2015'
GROUP BY request
)
SELECT
    w.name,
    kgn.name,
    COUNT(*)
FROM kindergarten_request_table_materialized kr
JOIN rh ON rh.request = kr.id
JOIN requeststatuses s ON s.id = rh.status AND s.sysname IN ('confirmed', 'need_meet_completion', 'kindergarten_need_meet')
JOIN workareas kgn ON kr.kindergarten = kgn.id AND kgn.tree <@ CAST('83.86443.86445' AS LTREE) AND kgn.active
JOIN organizationforms of ON of.id = kgn.organizationform AND of.sysname IN  ('state','municipal','departmental')
JOIN workareas w ON w.tree @> kgn.tree AND w.active
JOIN workareatypes mt ON mt.id = w.type AND mt.sysname = 'management'
WHERE kr.requestyear = 2014
GROUP BY kgn.name, w.name
ORDER BY w.name, kgn.name;

Total runtime: 2156.740 ms. Why function executed so longer than the same query? Thank's

Can you show us both execution plans? (e.g. upload them to explain.depesz.com) — user330315
– user330315, Commented Apr 17, 2015 at 10:02
I'm a bit simplified queries for better readability. Explain analyze result for function: explain.depesz.com/s/AfeU and for query: explain.depesz.com/s/OKN Besides than, I'm trying explain in function: explain.depesz.com/s/jxnb — Alina Didenko
– Alina Didenko, Commented Apr 17, 2015 at 14:36
I can't find it right now but I think I once read that a query in a function is ran as a prepared query which means it's plan is made before the parameters are known which can result in bad plans (but saves the overhead of planning the query each time the function is executed). If you would execute the query as a dynamic query with EXECUTE postgresql.org/docs/9.2/static/… the plan should be made when the parameters are known. — Eelke
– Eelke, Commented May 3, 2015 at 6:58

matthew.peters · Accepted Answer · 2015-05-10 00:36:41Z

Your query runs faster because the "variables" are not actually variable -- they are static values (IE strings in quotes). This means the execution planner can leverage indexes. Within your stored procedure, your variables are actual variables, and the planner cannot make assumptions about indexes. For example - you might have a partial index on requeststatushistory where "date" is <= '2012-12-31'. The index can only be used if the $3 is known. Since it might hold a date from 2015, the partial index would be of no use. In fact, it would be detrimental.

I frequently construct a string within my functions where I concatenate my variables as literals and then execute the function using something like the following:

DECLARE
    my_dynamic_sql TEXT;
BEGIN
    my_dynamic_sql := $$
        SELECT * 
        FROM my_table 
        WHERE $$ || quote_literal($3) || $$::TIMESTAMPTZ BETWEEN start_time
                                                             AND end_time;$$;

    /* You can only see this if client_min_messages = DEBUG */
    RAISE DEBUG '%', my_dynamic_sql; 
    RETURN QUERY EXECUTE my_dynamic_sql;
END;

The dynamic SQL is VERY useful because you can actually get an explain of the query when I have set client_min_messages=DEBUG; I can scrape the query from the screen and paste it back in after EXPLAIN or EXPLAIN ANALYZE and see what the execution planner is doing. This also allows you to construct very different queries as needed to optimize for variables (IE exclude unnecessary tables if warranted) and maintain a common API for your clients.

You may be tempted to avoid the dynamic SQL for fear of performance issues (I was at first) but you will be amazed at how LITTLE time is spent in planning compared to some of the cost of a couple of table scans on your seven-table join!

Good luck!

Follow-up: You might experiment with Common Table Expressions (CTEs) for performance as well. If you have a table that has a low signal-to-noise ratio (has many, many more records in it than you actually want to return) then a CTE can be very helpful. PostgreSQL executes CTEs early in the query, and materializes the resulting rows in memory. This allows you to use the same result set multiple times and in multiple places in your query. The benefit can really be surprising if you design it correctly.

sql_txt := $$
WITH my_cte as (
   select fk1 as moar_data 1
        , field1
        , field2 /*do not need all other fields taking up RAM!*/
   from my_table
   where field3 between $$ || quote_literal(input_start_ts) || $$::timestamptz
                    and $$ || quote_literal(input_end_ts) || $$::timestamptz
                ),
      keys_cte as ( select key_field
                    from big_look_up_table
                    where look_up_name = ANY($$ || 
                         QUOTE_LITERAL(input_array_of_names) || $$::VARCHAR[])
                  )
SELECT field1, field2, moar_data1, moar_data2
FROM moar_data_table
INNER JOIN my_cte
  USING (moar_data1)
WHERE moar_data_table.moar_data_key in (select key_field from keys_cte) $$;

An execution plan is likely to show that it chooses to use an index on moar_data_tale.moar_data_key. This would appear to go against what I said above in my prior answer - except for the fact that the keys_cte results are materialized (and therefore cannot be changed by another transaction in a race-condition) -- you have your own little copy of the data for use in this query.

Oh - and CTEs can use other CTEs that are declared earlier in the same query. I have used this "trick" to replace sub-queries in very complex joins and seen great improvements.

Happy Hacking!

Collectives™ on Stack Overflow

Postgresql function executed much longer than the same query

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related