2

This is simplified version of my function with the query in it (therefore any variables are now useless) and this function won't finished but if I run the same query separately it finishes in under a second.

function that never finishes

select * from test_function_difference(1);


CREATE OR REPLACE FUNCTION test_function_difference ( 
  p_does_nothing int

)  
RETURNS TABLE(
  t_datum date,
  t_capacity numeric,
  t_used numeric,
  t_category int,
  t_category_name text,  
  t_used_p numeric,
  t_unused_p numeric
)
  VOLATILE
AS $dbvis$

declare
p_sql text := '';
p_execute text := '';
rec record;
begin

p_sql := 
'
 with 
vytizeni as (
  select 
    date_trunc(''day'',mcz.datum)::date as datum ,  
    sum(zd.v_vytizeni)/3600.0 used
  from v_ui_cdc_s5_misto_cas_zdroj_aggregace mcz
  left join (select * , pul_den as den_noc from v_ui_cdc_s5_misto_cas_zdroj_aggregace_zdrobneni) zd on mcz.id = zd.id
  where
    datum between  ''2018-12-31'' and ''2018-12-31''

    and ( zahranicni = 0 or zahranicni is null )
     and den_noc = -1 
  group by 
    date_trunc(''day'',mcz.datum)::date
)
,kapacita as (
  select
    date_trunc(''day'',datum)::date as datum , 
    sum(obsazeni_g)/3600.0 capacity
  from v_ui_cdc_s5_misto_cas_zdroj_aggregace
  where
    datum between  ''2018-12-31'' and ''2018-12-31''

  group by
    date_trunc(''day'',datum)::date 
)
,zdroj as (
  select 
    k.datum,  
    k.capacity,
    v.used,
    -1 category
  from kapacita k 
  join vytizeni v on k.datum = v.datum
)

select
  c.* , 
  kc.nazev::text categeroy_name,
  case when sum(capacity)over(partition by datum) = 0 then 1 else used/sum(capacity)over(partition by datum) end as used_p,
  greatest(1 - case when sum(capacity)over(partition by datum) = 0 then 1 else sum(used)over(partition by datum)/sum(capacity)over(partition by datum) end,0) as unused_p
from  zdroj  c
left join v_ui_cdc_s5_kategorie_cinnosti kc on kc.id = c.category
order by c.datum
';

raise notice '% ' , p_sql;

RETURN QUERY 
execute p_sql;

END;
$dbvis$ LANGUAGE plpgsql

and the query I run separately (finished in 533 ms)

with 
vytizeni as (
  select 
    date_trunc('day',mcz.datum)::date as datum ,  
    sum(zd.v_vytizeni)/3600.0 used
  from v_ui_cdc_s5_misto_cas_zdroj_aggregace mcz
  left join (select * , pul_den as den_noc from v_ui_cdc_s5_misto_cas_zdroj_aggregace_zdrobneni) zd on mcz.id = zd.id
  where
    datum between  '2018-12-31' and '2018-12-31'

    and ( zahranicni = 0 or zahranicni is null )
     and den_noc = -1 
  group by 
    date_trunc('day',mcz.datum)::date
)
,kapacita as (
  select
    date_trunc('day',datum)::date as datum , 
    sum(obsazeni_g)/3600.0 capacity
  from v_ui_cdc_s5_misto_cas_zdroj_aggregace
  where
    datum between  '2018-12-31' and '2018-12-31'

  group by
    date_trunc('day',datum)::date 
)
,zdroj as (
  select 
    k.datum,  
    k.capacity,
    v.used,
    -1 category
  from kapacita k 
  join vytizeni v on k.datum = v.datum
)

select
  c.* , 
  kc.nazev::text categeroy_name,
  case when sum(capacity)over(partition by datum) = 0 then 1 else used/sum(capacity)over(partition by datum) end as used_p,
  greatest(1 - case when sum(capacity)over(partition by datum) = 0 then 1 else sum(used)over(partition by datum)/sum(capacity)over(partition by datum) end,0) as unused_p
from  zdroj  c
left join v_ui_cdc_s5_kategorie_cinnosti kc on kc.id = c.category
order by c.datum

Edit: I was able to get results from the function after almost 28 minutes (also I tried it on Sunday night, means I had the resources of the whole server, because during normal load function have not finished even after one hour) and right after that I ran query standalone and get results after 2.1 sec Here are the explain analyze

function: 28 minutes https://explain.depesz.com/s/v9xJ

standalone query: 2.1 sec https://explain.depesz.com/s/aBri

second run stand alone 430ms https://explain.depesz.com/s/ENva

Interesting note: if I edit start date for the interval to '2018-12-30' or any other date the function finishes as well

which means

start date = '2018-12-31'
query => finishes under 1 second
function => won't finish
start date = '2018-12-30'
query => finishes under 1 second
function => finishes under 1 second

Version details: PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit

15
  • From the execution plans you originally posted, it vas clear that the estimates for the number of rows was much higher with 2018-12-30. Give it some time - if you can produce EXPLAIN (ANALYZE, BUFFERS) output, it will be easier to analyze the problem. Commented Mar 1, 2019 at 14:10
  • I can give you explain analyze for the query run separately but I can't do it for the function since it doesn't finish (I stopped it after 45 minutes of running) Commented Mar 1, 2019 at 14:35
  • explain analyse if query run separately explain.depesz.com/s/Zji6 Commented Mar 1, 2019 at 14:46
  • Then the plain EXPLAIN for the long running query will be enough. Please put that stuff into the question, not a comment. Commented Mar 1, 2019 at 14:50
  • @LaurenzAlbe do you know if there is a system table with stored plans that was actually executed for the question, I am bit suspicious that the execution plan I gave you earlier may not be correct since it look exactly the same as the of from separate query, I just put into the function 'EXPLAIN VERBOSE ' and stored to variable and printed Commented Mar 1, 2019 at 15:06

1 Answer 1

2

The reason for the difference in the performance is that the execution inside the function does not use parallel queries, and parallel execution by chance picks a better execution plan.

Is the function not marked as parallel safe? That may make all the difference.

The core problem, however, is the gross mies-estimate of the number of result rows for the scan on ui_cdc_s5_misto_cas_zdroj_aggregace, which estimates 1 instead of 2243 rows.

You should ANALYZE that table to get a better estimate. If that alone doesn't improve the estimate, try to raise default_statistics_target before the ANALYZE.

If you need to raise default_statistics_target for a better estimate, persist that change with

ALTER TABLE ui_cdc_s5_misto_cas_zdroj_aggregace
   ALTER datum SET STATISTICS <whatever proved useful>
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you very much for the analysis 1)function was exactly as I provided in topic, I tried it to modify it as parallel safe, but I found only a very little in documentation about it, I put PARALLEL SAFE behind VOLATILE in my function which compiler accepted but it made no difference for running the function(log showed no parallel use and function was also not finishing), therefore I am not sure whether I used parallel safe correctly 2) as for the analyse I will try to do it if parallel safe won't help
You did the right thing. Not sure if it makes a difference.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.