Combining two Postgresql queries

Question

I have this query that returns how many days a driver worked based on driver number. The way I check for days worked is just to count distinct order_date with their driver number on it. Let's call it days worked:

SELECT driver_no, count(distinct order_date)

FROM orders o INNER JOIN order_settlements os ON o.control_no = 
os.control_no 

WHERE os.company_no = '001' and o.service_type not in (17, 30, 31, 34, 35, 
90, 94, 96, 97, 98, 99) and customer_reference != 'PARCEL ADJUSTMENT' and 
order_date between (date '2017-6-11' - integer '7') and '2017-6-11' and 
posting_status <> '9' and settlement_period_end_date is null

GROUP BY driver_no

And I have this query that calculates how much the driver earned, how much he delivered, etc. Let's call it main:

    SELECT Driver_Number, Driver_Name, Branch, Driver_Type, sum(Revenue) AS Revenue, sum(Booking) as Booking, CASE WHEN round(sum(Support_Pay * Settlement_Per/100), 2) != 0 THEN round(sum(Support_Pay * Settlement_Per/100), 2) END as Support_Pay, round(sum(fuel * Settlement_Per/100), 2) as Fuel, round(sum(Booking * Settlement_Per/100), 2) as Settlement, sum(Stops) As Stops, sum(Pieces) As Pieces

    FROM 
    (  SELECT os.driver_no as Driver_Number, d.driver_name as Driver_Name, d.report_sort_key as Branch, (CASE WHEN d.driver_type = '0' THEN 'Contractor' WHEN d.driver_type = '1' THEN 'Employee' END) as Driver_Type,
      sum(o.rate_bucket1+o.rate_bucket2+o.rate_bucket3+o.rate_bucket4+o.rate_bucket5+o.rate_bucket6+ 
      o.rate_bucket7+o.rate_bucket8+o.rate_bucket9+o.rate_bucket10+o.rate_bucket11) as Revenue,
    sum(os.charge1+os.charge2+os.charge3+os.charge4+os.charge5+os.charge6) as Booking, CASE WHEN (o.service_type = '35') THEN sum(os.charge1+os.charge2+os.charge3+os.charge4+os.charge5+os.charge6) END AS Support_Pay, CASE WHEN (o.service_type = '34') THEN sum(os.charge1+os.charge2+os.charge3+os.charge4+os.charge5+os.charge6) END AS Fuel,
    os.settlement_percent as Settlement_Per, CASE WHEN (o.service_type != '17' or o.service_type != '30' or o.service_type != '31' or o.service_type != '34' or o.service_type != '35' or o.service_type != '90' or o.service_type != '94' or o.service_type != '96' or o.service_type != '97' or o.service_type != '98' or o.service_type != '99') THEN count(os.control_no) END as Stops, CASE WHEN (o.service_type != '17' or o.service_type != '30' or o.service_type != '31' or o.service_type != '34' or o.service_type != '35' or o.service_type != '90' or o.service_type != '94' or o.service_type != '96' or o.service_type != '97' or o.service_type != '98' or o.service_type != '99') THEN sum(o.pieces) END as Pieces


    FROM 
    orders o INNER JOIN order_settlements os ON o.control_no = os.control_no INNER JOIN drivers d ON os.driver_no = d.driver_no

    WHERE 
    d.company_no = '001' and 
    order_date BETWEEN '2017-4-9' AND '2017-6-11' AND
      os.company_no = '001' and o.company_no = '001' AND posting_status <> '9' AND
    settlement_period_end_date is NULL AND os.driver_no is not null and os.driver_no !=0 and d.driver_no between '1' and '7999'

    GROUP BY
    o.service_type, order_date, Settlement_Per, o.customer_no, os.driver_no, d.driver_name, d.driver_type, d.report_sort_key) Sub

    GROUP BY
    Branch, Driver_Number, Driver_Name, Driver_Type
    ORDER BY
    Driver_Number

Now I want to put the days worked as a column in the main query, but I need to keep the date ranges the same (days worked should only look back at the last week, whereas the main query needs to look back for a couple months for any orders that got entered retroactively).

I tried putting the days worked query into a "CASE WHEN" statement at the end of the main query, and it returned incorrect data (I think it was excluding any days where those service types registered instead of just passing over them). I tried putting an inner select statement at the end of the Main query where it would match driver_no from days_worked to main query and it took forever to run. I tried creating two different queries in the main "FROM" statement, one looking at the Sub and one looking at days_worked, and it returned way too many records and not how I wanted at all.

What's the best way to get this information returned into one query? This is on Postgresql 8.1 by the way. Thanks for all your help.

Elliot B. · Accepted Answer · 2017-06-16 19:43:37Z

I would advise against combining both of those large queries into one mega query. It'll unnecessarily add complexity and make debugging more difficult.

I'd suggest that you put your two queries into stored procedures. You can query other stored procedures from inside a stored procedure. This will allow you to use the output from your larger query in your smaller query without the complexity of mashing the queries together.

Without knowing the data types of your fields, it's not possible to give a full working example, but here is a rough approximation of how you would setup the two stored procedures:

CREATE OR REPLACE FUNCTION get_driver_stats (
    _company_no bigint
    ,_start_date text
    ,_end_date text
    ,_posting_status bigint
) RETURNS RECORD (
    Driver_Number
    ,Driver_Name text
    ,Branch text
    ,Driver_Type text
    ,Revenue text
    ,Booking text
    ,Support_Pay text
    ,Fuel text
    ,Settlement text
    ,Stops text
    ,Pieces text
) AS $$
DECLARE vout RECORD;
BEGIN


    SELECT driver_number, 
           driver_name, 
           branch, 
           driver_type, 
           Sum(revenue)                                  AS Revenue, 
           Sum(booking)                                  AS Booking, 
           CASE 
             WHEN Round(Sum(support_pay * settlement_per / 100), 2) != 0 THEN 
             Round(Sum(support_pay * settlement_per / 100), 2) 
           END                                           AS Support_Pay, 
           Round(Sum(fuel * settlement_per / 100), 2)    AS Fuel, 
           Round(Sum(booking * settlement_per / 100), 2) AS Settlement, 
           Sum(stops)                                    AS Stops, 
           Sum(pieces)                                   AS Pieces 
    FROM   (SELECT os.driver_no                   AS Driver_Number, 
                   d.driver_name                  AS Driver_Name, 
                   d.report_sort_key              AS Branch, 
                   ( CASE 
                       WHEN d.driver_type = '0' THEN 'Contractor' 
                       WHEN d.driver_type = '1' THEN 'Employee' 
                     END )                        AS Driver_Type, 
                   Sum(o.rate_bucket1 + o.rate_bucket2 
                       + o.rate_bucket3 + o.rate_bucket4 
                       + o.rate_bucket5 + o.rate_bucket6 
                       + o.rate_bucket7 + o.rate_bucket8 
                       + o.rate_bucket9 + o.rate_bucket10 
                       + o.rate_bucket11)         AS Revenue, 
                   Sum(os.charge1 + os.charge2 + os.charge3 + os.charge4 
                       + os.charge5 + os.charge6) AS Booking, 
                   CASE 
                     WHEN ( o.service_type = '35' ) THEN Sum( 
                     os.charge1 + os.charge2 + os.charge3 + os.charge4 
                     + os.charge5 + os.charge6) 
                   END                            AS Support_Pay, 
                   CASE 
                     WHEN ( o.service_type = '34' ) THEN Sum( 
                     os.charge1 + os.charge2 + os.charge3 + os.charge4 
                     + os.charge5 + os.charge6) 
                   END                            AS Fuel, 
                   os.settlement_percent          AS Settlement_Per, 
                   CASE 
                     WHEN ( o.service_type != '17' 
                             OR o.service_type != '30' 
                             OR o.service_type != '31' 
                             OR o.service_type != '34' 
                             OR o.service_type != '35' 
                             OR o.service_type != '90' 
                             OR o.service_type != '94' 
                             OR o.service_type != '96' 
                             OR o.service_type != '97' 
                             OR o.service_type != '98' 
                             OR o.service_type != '99' ) THEN Count(os.control_no) 
                   END                            AS Stops, 
                   CASE 
                     WHEN ( o.service_type != '17' 
                             OR o.service_type != '30' 
                             OR o.service_type != '31' 
                             OR o.service_type != '34' 
                             OR o.service_type != '35' 
                             OR o.service_type != '90' 
                             OR o.service_type != '94' 
                             OR o.service_type != '96' 
                             OR o.service_type != '97' 
                             OR o.service_type != '98' 
                             OR o.service_type != '99' ) THEN Sum(o.pieces) 
                   END                            AS Pieces 
            FROM   orders o 
                   INNER JOIN order_settlements os 
                           ON o.control_no = os.control_no 
                   INNER JOIN drivers d 
                           ON os.driver_no = d.driver_no 
            WHERE  d.company_no = _company_no 
                   AND order_date BETWEEN _start_date AND _end_date 
                   AND os.company_no = _company_no 
                   AND o.company_no = _company_no 
                   AND posting_status <> _posting_status 
                   AND settlement_period_end_date IS NULL 
                   AND os.driver_no IS NOT NULL 
                   AND os.driver_no != 0 
                   AND d.driver_no BETWEEN '1' AND '7999' 
            GROUP  BY o.service_type, 
                      order_date, 
                      settlement_per, 
                      o.customer_no, 
                      os.driver_no, 
                      d.driver_name, 
                      d.driver_type, 
                      d.report_sort_key) Sub 
    GROUP  BY branch, 
              driver_number, 
              driver_name, 
              driver_type 
    ORDER  BY driver_number 
    INTO vout;

    RETURN vout;


END; $$ LANGUAGE plpgsql;


CREATE OR REPLACE FUNCTION get_driver_report (
    _company_no bigint
) RETURNS TABLE (
    driver_no text
) AS $$
    DECLARE vout record;
BEGIN

    RETURN QUERY
    SELECT * 
    FROM   get_driver_stats(_company_no) 
    INTO   vout;SELECT     driver_no, 
               Count(DISTINCT order_date), 
               vout.revenue AS revenue 
    FROM       orders o 
    INNER JOIN order_settlements os 
    ON         o.control_no = os.control_no 
    WHERE      os.company_no = '001' 
    AND        o.service_type NOT IN (17,30, 31, 34, 35, 90, 94, 96, 97, 98, 99) 
    AND        customer_reference != 'PARCEL ADJUSTMENT' 
    AND        order_date BETWEEN (date '2017-6-11' - integer '7') AND        '2017-6-11' 
    AND        posting_status <> '9' 
    AND        settlement_period_end_date IS NULL 
    GROUP BY   driver_no;

END; $$ LANGUAGE plpgsql;

The key here is that you can store the output of your larger query (now in the stored procedure get_driver_stats) into the variable vout and use it to complete the your smaller query (now in the stored procedure get_driver_report).

In my example above, I am assume that get_driver_stats returns a single record. This may or may not be accurate.

Dan Getz · Accepted Answer · 2017-06-16 19:24:38Z

The queries are way to big. There must be some ways to split them up into smaller subqueries. At least formatting them better would help. The following might work (with no data or working example this is, of course, hard to test):

SELECT a.*, b.days_worked FROM (
    SELECT driver_number, 
        driver_name, 
        branch, 
        driver_type, 
        SUM(revenue)                                  AS Revenue, 
        SUM(booking)                                  AS Booking, 
        CASE 
            WHEN Round(SUM(support_pay * settlement_per / 100), 2) != 0 THEN 
            Round(SUM(support_pay * settlement_per / 100), 2) 
        END                                           AS Support_Pay, 
        Round(SUM(fuel * settlement_per / 100), 2)    AS Fuel, 
        Round(SUM(booking * settlement_per / 100), 2) AS Settlement, 
        SUM(stops)                                    AS Stops, 
        SUM(pieces)                                   AS Pieces
    FROM   (SELECT os.driver_no                   AS Driver_Number, 
                d.driver_name                  AS Driver_Name, 
                d.report_sort_key              AS Branch, 
                ( CASE 
                    WHEN d.driver_type = '0' THEN 'Contractor' 
                    WHEN d.driver_type = '1' THEN 'Employee' 
                    END )                        AS Driver_Type, 
                SUM(o.rate_bucket1 + o.rate_bucket2 
                    + o.rate_bucket3 + o.rate_bucket4 
                    + o.rate_bucket5 + o.rate_bucket6 
                    + o.rate_bucket7 + o.rate_bucket8 
                    + o.rate_bucket9 + o.rate_bucket10 
                    + o.rate_bucket11)         AS Revenue, 
                SUM(os.charge1 + os.charge2 + os.charge3 + os.charge4 
                    + os.charge5 + os.charge6) AS Booking, 
                CASE 
                    WHEN ( o.service_type = '35' ) THEN SUM( 
                    os.charge1 + os.charge2 + os.charge3 + os.charge4 
                    + os.charge5 + os.charge6) 
                END                            AS Support_Pay, 
                CASE 
                    WHEN ( o.service_type = '34' ) THEN SUM( 
                    os.charge1 + os.charge2 + os.charge3 + os.charge4 
                    + os.charge5 + os.charge6) 
                END                            AS Fuel, 
                os.settlement_percent          AS Settlement_Per, 
                CASE 
                    WHEN ( o.service_type != '17' 
                            OR o.service_type != '30' 
                            OR o.service_type != '31' 
                            OR o.service_type != '34' 
                            OR o.service_type != '35' 
                            OR o.service_type != '90' 
                            OR o.service_type != '94' 
                            OR o.service_type != '96' 
                            OR o.service_type != '97' 
                            OR o.service_type != '98' 
                            OR o.service_type != '99' ) THEN Count(os.control_no) 
                END                            AS Stops, 
                CASE 
                    WHEN ( o.service_type != '17' 
                            OR o.service_type != '30' 
                            OR o.service_type != '31' 
                            OR o.service_type != '34' 
                            OR o.service_type != '35' 
                            OR o.service_type != '90' 
                            OR o.service_type != '94' 
                            OR o.service_type != '96' 
                            OR o.service_type != '97' 
                            OR o.service_type != '98' 
                            OR o.service_type != '99' ) THEN SUM(o.pieces) 
                END                            AS Pieces 
            FROM   orders o 
                inner join order_settlements os 
                        ON o.control_no = os.control_no 
                inner join drivers d 
                        ON os.driver_no = d.driver_no 
            WHERE  d.company_no = '001' 
                AND order_date BETWEEN '2017-4-9' AND '2017-6-11' 
                AND os.company_no = '001' 
                AND o.company_no = '001' 
                AND posting_status <> '9' 
                AND settlement_period_end_date IS NULL 
                AND os.driver_no IS NOT NULL 
                AND os.driver_no != 0 
                AND d.driver_no BETWEEN '1' AND '7999' 
            GROUP  BY o.service_type, 
                    order_date, 
                    settlement_per, 
                    o.customer_no, 
                    os.driver_no, 
                    d.driver_name, 
                    d.driver_type, 
                    d.report_sort_key) Sub 
    GROUP  BY branch, 
            driver_number, 
            driver_name, 
            driver_type 
    ORDER  BY driver_number 
) a JOIN (
    SELECT     driver_no, 
            Count(DISTINCT order_date) as days_worked 
    FROM       orders o 
    inner join order_settlements os 
    ON         o.control_no = os.control_no 
    WHERE      os.company_no = '001' 
    AND        o.service_type NOT IN (17, 
                                    30, 
                                    31, 
                                    34, 
                                    35, 
                                    90, 
                                    94, 
                                    96, 
                                    97, 
                                    98, 
                                    99) 
    AND        customer_reference != 'PARCEL ADJUSTMENT' 
    AND        order_date BETWEEN (DATE '2017-6-11' - INTEGER '7') AND        '2017-6-11' 
    AND        posting_status <> '9' 
    AND        settlement_period_end_date IS NULL 
    GROUP BY   driver_no
) b ON (a.driver_number = b.driver_no) ;

That worked! You guys are amazing. Perhaps you're right that the queries are too large, but I'm just trying to get it all the data returned into one place. It only ends up being 11 columns, it's just a complicated way of getting to them.
@Emac BTW: The bunch of conditions o.service_type != '17' OR o.service_type != '30' ... could be replaced by o.service_type != any(array[17,30,31,34,35,90,94,96,97,98,99]) (but check the performance)
Thanks for the tip Abelisto! I actually never thought of putting them in an array. In my "Days Worked" query, I actually use the "o.service_type not in (17,30,31,34,35,90,94,96,97,98,99)", but when I wrote the first query, I didn't know about that and listed every type in a large, redundant "or" statement. So at least I made some improvements since the first query.
@Emac I hope that you know that o.service_type not in (17,30) is not equal to o.service_type != '17' OR o.service_type != '30' :o) It is equal to o.service_type != '17' AND o.service_type != '30'

Collectives™ on Stack Overflow

Combining two Postgresql queries

2 Answers 2

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related