parse nested function to extract each inner function in python

Question

I have a nested expression as below

 expression = 'position(\'a\' IN Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ))'

I want the output as by retreiving nested function first and then outer functions

['Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" )','position(\'a\' IN Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ))']

Below is the code I have tried

 result = []
 for i in range(len(expression)):
     if expression[i]=="(":
        a.append(i)
     elif expression[i]==")":
          fromIdx=a.pop()
          fromIdx2=max(a[-1],expression.rfind(",", 0, fromIdx))
          flag=False
          for (fromIndex, toIndex) in first_Index:
             if fromIdx2 + 1 >= fromIndex and i <= toIndex:
                 flag=True
                 break
          if flag==False:
             result.append(expression[fromIdx2+1:i+1])

But this works only if expression is separated by ',' for ex:

expression = 'position(\'a\' , Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ))'

and result for this expression from my code will be correct as exprected. In first expression ,I mentioned ,there is IN operator instead of ',' hence my code doesnt work.

Please help

pls format this.

leoOrion
– leoOrion

2021-03-01 13:59:30 +00:00
Commented Mar 1, 2021 at 13:59 — leoOrion
– leoOrion, Commented Mar 1, 2021 at 13:59
@leoOrion have formatted properly now.Please check

shee
– shee

2021-03-01 14:29:07 +00:00
Commented Mar 1, 2021 at 14:29 — shee
– shee, Commented Mar 1, 2021 at 14:29

aparpara · Accepted Answer · 2021-03-02 10:07:05Z

2

If you want it to be reliable, you need a full-fledged SQL parser. Fortunately, there is an out-of-box solution for that: https://pypi.org/project/sqlparse/. As soon as you have a parsed token tree, you can walk through it and do what you need:

import sqlparse

def extract_functions(tree):
    res = []
    def visit(token):
        if token.is_group:
            for child in token.tokens:
                visit(child)
        if isinstance(token, sqlparse.sql.Function):
            res.append(token.value)
    visit(tree)
    return res

extract_functions(sqlparse.parse(expression)[0])

Explanation. sqlparse.parse(expression) parses the string and returns a tuple of statements. As there is only one statement in the example, we can just take the first element. If there are many statements, you should rather iterate over all tuple elements.

extract_functions recursively walks over a parsed token tree depth first (since you want inner calls appear before outer ones) using token.is_group to determine if the current token is a leaf, tests if the current token is a function, and if it is, appends its string representation (token.value) to the result list.

edited Mar 2, 2021 at 10:07

answered Mar 1, 2021 at 15:54

aparpara

2,20110 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

shee Over a year ago

Thankyou for the answer.will it work for any kind of complex expressions and return the nested functions starting from innermost. And is it feasible to use the package sqlparse?Any disadvantage of using sqlparse?

aparpara Over a year ago

It should work unless there are bugs in the parser. I don't know what may be disadvantages. May be if you have some very special restrictions on using third-party packages, this may cause troubles. But in general case it should be perfectly OK. It is one of the coolest things in Python, that you can find an out-of-box package virtually for everything.

shee Over a year ago

okay thankyou.Can you please explain me this code.

aparpara Over a year ago

Added an explanation.

shee Over a year ago

Have found one issue in this code recently saying it will not support spaces after function name example upper (name) will not work.Can you please help me with this

|

Collectives™ on Stack Overflow

parse nested function to extract each inner function in python

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related