Using regex to find all lines matching a pattern in Python [duplicate]

Question

I have:

# runPath is the current path, commands is a list that's mostly irrelevant here
def ParseShellScripts(runPath, commands):
    for i in range(len(commands)):
        if commands[i].startswith('{shell}'):
            # todo: add validation/logging for directory `sh` and that scripts actually exist
            with open(os.path.join(runPath, 'sh', commands[i][7:]),"r") as shellFile:
                for matches in re.findall("/^source.*.sh", shellFile):
                    print matches

However, I get this error:

Traceback (most recent call last):
  File "veri.py", line 396, in <module>
    main()
  File "veri.py", line 351, in main
    servList, labels, commands, expectedResponse = ParseConfig(relativeRunPath)
  File "veri.py", line 279, in ParseConfig
    commands = ParseShellScripts(runPath, commands)
  File "veri.py", line 288, in ParseShellScripts
    for matches in re.findall("/^source.*.sh", shellFile):
  File "/usr/lib/python2.7/re.py", line 177, in findall
    return _compile(pattern, flags).findall(string)
TypeError: expected string or buffer

Edit: adding some files as examples

#config.sh
#!/bin/bash

dbUser = 'user'
dbPass = 'pass'
dbSchema = ''
dbMaxCons = '4000'

#the shellFile I'm looking in
#!/bin/bash
source config.sh

OUTPUT=$(su - mysql -c "mysqladmin variables" | grep max_connections | awk '{print $4}')
if [[ ${OUTPUT} -ge ${dbMaxCons}]]; then
    echo "Success"
    echo ${OUTPUT}
else
    echo ${OUTPUT}
fi

Basically what I want to accomplish is to search through all specefied files in the sh directory, and if any of them contain source*.sh (e.g., source config.sh), print that file (eventually I will be expanding it out and appending it to the top of the current file so that I can pass the single command-string via ssh.. but that's not relevant here I don't think.)

What am I doing wrong?

shellfile is a file, you want to pass a string. you'll probably have to enumerate the lines first. typically for matches in re.findall("/^source.*.sh", line) for line in shellfile: — njzk2
– njzk2, Commented Oct 17, 2014 at 17:37

Ankush Jain · Accepted Answer · 2014-10-17 17:37:41Z

5

You are trying to run a regex.findall on the file handle shellFile. You need to read from that file, and run the regex on the data you read.

Maybe, something like this?

with open(os.path.join(runPath, 'sh', commands[i][7:]),"r") as shellFile:
    data = shellFile.read()
    for matches in re.findall("/^source.*.sh", data):
        print matches

answered Oct 17, 2014 at 17:37

Ankush Jain

4972 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Zav Over a year ago

It is same solutin i write, but take 1 string of code more. Also it use more memory.

Ankush Jain Over a year ago

It's not always about number of lines, especially when the change is trivial. You could write a lot of Python code in one line, and it'd be undecipherable to a reader. The point was to illustrate the OP's mistake. Also, I'm not sure I understand how calling a read in one line takes more memory than calling it in the next.

Zav Over a year ago

1) Ofc, but if you limit your line to 80-100 chars it's not a problem, 2) You assign data to variable, and memory cleanup will raise only after your "with" close, and with my variant memory cleanup will raise after findall method — before cycle.

Ankush Jain Over a year ago

Premature optimization is the root of all evil - Donald E. Knuth

Zav Over a year ago

It is not premature. If you know that is used in just 1 place — why not? :-)

Zav · Accepted Answer · 2014-10-17 17:37:28Z

2

You forgot to call .read() method

for matches in re.findall("/^source.*.sh", shellFile.read())

answered Oct 17, 2014 at 17:37

Zav

6713 silver badges8 bronze badges

Collectives™ on Stack Overflow

Using regex to find all lines matching a pattern in Python [duplicate]

2 Answers 2

5 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Linked

Related