Extracting multiple rows from text file using Python

Question

I have a text file like below. I would like to extract the rows below "Parameters------> up to See file nwirp_nsmc.sen for parameter sensitivities." and write in different text file. How can I do that?

OPTIMISATION RESULTS

Covariance matrix and parameter confidence intervals cannot be determined:-
Some form or regularisation was implemented so these are not applicable.
Use the PREDUNC7 utility to obtain a full posterior covariance matrix.
Parameters ----->
Parameter       Estimated value

1.hklay1         3.278692E-06
2.kppt1          4.249307E-07
3.kppt2          2.849132E-06
-------
-------

See file nwirp_nsmc.sen for parameter sensitivities.


Observations ----->

This is what I tried but it is not working for my file. I know I am missing something here but I dont know now what is the missing part.

New extracting

inFile = open('nwirp-1.txt')
outFile = open('result2.txt', 'w')
new = []
Set = True
for line in inFile:
    new.append(line)
    if line.startswith("Parameters------>"):
        #---- starts a new data set
        if Set:
            outFile.write("".join(new))
    elif line.startswith("See file nwirp_nsmc.sen for parameter sensitivities."):
        Set = False
        new = []
inFile.close()
outFile.close()

Isn't "Parameter" outside of the bounding "**"? And don't use Set, it's a built in. Finally, why don't you check if the line startswith("**") and start there? — kabanus
– kabanus, Commented May 1, 2017 at 6:35

Hackaholic · Accepted Answer · 2017-05-01 07:17:25Z

1

you can try like this:

f = open("nwirp-1.txt")

for line in f:
    if line.strip().startswith("Parameters ----->"):
       f2 = open('result2.txt', 'w')
       line = next(f)
       while not line.strip().startswith("See file"):
           f2.write(line)
           line = next(f)
       f2.close()
f.close()

Output

Parameter       Estimated value

1.hklay1         3.278692E-06
2.kppt1          4.249307E-07
3.kppt2          2.849132E-06
4.kppt3          1.548621E-06

edited May 1, 2017 at 7:17

answered May 1, 2017 at 6:39

Hackaholic

19.8k6 gold badges59 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Bulbul Ahmmed Over a year ago

Your output looks good, but I do not need top and bottom line. Is there any possible way to eliminate those lines.

Bulbul Ahmmed Over a year ago

This is working great. Thanks a lot. I appreciate your help

kabanus · Accepted Answer · 2017-05-01 07:12:32Z

1

Easiest way is to dump file and split:

with open('myfile') as fd:
    relevent = fd.read().split("**")[1]
with open('outfile','w') as fd:
    fd.write(relevent)

This is assuming there is one "**" part in the middle, and no other "**".

=====================================

Changed question

with open('nwirp-1.txt') as inFile, open('result2.txt', 'w') as outFile:
    writing = False
    for line in inFile:
        if line.startswith("Parameters ----->"):
            writing = True
        if writing:
            outFile.write(line)
        if line.startswith("See file"):
            writing = False

Note especially not to use Set, it's a built-in. You were close, I fixed it up a little. Also note the cool with statement, no need for close with it.

edited May 1, 2017 at 7:12

answered May 1, 2017 at 6:23

kabanus

26.3k7 gold badges48 silver badges79 bronze badges

6 Comments

Bulbul Ahmmed Over a year ago

There is no ** sign in my file. I added it to show where I want to extract data. Sorry for misunderstanding.

kabanus Over a year ago

YEa I see you completely changed stuff, I'll have a look.

Bulbul Ahmmed Over a year ago

Yes. Sorry. This is my first question here. So, it took a while to figure out the way stackoverflow works

Bulbul Ahmmed Over a year ago

Thanks. I get your point. But, your code is not generating any output file?

kabanus Over a year ago

Probably a mismatch between the "------>" in your file and in the code. Double check the match strings. Also I added a ":" in the stop string by mistake.

|

TimGJ · Accepted Answer · 2017-05-01 07:13:51Z

1

You need to use a re with the DOTALL flag.

import re

myre = re.compile(r"Parameters ----->(.*?)See file ", re.DOTALL)
parts = myre.findall(text)

if parts:
    with open('foo.txt', 'w') as output:
        for part in parts:
            print(part, file=output)
else:
    print("No match!")

Note that this assumes that there may be multiple blocks, hence the ? in the re which makes it non-greedy.

edited May 1, 2017 at 7:13

answered May 1, 2017 at 7:06

TimGJ

1,6292 gold badges17 silver badges32 bronze badges

Collectives™ on Stack Overflow

Extracting multiple rows from text file using Python

New extracting

3 Answers 3

2 Comments

6 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

New extracting

3 Answers 3

2 Comments

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related