Python find, replace or add string in file

Question

Hi and thanks for looking :)

I have a text tile that is over 2500 lines, each line contains information about a video file. One of the tags(so to speak) is for watched status, I am looking for a way of changing it from one value to another or add in a new value if it is not set. The code below works but it has to open and close the file for each searchvalue, this means its very slow. Can any one suggest a way of opening the file once and doing all the searches in one pass?

Thanks

for x in y:
    print '    --> ' + x['title'].encode('utf-8')

    searchValue = x['movieid']
    addValue = "\t_w\t1\t"
    checkvalue = "\t_w\t0\t"
    for line in fileinput.input(file, inplace=1):
        if searchValue in line:
            if checkvalue in line:
                line = line.replace(checkvalue, addValue)
            elif not addValue in line:
                line = line + addValue
        sys.stdout.write(line)

This is what i ended up with, thanks to everyone for your input.

    myfile_list = open(file).readlines()
    newList = []
    for line in myfile_list:
        for x in y:
            if x['movieid'] in line:
                print '    --> ' + x['title'].encode('utf-8')
                if checkvalue in line:
                    line = line.replace(checkvalue, addValue)
                elif not addValue in line:
                    line = line.replace('\n', addValue+'\n')
        newList.append(line)
    outref = open(file,'w')
    outref.writelines(newList)
    outref.close()

Edit I have come across an issue with the encoding, The file is encoded in utf-8 but it errors out or just does not find a match when the search value is

'Hannibal - S01E01 - Ap\xe9ritif.mkv'

the matching line in the file looks like

_F  /share/Storage/NAS/Videos/Tv/Hannibal/Season 01/Hannibal - S01E01 - Apéritif.mkv    _rt 43  _r  8.4 _s  1   _v  c0=h264,f0=24,h0=720,w0=1280    _IT 717ac9d _id 1671    _et Apéritif    _DT 7142d53 _FT 7142d53 _A  4212,4211,2533,4216 _C  T   _G  j|d|h|t _R  GB:TV-MA    _T  Hannibal    _U   thetvdb:259063 imdb:tt2243973  _V  HDTV    _W  4210    _Y  71  _ad 2013-04-04  _e  1   _ai Apéritif    _m  1117

I have tried codecs.open and decode().encode() options but it allways errors out, I believe its the accented letters in the line that is the issue as it can do the if searchValue in line: if the line does not have an accented letter. This is what I am currently trying but I am open to other methods.

if os.path.isfile("/share/Apps/oversight/index.db"):
    newfile = ""
    #searchValueFix = searchValue.decode('latin-1', 'replace').encode('utf8', 'replace')
    #print searchValueFix
    #print searchValue
    addValue = "\t_w\t1\t"
    replacevalue = "\t_w\t0\t"
    file = open("/share/Apps/oversight/index.db", "r")
    for line in file:
        if searchValue in line:
            if replacevalue in line:
                line = line.replace(replacevalue, addValue)
            elif not addValue in line:
                line = line.replace(searchValue+"\t", searchValue+addValue)
        newfile = newfile + line
    file.close()
    file = open("/share/Apps/oversight/index.db", "w")
    file.write(newfile)
    file.close()
    newfile = ""

Open the file with "r+" rather than "w" and "r" seperately. Then you can read, and write without having to open and close it. — rlms
– rlms, Commented Aug 9, 2013 at 14:49

Imanol Luengo · Accepted Answer · 2013-08-09 16:29:43Z

1

Similar to the method proposed by PyNEwbie, you can write lines 1 by 1:

myfile_list = open(file).readlines()
outref = open(myfile, 'w')
for line in myfile_list:
    # do something to line
    outref.write(line)

outref.close()

answered Aug 9, 2013 at 16:29

Imanol Luengo

16k3 gold badges52 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

PyNEwbie · Accepted Answer · 2013-08-09 16:33:01Z

0

Yes read in your file to a list

myfile_list = open(file).readlines()
newList = []
for line in myfile_list:
  .
  .

  newList.append(line)   # this is the line after you have made your changes
outref = open(myfile,'w')
 outref.writelines(newList)
outref.close()

edited Aug 9, 2013 at 16:33

answered Aug 9, 2013 at 14:49

PyNEwbie

4,9707 gold badges48 silver badges88 bronze badges

2 Comments

jhmiller Over a year ago

PyNEwbie, I tried your method but although its a lot faster the file written is the same as it was before. I checked that line does get changed but the changes are not written?

PyNEwbie Over a year ago

well that just means that the things you changed were not changed in myfile list just because I can't see your code I am going to annotate mine a little - if you find an answer useful you should mark it up and if it the accepted answer you should check it

Collectives™ on Stack Overflow

Python find, replace or add string in file

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related