Python text processing

Question

I have a data file with huge data in it. I am only interested in two codes per user that needs to be updated. I have the new codes in separate file. I just want to compare both files and add new codes to existing files.

Old File: (txt2)

.
..
..
alpha Donec vulputate lorem tortor, nec fermentum nibh bibendum vel.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Praesent dictum luctus massa, non euismod lacus.
${alpha_john}: 'Lorem ipsum dolor sit amet, consectetur'
${beta_john}: 'iuhertgh jndsfbjpijwrg'
${alpha_mac}: 'acerat a lorem eget, ultricies'
${beta_mac}: 'elit nibh, eu condimentum orci viverra q'
${alpha_joe}: 'gravida lorem, ut congue diam.'
${beta_joe}: 'orttitor in condimentum nec, venenatis eu urna'
${alpha_mark}: ''
${beta_mark}: ''
${alpha_ross}: 'suscipit vitae felis non suscipit.'
${beta_ross}: 'non vulputate convallis, ligula diam sagittis urna, in venenatis'
${alpha_don}: 'Pellentesque feugiat diam est, at rhoncus orci porttitor'
${beta_don}: 'Sed elementum elit nibh'
${alpha_harry}: 'Proin tempor lacus arcu.'
${beta_harry}: 'posuere sollicitudin mi, et vulputate nisl fringilla non'
Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos.
Aliquam euismod ultrices lorem, sit amet imperdiet est tincidunt vel.
Phasellus dictum justo sit amet ligula varius aliquet auctor et metus.
..
..
.

Codes file: (txt1)

${alpha_john}: 'XXXXXHHHHHHHXXXXXX'
${beta_john}: 'XFFFFFFFFFGGGGGGGGDDDDDD'
${alpha_mac}: 'DDDDDDKKKKKKKKK'
${beta_mac}: 'KKKKKKKKKKKYYYYYYYYYYYYD'
${alpha_joe}: 'TTTTTVVVVVVVVVVVKK'
${beta_joe}: 'OOOOOOOSSSSSSSSSSPPPPPP'
${alpha_ross}: 'SSSSSHHHHHHHHTTTTTTTT'
${beta_ross}: 'PPPPPWWWWWHHHHHHHHHH'
${alpha_harry}: 'IIIIIIEEEEEEETTTTTTTTTT'
${beta_harry}: 'YYYYYYYYEEEEEEEEEEMMMMMMMMMM'

My Code:

#!/usr/bin/env python

import os, sys, re, time
import argparse
import logging
import time

cat /dev/null > /home/user/scripts/temp/txt3

file1=open("/home/user/scripts/temp/txt1",'r+')
file2=open("/home/user/scripts/temp/txt2", 'r+')
file3=open("/home/user/scripts/temp/txt3", 'r+')

for line1 in file1:
    keyword=line1[line1.find("{")+1:line1.find("}")]
    for line2 in file2:
        if keyword in line2:
            file3.write(line1)
        else:
            file3.write(line2)
file1.close()
file2.close()
file3.close()

Output:

alpha Donec vulputate lorem tortor, nec fermentum nibh bibendum vel.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Praesent dictum luctus massa, non euismod lacus.
${alpha_john}: 'XXXXXHHHHHHHXXXXXX'
${beta_john}: 'iuhertgh jndsfbjpijwrg'
${alpha_mac}: 'acerat a lorem eget, ultricies'
${beta_mac}: 'elit nibh, eu condimentum orci viverra q'
${alpha_joe}: 'gravida lorem, ut congue diam.'
${beta_joe}: 'orttitor in condimentum nec, venenatis eu urna'
${alpha_mark}: ''
${beta_mark}: ''
${alpha_ross}: 'suscipit vitae felis non suscipit.'
${beta_ross}: 'non vulputate convallis, ligula diam sagittis urna, in venenatis'
${alpha_don}: 'Pellentesque feugiat diam est, at rhoncus orci porttitor'
${beta_don}: 'Sed elementum elit nibh'
${alpha_harry}: 'Proin tempor lacus arcu.'
${beta_harry}: 'posuere sollicitudin mi, et vulputate nisl fringilla non'
Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos.
Aliquam euismod ultrices lorem, sit amet imperdiet est tincidunt vel.
Phasellus dictum justo sit amet ligula varius aliquet auctor et metus.

this code just prints one line from txt1 '${alpha_john}: 'XXXXXHHHHHHHXXXXXX'' in new file but rest of the lines stays as those are in old file (txt2).

What to do so all lines from (txt1) get overwritten?

Let me know for any additional infromation required.

FHTMitchell · Accepted Answer · 2019-01-03 11:16:26Z

You're iterating through file2 len(file1) times which is certainly not what you want to do. You want to construct a replacement dictionary from file1 like so:

import re

# regex to find usernames.
# You can use str.split to find the usernames like you did if you're
# not comfortable with regular expressions.
user_regex = re.compile(r'^\${([a-zA-Z0-9_]+)}: ')

# rename files to something better
codes_file = "/home/user/scripts/temp/txt1"
old_file = "/home/user/scripts/temp/txt2"
new_file = "/home/user/scripts/temp/txt3"

codes = {}
with open(codes) as f:  # use with to safely open files
    for line in f:
         match = user_regex.search(line)
         if match:
              codes[match.group(1)] = line

# now we have the codes in ram for easy lookup

with open(old_file) as old, open(new_file, 'w') as new:
     for line in old:
         match = user_regex.search(line)
         if match and match.group(1) in codes.keys():
             new.write(codes[match.group(1)])
         else:
             new.write(line)

Collectives™ on Stack Overflow

Python text processing

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related