read files with same name from two directories simultaneously using python

Question

I have directory A.
A contains a.txt, b.txt, c.txt.

I have directory B,
B contains a.csv, b.csv, c.csv.

I want am able to read and process files both separately as follows:

a.py

src = '/home/A'
file_paths = [os.path.join(src, f) for f in os.listdir(src)]
for file in file_paths:

    for line in open(file, 'rb').readlines():
    #do some some
    # print some stuff

b.py

src = '/home/B'
file_paths = [os.path.join(src, f) for f in os.listdir(src)]
for file in file_paths:

    with open(filename, 'r') as ff:
    for line in ff:
        _, end, label = line.strip().split('\t')

    #do some some
    # print some stuff

So results from a.py look like :

file_processed: a.txt   
output    
file processed: b.txt  
output  
file processed: c.txt  
output

and similarly from b.py

file_processed: a.csv   
output    
file processed: b.csv  
output  
file processed: c.csv  
output

I don't want to do this ^ process as it is hard to visually compare results of a.csv with a.txt because they are on two different terminals or files and so on.

What I want to do is :read files with same name from dir's at once and then perform computations and then print the results

Example: I should read a.txt from /A and process it , and then read a.csv from /B and process it. then print both results. Then move on to b.txt from /A and b.csv from /Band so on.

Please advise. Let me know if you want me to update the logic for do some stuff portion of the code.

Please provide a minimal reproducible example. Also, why would use use C-style comments in your Python code? Use python comments. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Feb 26, 2018 at 18:59

sytech · Accepted Answer · 2018-02-26 19:09:15Z

1

Just form both lists of filepaths then loop over them at the same time.

a_root = '/home/A'
b_root = '/home/B'
a_paths = [os.path.join(a_root, f) for f in os.listdir(a_root)]
b_paths = [os.path.join(b_root, f) for f in os.listdir(b_root)]
for a_path, b_path in zip(a_paths, b_paths):
    with open(a_path) as a_file, open(b_path) as b_file:
        a_data = a_file.read()
        b_data = b_file.read()
    # do stuff with a and b

answered Feb 26, 2018 at 19:09

sytech

42.7k8 gold badges77 silver badges127 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Aphid · Accepted Answer · 2018-02-26 19:09:27Z

1

Example: I should read a.txt from /A and process it , and then read a.csv from /B and process it. then print both results. Then move on to b.txt from /A and b.csv from /Band so on.

I feel like you have already answered your own question. This even works as pseudocode. Here is an inelegant but effective example.

filenames = [f for f in however_you_scrape_you_filenames]
directory_extension_map = {'/path/to/a': 'txt',
                           '/path/to/b': 'csv'
                          }
for fname in filenames:
    for directory in directory_extension_map:
        extension = directory_extension_map[directory]
        file_path = os.path.join(directory, f'{fname}.{extension}')
        # Read and parse file_path

I feel like you can take it from here.

answered Feb 26, 2018 at 19:09

Aphid

3631 silver badge9 bronze badges

1 Comment

Aphid Over a year ago

How did it go? Did this answer your question?

Collectives™ on Stack Overflow

read files with same name from two directories simultaneously using python

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related