Read multiple files from a list of directory Python

Question

I have a list of directories with same structure/same file name. Under each there are files with different types and each directory has the same file names (number of files may vary) but different contents.

parentDIR-
     - subDir1
       - file1.ext1 (contains taskID)
       - file2.ext2 (contains userID)
       - file3.ext3
     - subDir2
       - file1.ext1 (contains taskID)
       - file2.ext2 (contains userID)
       - file3.ext3
     - subDir100

How do I search all the directories and extract taskID and userID as a pair from file1/file2? I need the file1/file2 in the same directory to be opened/closed properly and read the contents because they have some information shared in common.

I am thinking to use

glob, with xxxx as, ExitStack(), but not sure how to connect them all together

What have you tried yourself so far?

Jens
– Jens

2020-07-22 05:00:01 +00:00
Commented Jul 22, 2020 at 5:00 — Jens
– Jens, Commented Jul 22, 2020 at 5:00

bigbounty · Accepted Answer · 2020-07-22 05:00:29Z

1

You can use regex in glob

In [1]: import glob
   ...: for x in glob.glob('parent/**/*.ext[1|2]', recursive=True):
   ...:     # Open, read content close here
   ...:     print(x)
   ...:
parent/subdir2/file1.ext1
parent/subdir2/file2.ext2
parent/subdir1/file1.ext1
parent/subdir1/file2.ext2

answered Jul 22, 2020 at 5:00

bigbounty

17.5k7 gold badges45 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

tudou Over a year ago

thanks, in my case, ext1, ext2 are actually quite different pattern, e.g., abc.xy1, cde.dat.zst etc

bigbounty Over a year ago

@tudou you can customize the regex as per your needs

tudou · Accepted Answer · 2020-07-23 13:16:01Z

0

I used a slightly different approach

types={"*.ext1", "*.ext2"}

for type in types:
    path(dir_name).rglob(type):
        do_something

answered Jul 23, 2020 at 13:16

tudou

5471 gold badge10 silver badges22 bronze badges

Collectives™ on Stack Overflow

Read multiple files from a list of directory Python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related