Python searching multiple directories and reading select files

Question

I am looking for some help performing actions on a set of files in two different directories using Python.

I am attempting to:

Search two different directories
Find the 15 last modified files (comparing files in both directories)
Read all 15 recently modified files line by line

I can accomplish reading through one file directory using glob. However, I cannot specify multiple directories. Is there another way I can accomplish this?

Below is my code which accomplishes grabbing the latest 15 files in dir1 but not dir2.

dir1 = glob.iglob("/dir1/data_log.*")
dir2 = glob.iglob("/dir2/message_log.*")

latest=heapq.nlargest(10, dir1, key=os.path.getmtime)
for fn in latest:
    with open(fn) as f:
        for line in f:
            print(line)

Julien Spronck · Accepted Answer · 2016-07-27 14:16:14Z

4

I'm not sure this is what you are after but if you were to use glob.glob instead of glob.iglob, you could do

dir1 = glob.glob("/dir1/data_log.*")
dir2 = glob.glob("/dir2/message_log.*")

latest=heapq.nlargest(10, dir1+dir2, key=os.path.getmtime)

And actually, if you don't like the idea of using lists (glob.glob) instead of generators (glob.iglob), you can do

from itertools import chain

dir1 = glob.iglob("/dir1/data_log.*")
dir2 = glob.iglob("/dir2/message_log.*")

latest=heapq.nlargest(10, chain(dir1, dir2), key=os.path.getmtime)

edited Jul 27, 2016 at 14:16

answered Jul 27, 2016 at 13:55

Julien Spronck

15.5k5 gold badges50 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python searching multiple directories and reading select files

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related