Searching filesystem using Python

Question

I have been coding in Python since the last 2 weeks and pretty new to it.

I have written a code to kind of emulate the way "find" command works in *NIX systems. My code works okay-ish for not so deep directories but if I start searching from the "root" directory, it takes too much time and processor heats up :D which on the other hand takes about 8 seconds using "find" cmd.

Hey I know I am kinda noob in Python now but any hint at trying to improve the search efficiency will be greatly appreciated.

Here's what I have written:

 #!/usr/bin/python3

import os

class srchx:
    file_names = []
    is_prohibit = False

    def show_result(self):
        if(self.is_prohibit):
            print("some directories were denied read-access")
        print("\nsearch returned {0} result(s)".format(len(self.file_names)))
        for _file in self.file_names:
            print(_file)

    def read_dir(self, cur_dir, srch_name, level):
        try:
            listing = os.listdir(cur_dir)
        except:
            self.is_prohibit = True
            return
        dir_list = []
        #print("-"*level+cur_dir)
        for entry in listing:
            if(os.path.isdir(cur_dir+"/"+entry)):
                dir_list.append(entry)
            else:
                if(srch_name == entry):
                    self.file_names.append(cur_dir+"/"+entry)
        for _dir in dir_list:
            new_dir = cur_dir + "/" + _dir
            self.read_dir(new_dir, srch_name, level+1)
        if(level == 0):
            self.show_result()

    def __init__(self, dir_name=os.getcwd()):
        srch_name = ""
        while(len(srch_name) == 0):
            srch_name = input("search for: ")
        self.read_dir(dir_name, srch_name, 0)




def main():
    srch = srchx()

if (__name__ == "__main__"):
    main()

Take a look at and please help me to solve this issue.

user1767754 · Accepted Answer · 2014-09-14 11:58:45Z

2

There is a built-in Directory-Browsing Framework called os.walk() but even os.walk() is slow, if you want to browse faster, you need access to the operating systems file-browser.

https://pypi.python.org/pypi/scandir

scandir is a solution.

answered Sep 14, 2014 at 11:58

user1767754

25.5k21 gold badges167 silver badges177 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Terry Jan Reedy Over a year ago

I expect scandir to be in 3.5. os.walk may be modified to use it and run faster. If you download scandir, you could probably do the substitution yourself.

Marty Over a year ago

@user1767754 scandir does work much better. It took a while for me to understand the code though :) thank u

PM 2Ring · Accepted Answer · 2014-09-14 14:40:45Z

0

What user1767754 said. You can't really improve the speed much using the methods you're calling. os.walk() is a bit more efficient, though. I've never used scandir (or pypi) so I can't comment.

BTW, that's rather good looking code for a noob, Marty! But there are a couple of issues with it.

It's not a good idea to initialise file_names and is_prohibit like that because it makes them class variables; initialise them in __init__.
You should read srch_name outside the class and pass it your class constructor. You do that by making it an arg of __init__, as described in the link above.

It's generally good policy to handle user input in the outermost parts of your code (when practical) rather than doing it in the inner parts of your code. I like to think of my user input routines as border guards that only let good input into the inner sanctum of my code. Users are unpredictable critters and there's no telling what mischief they'll get up to. :)

answered Sep 14, 2014 at 14:40

PM 2Ring

55.6k6 gold badges96 silver badges202 bronze badges

2 Comments

user1767754 Over a year ago

There has been a proposal to python for using os.scandir() in the future implementations, as it is really fast.

Marty Over a year ago

@PM 2Ring: your advices were quite valuable. I'll certainly keep them in mind next time around.

Collectives™ on Stack Overflow

Searching filesystem using Python

2 Answers 2

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related