recursive file list in Ruby

Question

I'm new to Ruby (being a Java dev) and trying to implement a method (oh, sorry, a function) that would retrieve and yield all files in the subdirectories recursively.

I've implemented it as:

def file_list_recurse(dir)
  Dir.foreach(dir) do |f|
    next if f == '.' or f == '..'
    f = dir + '/' + f
    if File.directory? f
      file_list_recurse(File.absolute_path f) { |x| yield x }
    else
      file = File.new(f)
      yield file
    end
  end
end

My questions are:

Does File.new really OPEN a file? In Java new File("xxx") doesn't... If I need to yield some structure that I could query file info (ctime, size etc) from what would it be in Ruby?
{ |x| yield x } looks a little strange to me, is this OK to do yields from recursive functions like that, or is there some way to avoid it?
Is there any way to avoid checking for '.' and '..' on each iteration?
Is there a better way to implement this?

Thanks

PS: the sample usage of my method is something like this:

curr_file = nil

file_list_recurse('.') do |file|
  curr_file = file if curr_file == nil or curr_file.ctime > file.ctime
end

puts curr_file.to_path + ' ' + curr_file.ctime.to_s

(that would get you the oldest file from the tree)

==========

So, thanks to @buruzaemon I found out the great Dir.glob function which saved me a couple of lines of code. Also, thanks to @Casper I found out the File.stat method, which made my function run two times faster than with File.new

In the end my code is looking something like this:

i=0
curr_file = nil

Dir.glob('**/*', File::FNM_DOTMATCH) do |f|
  file = File.stat(f)
  next unless file.file?
  i += 1
  curr_file = [f, file] if curr_file == nil or curr_file[1].ctime > file.ctime
end

puts curr_file[0] + ' ' + curr_file[1].ctime.to_s
puts "total files #{i}"

=====

By default Dir.glob ignores file names starting with a dot (considered to be 'hidden' in *nix), so it's very important to add the second argument File::FNM_DOTMATCH

buruzaemon · Accepted Answer · 2011-11-02 16:32:10Z

13

How about this?

puts Dir['**/*.*']

answered Nov 2, 2011 at 16:32

buruzaemon

3,9251 gold badge26 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Oleg Mikheev Over a year ago

That's great! But it produces an Array of String objects. What I'm looking for is function that would yield a File-like structure so that I could do my own calculations based on that. Finding the biggest file, the earliest ctime etc.

Oleg Mikheev Over a year ago

Dir['.'] doesn't accept a block. But Dir.glob does! It answers my questions, except for question #1

Casper · Accepted Answer · 2011-11-02 16:36:58Z

6

According to the docs File.new does open the file. You might want to use File.stat instead, which gathers file-related stats into a queryable object. But note that the stats are gathered at point of creation. Not when you call the query methods like ctime.

Example:

Dir['**/*'].select { |f| File.file?(f) }.map { |f| File.stat(f) }

answered Nov 2, 2011 at 16:36

Casper

34.5k4 gold badges89 silver badges84 bronze badges

1 Comment

Oleg Mikheev Over a year ago

File.stat ironically doesn't provide the name of the File, so I can't use it as a data object to return from my method. Also, I have a tree of 200,000 files. Running your example results in ruby process grow above 60 Mb, while running my method (even with File.new) never makes ruby go above 6 Mb. (I'm testing with watch -n 0,1 "ps ax -o comm,rss|grep ruby >> /tmp/q"). But you sample line of code indeed looks cool ;-)

Oleg Mikheev · Accepted Answer · 2011-11-09 12:32:05Z

6

this thing tells me to consider accepting an answer, I hope it wouldn't mind me answering it myself:

i=0
curr_file = nil

Dir.glob('**/*', File::FNM_DOTMATCH) do |f|
  file = File.stat(f)
  next unless file.file?
  i += 1
  curr_file = [f, file] if curr_file == nil or curr_file[1].ctime > file.ctime
end

puts curr_file[0] + ' ' + curr_file[1].ctime.to_s
puts "total files #{i}"

answered Nov 9, 2011 at 12:32

Oleg Mikheev

17.5k16 gold badges77 silver badges98 bronze badges

Comments

Dave Newton · Accepted Answer · 2011-11-02 16:10:36Z

3

You could use the built-in Find module's find method.

answered Nov 2, 2011 at 16:10

Dave Newton

161k27 gold badges263 silver badges310 bronze badges

Comments

peter · Accepted Answer · 2018-04-28 20:19:49Z

1

If you are on Windows see my answer here under for a mutch faster (~26 times) way than standard Ruby Dir. If you use mtime it's still going to be waaayyy faster.

If you use another OS you could use the same technique, I'm curious if the gain would be that big but I'm almost certain.

How to find the file path of file that is not the current file in ruby

answered Apr 28, 2018 at 20:19

peter

42.3k7 gold badges67 silver badges110 bronze badges

Collectives™ on Stack Overflow

recursive file list in Ruby

5 Answers 5

2 Comments

1 Comment

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

2 Comments

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related