Ruby parallel process in map

Question

Help me plz

How i can implement method pmap for Array like map but in two process

I have code

class Array

  def pmap
    out = []
    each do |e|
      out << yield(e)
    end
    out
  end

end

require 'benchmark'

seconds = Benchmark.realtime do
  [1, 2, 3].pmap do |x|
    sleep x
    puts x**x
  end
end

puts "work #{seconds} seconds"

In result i must get 3 second for benchmark

first match in google: github.com/bruceadams/pmap, second: stackoverflow.com/questions/5019247/… did you check them? — tokland
– tokland, Commented Apr 15, 2013 at 18:52
Yes. But its code use Thread. I ask for Process.fork implimentation — Semyon Pupkov
– Semyon Pupkov, Commented Apr 15, 2013 at 19:00
Why does it have to be Process.fork? Are you doing a quiz and that's the requirement? If so, aren't you supposed to come up with the answer yourself? — the Tin Man
– the Tin Man, Commented Apr 15, 2013 at 19:12

gmalette · Accepted Answer · 2013-04-15 19:51:36Z

3

To get absolutely 2 forks

You don't absolutely need RPC. Marshal + Pipe should usually work.

class Array

  def pmap
    first, last = self[0..(self.length/2)], self[(self.length/2+1)..-1]

    pipes = [first, last].map do |array|
      read, write = IO.pipe
      fork do
        read.close
        message = []
        array.each do |item|
          message << yield(item)
        end
        write.write(Marshal.dump message)
        write.close
      end
      write.close
      read
    end

    Process.waitall

    first_out, last_out = pipes.map do |read|
      Marshal.load(read.read)
    end

    first_out + last_out
  end

end

Edit

Now using fork

edited Apr 15, 2013 at 19:51

answered Apr 15, 2013 at 18:51

gmalette

2,50918 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Semyon Pupkov Over a year ago

Thanks for answer. But i find Process.fork implimentaion not in Thread

tadman Over a year ago

Then you'll need a heavy helping of RPC. This is not trivial.

tadman Over a year ago

That's quite a clever gem. Nice find.

dbenhur · Accepted Answer · 2013-04-15 21:13:15Z

3

Try the parallel gem.

require 'parallel'

class Array
  def pmap(&blk)
    Parallel.map(self, {:in_processes: 3}, &blk)
  end
end

answered Apr 15, 2013 at 21:13

dbenhur

20.5k4 gold badges52 silver badges47 bronze badges

Collectives™ on Stack Overflow

Ruby parallel process in map

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related