3

I have a command line utility from a third party (it's big and written in Java) that I've been using to help me process some data. This utility expects information in a line delimited file and then outputs processed data to STDOUT.

In my testing phases, I was fine with writing some Perl to create a file full of information to be processed and then sending that file to this third party utility, but as I'm nearing putting this code in production, I'd really prefer to just pipe data to this utility directly instead of first writing that data to a file as this would save me the overhead of having to write unneeded information to disk. I recently asked on this board how I could do this in Unix, but have since realized that it would be incredibly more convenient to actually run it directly out of a Perl module. Perhaps something like:

system(bin/someapp do-action --option1 some_value --input $piped_in_data)

Currently I call the utility as follows:

bin/someapp do-action --option1 some_value --input some_file

Basically, what I want is to write all my data either to a variable or to STDOUT and then to pipe it to the Java app through a system call in the SAME Perl script or module. This would make my code a lot more fluid. Without it, I'd wind up needing to write a Perl script which calls a bash file half way through which in turn would need to call another Perl script to prep data. If at all possible I'd love to just stay in Perl the whole way through. Any ideas?

Improve this question
1
  • Does the tool accept data on STDIN? Commented Dec 7, 2010 at 22:01

3 Answers 3

Reset to default
9

If I am reading your question correctly, you are wanting to spawn a process and be able to both write to its stdin and read from its stdout. If that is the case, then IPC::Open2 is exactly what you need. (Also see IPC::Open3 you also need to read from the process' stderr.)

Here is some sample code. I have marked the areas you will have to change.

#!/usr/bin/perl

use strict;
use warnings;

use IPC::Open2;

# Sample data -- ignore this.
my @words = qw(the quick brown fox jumped over the lazy dog);

# Automatically reap child processes.  This is important when forking.
$SIG{'CHLD'} = 'IGNORE';

# Spawn the external process here.  Change this to the process you need.
open2(*READER, *WRITER, "wc -c") or die "wc -c: $!";

# Fork into a child process.  The child process will write the data, while the
# parent process reads data back from the process.  We need to fork in case
# the process' output buffer fills up and it hangs waiting for someone to read
# its output.  This could cause a deadlock.
my $pid;
defined($pid = fork()) or die "fork: $!";

if (!$pid) {
    # This is the child.

    # Close handle to process' stdout; the child doesn't need it.
    close READER;

    # Write out some data.  Change this to print out your data.
    print WRITER $words[rand(@words)], " " for (1..100000);

    # Then close the handle to the process' stdin.
    close WRITER;
    # Terminate the child.
    exit;
}

# Parent closes its handle to the process' stdin immediately!  As long as one
# process has an open handle, the program on the receiving end of the data will
# never see EOF and may continue waiting.
close WRITER;

# Read in data from the process.  Change this to whatever you need to do to
# process the incoming data.
print "READ: $_" while (<READER>);

# Close the handle to the process' stdin.  After this call, the process should
# be finished executing and will terminate on its own.
close READER;
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This recent question stackoverflow.com/questions/4377967/… shows some working examples of IPC::Open3
I think this is exactly what I want, but I'm having some trouble figuring out how to use it in my example. Any chance you can paste sample code more similar to the example in my post?
Thanks a ton! This is exactly what I wanted and now it makes perfect sense to me. You're awesome!
2

If all you want to do is pipe the STDOUT from your program into your other program's STDIN, you can do this via the standard Perl open command:

open (CMD, "|$command") or die qq(Couldn't execute $command for piping);

Then, all you have to do to send data to this command is to use the print statement:

print CMD $dataToCommand;

And, you finally close your pipe with the close statement:

close (CMD);

PERL HINT

Perl has a command called perldoc which can give you the documentation of any Perl function or Perl module installed on your system. To get more information about the open command, type:

$ perldoc -f open

The -f says this is a Perl function

If you're doing what cdhowie said in his answer, (you're spawning a process, then reading and writing to that process), you will need IPC::Open2. To get information about the IPC::Open2 module, type:

$ perldoc IPC::Open2
Improve this answer

1 Comment

One may prefer list form of open to avoid having the shell interpret metacharacters: my @grep_args = ("egrep", "-i", $some_pattern, @many_files); open(my $grep_pipe, "-|", @grep_args) || die "can't open @grep_args|: $!"; - taken from man 1 perlipc. This example pipes from command to perl; and to pipe from perl to a command, flip "-|" to "|-"
1

If it only accepts files, let it open "/proc/self/fd/0", which is the same as STDIN. For the rest, see cdhowies answer.

Improve this answer

1 Comment

Or /dev/stdin which may be a symlink to /proc/self/fd/0.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.