Golang - Copy exec output to a buffer

Question

I'm writing a function that exec's a program and returns stdout and stderr. It also has the option to display the output to the console. I'm clearly not waiting on something, as if I run the function twice in a row, the outputs are different. Here's a sample program, replace the dir var with a dir with a lot of files to fill up the buffers:

func main() {
    dir := "SOMEDIRECTORYWITHALOTOFFILES"
    out, err := run("ls -l "+dir, true)
    if err != nil {
        log.Fatalf("run returned %s", err)
    }
    log.Printf("Out: %s", out)
    out2, err := run("ls -l "+dir, false)
    if err != nil {
        log.Fatalf("run returned %s", err)
    }
    log.Printf("Out2: %s", out2)
    if out != out2 {
        log.Fatalf("Out mismatch")
    }
}

func run(cmd string, displayOutput bool) (string, error) {
    var command *exec.Cmd
    command = exec.Command("/bin/sh", "-c", cmd)
    var output bytes.Buffer

    stdout, err := command.StdoutPipe()
    if err != nil {
        return "", fmt.Errorf("Unable to setup stdout for command: %v", err)
    }
    go func() {
        if displayOutput == true {
            w := io.MultiWriter(os.Stdout, &output)
            io.Copy(w, stdout)
        } else {
            output.ReadFrom(stdout)
        }
    }()

    stderr, err := command.StderrPipe()
    if err != nil {
        return "", fmt.Errorf("Unable to setup stderr for command: %v", err)
    }
    go func() {
        if displayOutput == true {
            w := io.MultiWriter(os.Stderr, &output)
            io.Copy(w, stderr)
        } else {
            output.ReadFrom(stderr)
        }
    }()
    err = command.Run()
    if err != nil {
        return "", err
    }
    return output.String(), nil
}

amoebob · Accepted Answer · 2016-08-18 01:45:12Z

Here is a simplified and working revision of your example. Note that the test command was swapped out so that I could test within Windows and that your error checks have been omitted only for brevity.

The key change is that a sync.WaitGroup is preventing the run function from printing the output and returning until the goroutine has indicated that it's finished.

func main() {
    dir := "c:\\windows\\system32"
    command1 := exec.Command("cmd", "/C", "dir", "/s", dir)
    command2 := exec.Command("cmd", "/C", "dir", "/s", dir)
    out1, _ := run(command1)
    out2, _ := run(command2)
    log.Printf("Length [%d] vs [%d]\n", len(out1), len(out2))
}

func run(cmd *exec.Cmd) (string, error) {
    var output bytes.Buffer
    var waitGroup sync.WaitGroup

    stdout, _ := cmd.StdoutPipe()
    writer := io.MultiWriter(os.Stdout, &output)

    waitGroup.Add(1)
    go func() {
        defer waitGroup.Done()
        io.Copy(writer, stdout)
    }()

    cmd.Run()
    waitGroup.Wait()
    return output.String(), nil
}

Andy Schweig · Accepted Answer · 2016-08-18 00:17:38Z

0

I see some problems:

You should be waiting for the goroutines to finish (e.g., using sync.WaitGroup).
You're accessing output concurrently in two goroutines, which is not safe.

You could collect stdout and stderr in two separate buffers and return them separately, if that works for what you're trying to do.

edited Aug 18, 2016 at 0:17

answered Aug 18, 2016 at 0:05

Andy Schweig

6,7592 gold badges18 silver badges25 bronze badges

Collectives™ on Stack Overflow

Golang - Copy exec output to a buffer

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related