Getting git fetch output to file through python

Question

I am trying to save git fetch output to file through python, using:

subprocess.check_output(["git", "fetch", "origin", ">>", "C:/bitbucket_backup/backup.log", "2>&1"], cwd='C:/bitbucket_backup/loopx')

but I believe there is something missing in subprocess.check_output args because when adding >> C:/bitbucket_backup/backup.log 2>&1 I receive this error:

Traceback (most recent call last):
  File "<pyshell#28>", line 1, in <module>
     subprocess.check_output(["git", "fetch", "origin", ">>", "C://bitbucket_backup//backup.log", "2>&1"], cwd='C://bitbucket_backup//loopx')
  File "C:\Users\fabio\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 336, in check_output
     **kwargs).stdout
  File "C:\Users\fabio\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 418, in run
     output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'fetch', 'origin', '>>', 'C://bitbucket_backup//backup.log', '2>&1']' returned non-zero exit status 128.

Don't forward STDOUT/STDERR as the part of the command, that's not how it's meant to be run from Python. subprocess.check_output will automatically pipe your STDOUT, and if you're expecting an error use subprocess.communicate() instead to capture both streams. You can use subprocess.Popen() to fully control the piping, including automatic piping to a file handle if you want to save the output as a file. — zwer
– zwer, Commented Jul 25, 2017 at 14:44
to complete what zwer just said, your command line would need shell=True (but that's dirty :)) — Jean-François Fabre
– Jean-François Fabre ♦, Commented Jul 25, 2017 at 14:44
It's also not a great idea to depend on git fetch's output: it's not designed for machine parsing. Moreover it behaves differently if stdout is a pipe vs a tty. (There are some flags to try to control this but I ran into a bug trying to use them to show progress in parallel fetching.) — torek
– torek, Commented Jul 25, 2017 at 14:58
I know, I had to do some work arounds to be able to depend on git fetch. — marafado88
– marafado88, Commented Jul 25, 2017 at 15:11

Jean-François Fabre · Accepted Answer · 2017-07-25 15:10:56Z

2

Quickfix: enable shell features to handle redirection arguments:

subprocess.check_output(["git", "fetch", "origin", ">>", "C:/bitbucket_backup/backup.log", "2>&1"], cwd='C:/bitbucket_backup/loopx', shell=True)

But that's really dirty as python is able to do that really nicely:

output = subprocess.check_output(["git", "fetch", "origin"], stderr=subprocess.STDOUT, cwd='C:/bitbucket_backup/loopx')
with open("C:/bitbucket_backup/backup.log","ab") as f:  # append to file
    f.write(output)

That said, if you're to rewrite all git commands in python, maybe you should use a git python API like GitPython for instance.

edited Jul 25, 2017 at 15:10

answered Jul 25, 2017 at 14:47

Jean-François Fabre♦

141k24 gold badges179 silver badges246 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

marafado88 Over a year ago

Thanks for the solution and also the "better" alternative eheh =P

marafado88 Over a year ago

also with the "better" alternative I had to do f.write(str(output)) to be able to write in backup.log, but through that it lost line breaks from the default git fetch formatation output.

Jean-François Fabre Over a year ago

check my edit. Just open the file in binary mode. that should do the trick instead of converting to string.

Collectives™ on Stack Overflow

Getting git fetch output to file through python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related