R: system() cannot use bash function defined in .bashrc

Question

Question

My default Python is 2.7, but I have a script that requires Python 3.4. I am trying to create a function in R that will:

Switch to Python 3.4
Run this script
Switch back to Python 2.7
Import results into R

To switch between Python versions, I use my cluster's "dotkit" system, like this:

use Python-2.7
use Python-3.4

"use" is a bash function that is imported in my .bashrc file. It sets all of my path variables (PATH, LIBRARY_PATH, LD_LIBRARY_PATH, CPATH, C_INCLUDE_PATH, etc). The problem is that when I try to call this function in R, I get the following error:

system('use Python-3.4')
sh: use: command not found

It seems like this is a problem with my PATH. I am using the correct shell:

system('echo $SHELL')
/bin/bash

My $PATH variable also looks good. However, when I create a script that essentially does the same thing:

load_py34.sh:

#!/bin/bash
source ~/.bashrc
use Python-3.4

and call this script through R, then it actually runs, but for some reason, it doesn't change my python version within R. (I have verified that this script works from the command line.)

> R
> system('python --version')
Python 2.7.1
> system('sh load_py34.sh')
Prepending: R-3.4 (ok)
> system('python --version')
Python 2.7.1

So I'm a little confused, but if anyone can help, I would really appreciate it.

Suggested fixes

When I combine them into a single command, I still have the same problem:

> system("sh load_py34.sh; python --version")
Prepending: Python-3.4 (already loaded)
Python 2.7.1

When I try calling bash directly, I still have a problem with the PATH:

> system("bash -c 'use Python-3.4; python --version'")
bash: use: command not found
Python 2.7.1

When I combine them in a single call, I get the same error. I will update the question to reflect this. — adn bps
– adn bps, Commented Oct 10, 2017 at 19:04

paladin324 · Accepted Answer · 2021-10-10 19:41:48Z

.bashrc is only loaded for interactive bash sessions.

"use" is a bash function that is imported in my .bashrc file. It sets all of my path variables.

If set via export, the environment of the calling process will not be altered.

export [-fn] [name[=word]] ... The supplied names are marked for automatic export to the environment of subsequently executed commands. (https://man7.org/linux/man-pages/man1/bash.1.html)

Child processes do not normally have access to the parent process' environment. (This poses a problem because system() creates a sub-process.)

The source and . built-ins execute the commands in the current shell environment, hence why your script works.

Other commands (executables, non-shell-builtins) are executed by the fork-and-exec mechanism, whereby the executing shell process forks, creating a child process with an identical environment and state. This new child process is the process in which the command is executed. Changes to the environment of that process are not replicated to the parent's environment.

This means that you will not be able to rely on system('...') to modify the environment of the R process, or that of processes spawned by subsequent system() invocations.

In a single invocation to system(), you can construct a command-line that changes the environment of the spawned shell like so:

bash -c 'source ~/.bashrc; use Python-3.4; python --version'

Mind you, ~/.bashrc is not really the best place to put this functionality (might be subjective).

N. Kern · Accepted Answer · 2017-10-10 19:06:19Z

0

When you call system() it uses /bin/sh, not /bin/bash. sh doesn't read your .bashrc file when it starts up, so it does not know any of the functions you've defined there.

To use the function from your .bashrc, you must get bash to run it instead:

system("bash -c 'use Python-3.4; python --version'")

Edit: placement of closing single quote.

edited Oct 10, 2017 at 19:06

answered Oct 10, 2017 at 18:57

N. Kern

4112 silver badges7 bronze badges

1 Comment

adn bps Over a year ago

This gives me the same error (see updated question).

Collectives™ on Stack Overflow

R: system() cannot use bash function defined in .bashrc

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related