Add executable to Ubuntu-latest PATH using GitHub workflow

Question

This problem is reproduceable in any GitHub account using the minimal code given below.

The two files myapp.py and .github/workflows/myworkflow.yaml given below are all you need to reproduce this in minutes in any GitHub account.

What specific syntax needs to be changed below in order to get the .github/workflows/myworkflow.yaml GitHub workflow given below to successfully add myapp to the Ubuntu-latest PATH and then be able to use the myapp command?

BACKGROUND:

A GitHub workflow compiles a python app using pyinstaller and needs to add that compiled app to the PATH so that the myapp command can be used by subsequent jobs.

CURRENT ERROR:

The error we are currently getting is that the myapp --version step fails at the very first line with the following message:

/home/runner/work/_temp/long-alpha-numeric-string.sh: line 1: myapp: command not found
Error: Process completed with exit code 127.

This error shows that myapp has not been added to the PATH.

However, the logs show that a preceding command within the same step in the same job did successfully add the myapp folder to the /usr/local/bin directory and change its ownership.

COMPLETE LOGS:

The complete logs for the workflow are as follows:

Collecting pyinstaller
  Downloading pyinstaller-5.3-py3-none-manylinux2014_x86_64.whl (555 kB)
Requirement already satisfied, skipping upgrade: setuptools in /usr/lib/python3/dist-packages (from pyinstaller) (45.2.0)
Collecting altgraph
  Downloading altgraph-0.17.2-py2.py3-none-any.whl (21 kB)
Collecting pyinstaller-hooks-contrib>=2021.4
  Downloading pyinstaller_hooks_contrib-2022.10-py2.py3-none-any.whl (244 kB)
Installing collected packages: altgraph, pyinstaller-hooks-contrib, pyinstaller
Successfully installed altgraph-0.17.2 pyinstaller-5.3 pyinstaller-hooks-contrib-2022.10
49 INFO: PyInstaller: 5.3
49 INFO: Python: 3.8.10
62 INFO: Platform: Linux-5.15.0-1017-azure-x86_64-with-glibc2.29
62 INFO: wrote /home/runner/work/path/path/myapp.spec
300 INFO: UPX is available.
301 INFO: Extending PYTHONPATH with paths
['/home/runner/work/path/path']
502 INFO: checking Analysis
502 INFO: Building Analysis because Analysis-00.toc is non existent
502 INFO: Initializing module dependency graph...
503 INFO: Caching module graph hooks...
512 INFO: Analyzing base_library.zip ...
4127 INFO: Processing pre-find module path hook distutils from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
4127 INFO: distutils: retargeting to non-venv dir '/usr/lib/python3.8'
7332 INFO: Caching module dependency graph...
7458 INFO: running Analysis Analysis-00.toc
7494 INFO: Analyzing /home/runner/work/path/path/myapp.py
7497 INFO: Processing module hooks...
7497 INFO: Loading module hook 'hook-sysconfig.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7507 INFO: Loading module hook 'hook-encodings.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7671 INFO: Loading module hook 'hook-xml.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7721 INFO: Loading module hook 'hook-lib2to3.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7734 INFO: Loading module hook 'hook-heapq.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7736 INFO: Loading module hook 'hook-distutils.util.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7738 INFO: Loading module hook 'hook-pickle.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7740 INFO: Loading module hook 'hook-platform.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7741 INFO: Loading module hook 'hook-distutils.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7742 INFO: Loading module hook 'hook-difflib.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7743 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7744 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks'...
7749 INFO: Looking for ctypes DLLs
7810 INFO: Analyzing run-time hooks ...
7813 INFO: Including run-time hook '/home/runner/.local/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_subprocess.py'
7817 INFO: Looking for dynamic libraries
8271 INFO: Looking for eggs
8271 INFO: Python library not in binary dependencies. Doing additional searching...
8550 INFO: Using Python library /lib/x86_64-linux-gnu/libpython3.8.so.1.0
8554 INFO: Warnings written to /home/runner/work/path/path/build/myapp/warn-myapp.txt
8570 INFO: Graph cross-reference written to /home/runner/work/path/path/build/myapp/xref-myapp.html
8580 INFO: checking PYZ
8580 INFO: Building PYZ because PYZ-00.toc is non existent
8581 INFO: Building PYZ (ZlibArchive) /home/runner/work/path/path/build/myapp/PYZ-00.pyz
8753 INFO: Building PYZ (ZlibArchive) /home/runner/work/path/path/build/myapp/PYZ-00.pyz completed successfully.
8755 INFO: checking PKG
8755 INFO: Building PKG because PKG-00.toc is non existent
8755 INFO: Building PKG (CArchive) myapp.pkg
8781 INFO: Building PKG (CArchive) myapp.pkg completed successfully.
8782 INFO: Bootloader /home/runner/.local/lib/python3.8/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run
8782 INFO: checking EXE
8782 INFO: Building EXE because EXE-00.toc is non existent
8782 INFO: Building EXE from EXE-00.toc
8782 INFO: Copying bootloader EXE to /home/runner/work/path/path/build/myapp/myapp
8783 INFO: Appending PKG archive to custom ELF section in EXE
8826 INFO: Building EXE from EXE-00.toc completed successfully.
8828 INFO: checking COLLECT
8828 INFO: Building COLLECT because COLLECT-00.toc is non existent
8829 INFO: Building COLLECT COLLECT-00.toc
9418 INFO: Building COLLECT COLLECT-00.toc completed successfully.
........................About to ls -al /usr/local/bin 
total 1079472
drwxrwxrwx  3 root   root        4096 Aug 31 22:28 .
drwxr-xr-x 20 root   root        4096 Aug 28 22:36 ..
-rwxr-xr-x  1 root   root        3472 Aug 28 22:36 activate-global-python-argcomplete
-rwxrwxrwx  1 root   root    20702195 Aug 22 01:56 aliyun
lrwxrwxrwx  1 root   root          37 Aug 28 21:31 aws -> /usr/local/aws-cli/v2/current/bin/aws
lrwxrwxrwx  1 root   root          47 Aug 28 21:31 aws_completer -> /usr/local/aws-cli/v2/current/bin/aws_completer
-rwxrwxrwx  1 runner docker  27838676 Jul 21 16:31 azcopy
lrwxrwxrwx  1 root   root          21 Aug 28 21:22 azcopy10 -> /usr/local/bin/azcopy
lrwxrwxrwx  1 root   root          47 Aug 28 22:03 bazel -> ../lib/node_modules/@bazel/bazelisk/bazelisk.js
lrwxrwxrwx  1 root   root          47 Aug 28 22:03 bazelisk -> ../lib/node_modules/@bazel/bazelisk/bazelisk.js
lrwxrwxrwx  1 root   root          24 Aug 28 22:01 bcp -> /opt/mssql-tools/bin/bcp
-rwxrwxrwx  1 root   root    42040509 Aug 28 21:30 bicep
-rwxr-xr-x  1 root   root         539 Aug 28 22:12 bin-proxy
-rwxrwxrwx  1 root   root    13698488 Aug 17 17:47 ccmake
-rwxrwxrwx  1 root   root    13742840 Aug 17 17:47 cmake
-rwxrwxrwx  1 root   root    29159968 Aug 17 17:47 cmake-gui
-rwxr-xr-x  1 root   root         543 Aug 28 22:11 commander
lrwxrwxrwx  1 root   root          45 Aug 28 22:01 corepack -> ../lib/node_modules/corepack/dist/corepack.js
-rwxrwxrwx  1 root   root    14176408 Aug 17 17:47 cpack
-rwxrwxrwx  1 root   root    15153816 Aug 17 17:47 ctest
-rwxrwxrwx  1 root   root    12737304 Aug 28 21:16 docker-compose
-rwxr-xr-x  1 root   root         525 Aug 28 22:11 dotenv
-rwxr-xr-x  1 root   root         537 Aug 28 22:12 fastlane
lrwxrwxrwx  1 root   root          35 Aug 28 22:03 grunt -> ../lib/node_modules/grunt/bin/grunt
lrwxrwxrwx  1 root   root          36 Aug 28 22:03 gulp -> ../lib/node_modules/gulp/bin/gulp.js
-rwxrwxrwx  1 root   root    46374912 Aug 28 21:56 helm
lrwxrwxrwx  1 root   root          32 Aug 28 21:53 heroku -> /usr/local/lib/heroku/bin/heroku
-rwxr-xr-x  1 root   root         549 Aug 28 22:11 httpclient
-rwxrwxrwx  1 runner docker   9814937 Mar  5  2020 hub
-rwxrwxrwx  1 root   root     6658458 Aug 28 21:56 kind
-rwxrwxrwx  1 root   root    45002752 Aug 28 21:56 kubectl
-rwxrwxrwx  1 root   root    14381056 Aug 28 21:56 kustomize
-rwxrwxrwx  1 root   root       13700 Aug 28 21:56 lein
lrwxrwxrwx  1 root   root          32 Aug 28 22:03 lerna -> ../lib/node_modules/lerna/cli.js
-rwxrwxrwx  1 root   root    75566620 Aug 28 21:56 minikube
drwxr-xr-x  3 runner runner      4096 Aug 31 22:28 myapp
lrwxrwxrwx  1 root   root          27 Aug 28 22:03 n -> ../lib/node_modules/n/bin/n
lrwxrwxrwx  1 root   root          39 Aug 28 22:03 netlify -> ../lib/node_modules/netlify-cli/bin/run
lrwxrwxrwx  1 root   root          40 Aug 28 22:03 newman -> ../lib/node_modules/newman/bin/newman.js
-rwxrwxrwx  1 root   root    82214176 Aug 28 22:01 node
lrwxrwxrwx  1 root   root          21 Aug 28 22:03 now -> /usr/local/bin/vercel
lrwxrwxrwx  1 root   root          38 Aug 28 22:01 npm -> ../lib/node_modules/npm/bin/npm-cli.js
lrwxrwxrwx  1 root   root          38 Aug 28 22:01 npx -> ../lib/node_modules/npm/bin/npx-cli.js
lrwxrwxrwx  1 root   root          39 Aug 28 22:03 ntl -> ../lib/node_modules/netlify-cli/bin/run
-rwxrwxrwx  1 root   root   123873256 Aug 15 05:28 oc
-rwxr-xr-x  1 runner docker   8239688 Aug 22 08:44 oras
-rwxr-xr-x  1 root   root   156344320 Aug  2 10:03 packer
lrwxrwxrwx  1 root   root          37 Aug 28 22:03 parcel -> ../lib/node_modules/parcel/lib/bin.js
lrwxrwxrwx  1 root   root          59 Aug 28 22:04 phantomjs -> /usr/local/share/phantomjs-2.1.1-linux-x86_64/bin/phantomjs
-rwxr-xr-x  1 root   root     3211265 Aug 22 14:00 phpunit
-rwxr-xr-x  1 root   root         206 Aug 28 22:36 pipx
-rwxr-xr-x  1 runner docker  66364321 Aug 16 23:40 pulumi
-rwxr-xr-x  1 runner docker       245 Aug 16 17:01 pulumi-analyzer-policy
-rwxr-xr-x  1 runner docker      1853 Aug 16 17:01 pulumi-analyzer-policy-python
-rwxr-xr-x  1 runner docker  22150761 Aug 16 23:40 pulumi-language-dotnet
-rwxr-xr-x  1 runner docker  23434119 Aug 16 23:40 pulumi-language-go
-rwxr-xr-x  1 runner docker  22319202 Aug 11 21:20 pulumi-language-java
-rwxr-xr-x  1 runner docker  22267922 Aug 16 23:40 pulumi-language-nodejs
-rwxr-xr-x  1 runner docker  22220351 Aug 16 23:40 pulumi-language-python
-rwxr-xr-x  1 runner docker      6453 Aug 16 17:01 pulumi-language-python-exec
-rwxr-xr-x  1 runner docker  21643264 Aug 12 00:21 pulumi-language-yaml
-rwxr-xr-x  1 runner docker       238 Aug 16 17:01 pulumi-resource-pulumi-nodejs
-rwxr-xr-x  1 runner docker      1893 Aug 16 17:01 pulumi-resource-pulumi-python
-rwxr-xr-x  1 root   root        2555 Aug 28 22:36 python-argcomplete-check-easy-install-script
-rwxr-xr-x  1 root   root         383 Aug 28 22:36 python-argcomplete-tcsh
lrwxrwxrwx  1 root   root          36 Aug 28 22:45 pzstd -> /home/linuxbrew/.linuxbrew/bin/pzstd
-rwxrwxrwx  1 root   root      959664 Aug 28 21:41 rebar3
-rwxr-xr-x  1 root   root        1917 Aug 28 22:36 register-python-argcomplete
-rwxr-xr-x  1 root   root         523 Aug 28 22:11 rougify
lrwxrwxrwx  1 root   root          38 Aug 28 21:31 sam -> /usr/local/aws-sam-cli/current/bin/sam
lrwxrwxrwx  1 root   root          58 Aug 28 21:31 session-manager-plugin -> /usr/local/sessionmanagerplugin/bin/session-manager-plugin
lrwxrwxrwx  1 root   root          27 Aug 28 22:01 sqlcmd -> /opt/mssql-tools/bin/sqlcmd
lrwxrwxrwx  1 root   root          32 Aug 28 22:01 sqlpackage -> /usr/local/sqlpackage/sqlpackage
-rwxrwxrwx  1 root   root    79936320 Aug 28 21:53 stack
lrwxrwxrwx  1 root   root          30 Aug 28 21:34 swift -> /usr/share/swift/usr/bin/swift
lrwxrwxrwx  1 root   root          31 Aug 28 21:34 swiftc -> /usr/share/swift/usr/bin/swiftc
-rwxr-xr-x  1 root   root         591 Aug 28 22:11 terminal-notifier
-rwxr-xr-x  1 root   root    62963712 Aug 24 14:18 terraform
lrwxrwxrwx  1 root   root          38 Aug 28 22:03 tsc -> ../lib/node_modules/typescript/bin/tsc
lrwxrwxrwx  1 root   root          43 Aug 28 22:03 tsserver -> ../lib/node_modules/typescript/bin/tsserver
lrwxrwxrwx  1 root   root          37 Aug 28 22:45 unzstd -> /home/linuxbrew/.linuxbrew/bin/unzstd
-rwxr-xr-x  1 root   root         219 Aug 28 22:36 userpath
lrwxrwxrwx  1 root   root          40 Aug 28 22:03 vc -> ../lib/node_modules/vercel/dist/index.js
lrwxrwxrwx  1 root   root          28 Aug 28 22:23 vcpkg -> /usr/local/share/vcpkg/vcpkg
lrwxrwxrwx  1 root   root          40 Aug 28 22:03 vercel -> ../lib/node_modules/vercel/dist/index.js
lrwxrwxrwx  1 root   root          42 Aug 28 22:03 webpack -> ../lib/node_modules/webpack/bin/webpack.js
lrwxrwxrwx  1 root   root          42 Aug 28 22:03 webpack-cli -> ../lib/node_modules/webpack-cli/bin/cli.js
-rwxr-xr-x  1 root   root         529 Aug 28 22:11 ww
-rwxr-xr-x  1 root   root         543 Aug 28 22:11 xcodeproj
-rwxr-xr-x  1 root   root         537 Aug 28 22:11 xcpretty
-rwxr-xr-x  1 root   root         639 Aug 28 22:11 xcpretty-travis-formatter
lrwxrwxrwx  1 root   root          36 Aug 28 22:03 yarn -> ../lib/node_modules/yarn/bin/yarn.js
lrwxrwxrwx  1 root   root          36 Aug 28 22:03 yarnpkg -> ../lib/node_modules/yarn/bin/yarn.js
lrwxrwxrwx  1 root   root          35 Aug 28 22:45 zstd -> /home/linuxbrew/.linuxbrew/bin/zstd
lrwxrwxrwx  1 root   root          38 Aug 28 22:45 zstdcat -> /home/linuxbrew/.linuxbrew/bin/zstdcat
lrwxrwxrwx  1 root   root          39 Aug 28 22:45 zstdgrep -> /home/linuxbrew/.linuxbrew/bin/zstdgrep
lrwxrwxrwx  1 root   root          39 Aug 28 22:45 zstdless -> /home/linuxbrew/.linuxbrew/bin/zstdless
lrwxrwxrwx  1 root   root          37 Aug 28 22:45 zstdmt -> /home/linuxbrew/.linuxbrew/bin/zstdmt
........................About to ls -al /usr/local/bin/myapp 
total 11072
drwxr-xr-x 3 runner runner    4096 Aug 31 22:28 .
drwxrwxrwx 3 root   root      4096 Aug 31 22:28 ..
-rw-r--r-- 1 runner runner 1032366 Aug 31 22:28 base_library.zip
drwxr-xr-x 2 runner runner    4096 Aug 31 22:28 lib-dynload
-rwxr-xr-x 1 runner runner   74848 Aug 31 22:28 libbz2.so.1.0
-rwxr-xr-x 1 runner runner 2954080 Aug 31 22:28 libcrypto.so.1.1
-rwxr-xr-x 1 runner runner  182560 Aug 31 22:28 libexpat.so.1
-rwxr-xr-x 1 runner runner  162264 Aug 31 22:28 liblzma.so.5
-rwxr-xr-x 1 runner runner 5449112 Aug 31 22:28 libpython3.8.so.1.0
-rwxr-xr-x 1 runner runner  598104 Aug 31 22:28 libssl.so.1.1
-rwxr-xr-x 1 runner runner  108936 Aug 31 22:28 libz.so.1
-rwxr-xr-x 1 runner runner  739512 Aug 31 22:28 myapp
++++++++++++++++++++++++About to hash /usr/local/bin/myapp
------------------------About to hash /usr/local/bin/myapp.py
++++++++++++++++++++++++About to hash /usr/local/bin/myapp/myapp
------------------------About to hash /usr/local/bin/myapp/myapp.py
About to check version
/home/runner/work/_temp/484ed69e-d73c-41a7-9114-052661753f10.sh: line 21: myapp: command not found
Error: Process completed with exit code 127.

CURRENT WORKFLOW CODE:

The following workflow file named .github/workflows/myworkflow.yaml is all you need to reproduce this error in addition to the 3 line python program in the subsequent section below it.

name: add-to-path-ubuntu
on:
  push:
    branches:
      - main
jobs:
  create-exe-and-add-to-path:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - shell: bash
        name: Create exe and add exe to PATH
        run: |
          pip install -U pyinstaller
          pyinstaller myapp.py
          sudo cp -r dist/myapp /usr/local/bin
          sudo chown -R runner:runner /usr/local/bin/myapp
          echo "........................About to ls -al /usr/local/bin "
          ls -al /usr/local/bin
          echo "........................About to ls -al /usr/local/bin/myapp "
          ls -al /usr/local/bin/myapp
          echo "++++++++++++++++++++++++About to hash /usr/local/bin/myapp"
          hash /usr/local/bin/myapp
          echo "------------------------About to hash /usr/local/bin/myapp.py"
          hash /usr/local/bin/myapp.py
          echo "++++++++++++++++++++++++About to hash /usr/local/bin/myapp/myapp"
          hash /usr/local/bin/myapp/myapp
          echo "------------------------About to hash /usr/local/bin/myapp/myapp.py"
          hash /usr/local/bin/myapp/myapp.py
          sudo mkdir /home/runner/testdir
          sudo chown runner:runner /home/runner/testdir
          cd /home/runner/testdir
          echo 'About to check version'
          myapp --version

MINIMAL APP CODE:

You can reproduce this problem using the following python file named myapp.py and including the following 3 lines:

#!/usr/bin/env python
import sys
if sys.argv[1] == '--version':
  print("You are using version 1.0.")

@muru's SUGGESTION:

Here is the code that works based on @muru 's comment below, but we have some concerns about it that we will articulate below:

name: add-to-path-ubuntu
on:
  push:
    branches:
      - main
jobs:
  create-exe-and-add-to-path:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - shell: bash
        name: Create exe and add exe to PATH
        run: |
          pip install -U pyinstaller
          pyinstaller myapp.py
          cd dist/myapp
          sudo cp -R . /usr/local/bin/. 
          sudo chown -R runner:runner /usr/local/bin
          sudo mkdir /home/runner/testdir
          sudo chown runner:runner /home/runner/testdir
          cd /home/runner/testdir
          myapp --version

The main concerns we have about this approach are:

/usr/local/bin gets cluttered by all the dependencies of myapp which MUST be included in the same directory as myapp in order for myapp to function properly.
The entire /usr/local/bin is now owned by the runner user.

These concerns are partially mitigated by the fact that this is an ephemeral container whose life cycle begins and ends with the call from the GitHub workflow. But still, it seems messy.

Can you suggest a cleaner answer?

Does your myapp file have #!/path/to/python or #!/usr/bin/env python as the first line? Running just the check-myapp-version job works later? — Sotto Voce
– Sotto Voce, Commented Aug 31, 2022 at 20:50
@SottoVoce I just added the results of testing your approach to the OP. You can see that the hash commands fail to return anything, even though you can see that the ls -al commands show that the files have been moved. And also, the sequence should be assured because steps in GitHub workflows run sequentially. What else do you suggest? — CodeMed
– CodeMed, Commented Aug 31, 2022 at 21:21
@SottoVoce I added complete instructions including minimal code to reproduce this in any GitHub account within minutes. Are you willing to use the new code in the revised OP to noodle with this and suggest the answer? You can see the problem persists even with #!/usr/bin/env python at the start of myapp.py — CodeMed
– CodeMed, Commented Aug 31, 2022 at 23:21
@ CodeMed, with the myapp file created in /usr/local/bin, does an interactive login of this user to the target server find the app successfully by typing myapp? I see that the Github Actions job can't find it, but it will be helpful to know a normal login by the same user can find it. Also, what version of Ubuntu is on the target server, and which shell is invoked for this user? — Sotto Voce
– Sotto Voce, Commented Sep 1, 2022 at 0:18
@SottoVoce There is no interactive login for GitHub runners, but you can reproduce the error in minutes using the code in the OP if you have a GitHub account. The version of Ubuntu is Ubuntu 20.04.4 LTS, The result of echo $SHELL is /bin/bash . What else do you suggest? — CodeMed
– CodeMed, Commented Sep 1, 2022 at 1:17

Sotto Voce · Accepted Answer · 2022-09-04 06:07:26Z

Regarding your final question after @muru's comment, my suggestion is to follow the pattern that's visible in the /usr/local/bin/aws and /usr/local/bin/heroku commands in your question's directory listing. These commands are symbolic links to files that are installed along with their dependencies in places other than /usr/local/bin. For the aws command, the file+dependencies are installed under /usr/local/aws-cli/, and for heroku they're installed under /usr/local/lib/heroku/.

Following that pattern would involve picking a directory name for your app's command + dependencies, such as /usr/local/lib/myapp, copying the files there, and creating a symbolic link in /usr/local/bin to the executable script in that directory. Your app's executable and dependencies can exist in the same directory together, yet they don't clutter /usr/local/bin.

Stack Exchange Network

Add executable to Ubuntu-latest PATH using GitHub workflow

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Add executable to Ubuntu-latest PATH using GitHub workflow

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions