How to execute python scripts in their source folder when using colcon?

I've just started using colcon to build a bunch of ROS packages, mixed python/C++.

colcon moves the built packages to the install folder, including the python files. After sourcing install/setup.bash I can roslaunch my python nodes. However, the python files being executed are the ones copied to the install folder, so if I want to attach a debbuger to the running python code I have to do it with the files copied to the install folder. This is a big problem for me, because I'm constantly changing the python code while debugging and I want to do that in the source folder, not in the install folder.

I have tried to use colcon build with --symlink-install but it still copies the python files.

Is there any way to execute the python files in the source folder?

UPDATE:

I haven't got to work --symlink-install as I would expect yet. So I'll give a bit more of detail on what I've tried so far:

first: I use Ubuntu 18.04 with ROS Melodic. the colcon version is:

colcon version-check
colcon-argcomplete 0.3.3: up-to-date
colcon-bash 0.4.2: up-to-date
colcon-cd 0.1.1: up-to-date
colcon-cmake 0.2.18: up-to-date
colcon-core 0.5.3: up-to-date
colcon-defaults 0.2.3: up-to-date
colcon-devtools 0.2.2: up-to-date
colcon-library-path 0.2.1: up-to-date
colcon-metadata 0.2.3: up-to-date
colcon-notification 0.2.12: up-to-date
colcon-output 0.2.7: up-to-date
colcon-package-information 0.3.1: up-to-date
colcon-package-selection 0.2.5: up-to-date
colcon-parallel-executor 0.2.4: up-to-date
colcon-pkg-config 0.1.0: up-to-date
colcon-powershell 0.3.6: up-to-date
colcon-python-setup-py 0.2.3: up-to-date
colcon-recursive-crawl 0.2.1: up-to-date
colcon-ros 0.3.15: up-to-date
colcon-test-result 0.3.8: up-to-date
colcon-zsh 0.4.0: up-to-date

As my python node is very short I initially tried to install it as a script: My directory structure is:

workspace
    src
        gazebo_utils
            launch
                node.launch
           meshes
           urdf
           src
                python_node.py
           CMakeList.txt
           package.xml

My CMakeLists.txt is:

cmake_minimum_required(VERSION 3.5)
project(gazebo_utils)

find_package(catkin REQUIRED)
catkin_package()
catkin_install_python(PROGRAMS
    src/python_node.py
    DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)

install( 
    DIRECTORY launch urdf meshes 
    DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)

The package.xml is minimal with only catkin as buildtool_depend

I build with: colcon build --symlink-install --packages-select gazebo_utils

I source . install/setup.bash and I can roslaunch gazebo_utils node.launch successfully and the node works. However, what has been installed and is executed is:

workspace
   install
       gazebo_utils
          lib
             gazebo_utils
                python_node.py  (this is a copy not a symlink)

and my python code is a copy no a symlink.

Then I tried to put the code into a python package: I use this directory structure:

workspace
    src
        gazebo_utils
            launch
                node.launch
           meshes
           urdf
           src
                gazebo_utils
                    __init__.py
                    python_node.py
           CMakeList.txt
           package.xml   
           setup.py

init.py is empty

The CMakeLists.txt now is:

cmake_minimum_required(VERSION 3.5)
project(gazebo_utils)

find_package(catkin REQUIRED)
catkin_package()
catkin_python_setup()

install( 
    DIRECTORY launch urdf meshes 
    DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)

and setup.py is:

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

# fetch values from package.xml
setup_args = generate_distutils_setup(
    packages=['gazebo_utils'],
    package_dir={'': 'src'})

setup(**setup_args)

I build with: colcon build --symlink-install --packages-select gazebo_utils which installs:

install
    gazebo_utils
        lib
            python2.7
                dist-packages
                    gazebo_utils
                        python_node.py     (this is a copy not a symlink)
                    gazebo_utils-0.0.0.egg-info

again my python code is a copy no a symlink, and now I haven't found a way of launching this python_node.py from node.launch. Using used to work with a catkin workspace but now it doesn't.

Finally, I tried to add a script that calls the code in the python package:

my directory structure is:

workspace
    src
        gazebo_utils
            launch
                node.launch
           meshes
           urdf
           nodes
                python_node.py
           src
                gazebo_utils
                    __init__.py
                    python_code.py
           CMakeList.txt
           package.xml   
           setup.py

python_node.py is:

#! /usr/bin/env python
from gazebo_utils.python_code import main

if __name__ == '__main__':
    main()

and CMakeLists.txt is:

cmake_minimum_required(VERSION 3.5)
project(gazebo_utils)

find_package(catkin REQUIRED)
catkin_package()
catkin_python_setup()
catkin_install_python(PROGRAMS
    nodes/python_node.py
    DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)

install( 
    DIRECTORY launch urdf meshes 
    DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)

I build with colcon build --symlink-install --packages-select gazebo_utils and now after sourcing . install/setup.bash I can successfully launch the node and it works well.

Still the python files are copies. This is the resulting directory structure is the install folder:

   workspace
        install
            gazebo_utils
                lib
                    gazebo_utils
                        python_node.py  (this is a copy not symlink)
                    python2.7
                        dist-packages
                            gazebo_utils
                                python_code.py    (this is a copy not symlink)
                        gazebo_utils-0.0.0.egg-info      

I don't know if I have to use different boilerplate code for the setup.py or CMakeList.txt. This is what I've found online so far.

Any help would be greatly appreciated!

Originally posted by martinakos on ROS Answers with karma: 38 on 2020-02-19

Post score: 0

Original comments

Comment by Dirk Thomas on 2020-02-19:\

I have tried to use colcon build with --symlink-install but it still copies the python files.

I don't think that is the case. In the build directory the Python sources are symlinked and in the install directory only an .egg-link file is created which references the symlinks in the build directory. You might want to make sure that you are using the latest state of the tools, retry with a clean workspace, and if the problem persists share exact steps to reproduce your problem.

Comment by martinakos on 2020-02-20:
I see there are some symlinks in the build folder but not for my python file. In the CMakeLists for the ros package I use catkin_install_python() with my python file (just one file for the moment) and that file has been copied (not symlinked) both to install and build folders. If the file had been symlinked to the build folder how would I launch that ros node? what setup.bash should I source?