Executing python script using Jenkins parameterized pipeline

Question

I have set up a Jenkins parameterized job to execute python script using execute shell feature in Jenkin. The job has the following parameters:

user-name: string, order_area_name: comma-separated strings, country_name: string, country_code: string, and so on...

My use case is to split the order_area_name and execute the python script for every order_area_name sequentially. So, I wrote a script that looks something like this:

#!/bin/bash
export PYTHONHASHSEED=0
empty_string=""
parameters_list=""
IFS=","

#Checking every parameter if it is present or not
if [ "$user_name" != "$empty_string" ]
then
    parameters_list=$parameters_list" --user "$user_name
fi
if [ "$country_code" != "$empty_string" ]
then
    parameters_list=$parameters_list" --country_code "$country_code
fi
if [ "$country_category" != "$empty_string" ]
then
    parameters_list=$parameters_list" --country_category "$country_category
fi

parameters_list=$parameters_list" --aws_access_key_id "$aws_access_key_id
parameters_list=$parameters_list" --aws_secret_access_key "$aws_secret_access_key


##Checking if the parameter is present then splitting the string and storing it into array
##Then for each order_area_name executing the python script in sequential manner
if [ "$order_area_names" != "$empty_string" ]
then
    read -r -a order_area_name_array <<< "$order_area_names"
    for order_area in "${order_area_name_array[@]}";
    do      
        final_list=$parameters_list" --order_area_name "$order_area
        echo $final_list
        python3 ./main.py ${final_list}
    done
fi

exit

I am not able pass the final_list of values to the python script because of which the Jenkin job is failing. If I echo the final_list I see that the values are properly getting initialized:

--user [email protected] --mqs_level Q2 --num_parallel_pipelines 13 --sns_topic topicname --ramp_up_time 45 --max_duration_for_task 30 --batch_size 35 --lead_store_db_schema schema --airflow_k8s_web_server_pod_name airflow-web-xyz --aws_access_key_id 12345678 --aws_secret_access_key 12345678 --order_area_name London

The error looks something like this:

main.py: error: the following arguments are required: --user, --sns_topic, --aws_access_key_id, --aws_secret_access_key Build step 'Execute shell' marked build as failure Finished: FAILURE

I searched for this at a lot of places but didn't get any concrete answer for this. Could anyone please help me with this?

read stackoverflow.com/a/44055875/9072753 and research bash arrays. Research quoting in shell and check your scripts with shellcheck.net — KamilCuk
– KamilCuk, Commented May 13, 2021 at 9:43
@KamilCuk Sorry for the little delay in response, but your comments really helped me. I found my answer from the link mentioned by you, I used something like: echo $command | bash and it worked. — Jay
– Jay, Commented May 16, 2021 at 8:41
echo $final_list | bash Uch, that's horrible, it's as bad as eval. Use bash arrays - cmd=(something -param); "${cmd[@]}" — KamilCuk
– KamilCuk, Commented May 16, 2021 at 14:29

Jay · Accepted Answer · 2021-05-16 08:46:28Z

1

Instead of writing my command like:

python3 ./main.py ${final_list}

I used this and it worked very well:

echo $final_list | bash

"final_list" variable has the command which needs to be executed.

answered May 16, 2021 at 8:46

Jay

1311 gold badge4 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Executing python script using Jenkins parameterized pipeline

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related