How to override pipeline's parameters in Azure DevOps during the release?

Question

I have specified parameters in some pipelines in Azure Synapse Analytics, like this:

The value of the variables is different in Production and Pre-Production's environments. During the release in Azure DevOps I want default values of the parameters to stay as they are in Pre-Production but be updated in Production's environment.

In Azure DevOps > Pipelines > Releases > Deployment process > Agent jobs > Synpase deployment task for workspace > Override Parameters, I have added needed parameters together with $workspace parameter, it looks like:

Questions:

If I specify parameters, which have to be overridden this way they are not being updated after the release is finished. How do I have to specify the parameters (from the pipelines in Azure Synapse Analytics) then?
Is there a safer way how to specify storage key inside the pipeline and how to update it during the release (instead of specifying it so explicitly)?

An update after the comment:

TemplateForWorkspace.json

{
"$schema": "http://schema.management.azure.com/..",
"contentVersion": "1.0.0.0",
"parameters": {
    "workspaceName": {
.......
}
    "resources": {
.......
        "name": "PL_example",
        "type": "Microsoft.Synapse/workspaces/pipelines",
        "apiVersion": ".",
        "properties": {
            "description": ".",
            "activities": [
                {
                    "name": "Notebook",
                    "type": "SynapseNotebook",
                    "dependsOn": [..],
                    "policy": {..},
                    "userProperties": [],
                    "typeProperties": {
                        "notebook": {
                            "referenceName": "Notebook",
                            "type": "NotebookReference"
                        },
                        "parameters": {
                            "url": {
                                "value": {
                                    "value": "@pipeline().parameters.URL",
                                    "type": "Expression"

}
......
                "parameters": {
                "URL": {
                    "type": "string",
                    "defaultValue": "jdbc:sqlserver://..."
                },

}

In the end of the script it is stated as default value, which has to be updated after the release with the value in override parameters. but it stays the same value.

I think that after the release the parameter should be added to the beginning of TemplateForWorkspace.json script in the section parameters, but it does not, how do I do it? Am I doing something wrong?

Taking into account that I have added correct values (for the production, which should be updated after the release for the necessary parameters in following locations:

Azure DevOps > Pipelines > Releases > Deployment process > Tasks > Agent jobs > Synpase deployment task for workspace > Override Parameters
Azure DevOps > Pipelines > Releases > Deployment process > Variables > Pipeline variables

Per your 2nd question, would you consider defining secret variables in your release pipeline? — Alvin Zhao
– Alvin Zhao, Commented Sep 5, 2024 at 2:38
Regarding the issue you mentioned that Synapse pipeline parameters were not updated after release finished, please share your workspace JSON template and the YAML context of your Synpase deployment task for workspac step to understand if there was any ARM template parameter defined for your synapse pipeline parameters to be overridden for another environment. — Alvin Zhao
– Alvin Zhao, Commented Sep 5, 2024 at 2:41
Hi @Eli, May I know if you have got a chance to check my answer below to use custom parameters in the workspace template? Hope it could resolve your issue in this post. Thx for your time and efforts. — Alvin Zhao
– Alvin Zhao, Commented Sep 19, 2024 at 7:38

Alvin Zhao · Accepted Answer · 2024-09-05 07:31:06Z

0

To better align this post with the topic, it is recommended to include the azure-synapse tag, as the primary focus is on how to create custom parameters in the workspace template for parameterizing Synapse Pipeline features and settings that are not covered by the default deployment parameters template.

For more detailed guidance, please refer to this resource on Azure Synapse Analytics CI/CD with Custom Parameters - Made Easy! - Microsoft Community Hub.

In general, to expose Synapse pipeline parameters in the collaboration branch (main), you need to create a template-parameters-definition.json file that specifies the Synapse pipeline parameters. Once published, the new Synapse pipeline parameters should appear in the TemplateForWorkspace.json template within the publish branch (workspace_publish).

Here is an example of the template-parameters-definition.json file:

{
    "Microsoft.Synapse/workspaces/pipelines": {
        "properties": {
            "parameters": {
                "URL": {
                    "defaultValue": "="
                },
                "BlobStorageKey": {
                    "defaultValue": "="
                },
                "Server": {
                    "defaultValue": "="
                },
                "TempFolder": {
                    "defaultValue": "="
                }
            }
        }
    }
}

In the release pipeline, ensure to replace the ARM template parameters with the corresponding pipeline variables' values, rather than directly using the Synapse pipeline parameter names. These pipeline variables can also be configured as secret variables.

Images illustrating these steps are provided below.

answered Sep 5, 2024 at 7:31

Alvin Zhao

6,6831 gold badge7 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Eli Over a year ago

Thanks for a descriptive solution, I followed all the steps, added Variables (in Azure DevOps - Pipelines - Releases - Edit - Variables), also in Override variables, the variables were created in TemplateforWorkspace.json, but not were not updated with Production's values, despite the release was successful

Alvin Zhao Over a year ago

Can you edit your original post and share the TemplateforWorkspace.json from workspace_publish branch and your updates in the release pipeline? In you current post, your release pipeline seems to be using incorrect parameter names to override. Don't forget to remove or scrub the sensitive information in the contents and screenshots.

Eli Over a year ago

Zhao I have added the TemplateforWorkspace.json, could you please have a look? Thank you very much for the help

Alvin Zhao Over a year ago

Hi @Eli, From your TemplateforWorkspace.json that was added, I didn't see if there is any template parameters defined in your parameters or resources nodes. Please make sure you have followed the document I suggested to generate ARM template parameters which should be under the parameters node at the first place as where the key of workspaceName located. See the screenshot of my TemplateforWorkspace.json file to help understand the correct context. Thx.

Collectives™ on Stack Overflow

How to override pipeline's parameters in Azure DevOps during the release?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related