How to create a key-value map with pipeline expression (not data flow expression) in Azure Data Factory (Synapse Studio)

Question

I have two arrays that have respectively contain keys and values: array 1 ["key1", "key2", "key3"] and array 2 ["value1", "value2", "value3"]

With ADF data flow expression, I can construct a key-value map with these two arrays using keyValues function:

keyValues(["key1", "key2", "key3"],["value1", "value2", "value3"])

And this will return a result like this:

["key1" -> "value1", "key2" -> "value2", "key3" -> "value3"]

I want to construct the same key-value map within a pipeline, not data flow, but the keyValues function is not available in pipeline expression. How can I construct a key-value map with pipeline expression on Azure Data Factory / Synapse Studio?

Similar concept, close enough? stackoverflow.com/a/69286004/1527504 — wBob
– wBob, Commented Nov 16, 2022 at 19:31

Saideep Arikontham · Accepted Answer · 2022-11-17 06:18:04Z

1

There is no direct way to create a key value pair using pipeline expression builder as in dataflows. The following is one to build the required key value pairs in ADF pipelines:

I have taken 2 parameters with the following values:

enter image description here

Let's say req is the variable in which we would like to store our final key value pair. I have initialized it with the value {}

enter image description here

In for each, I have used items value as @range(0,length(pipeline().parameters.keys)) to generate index.

enter image description here

Now, I have taken a variable called temp to apply union on the current value of req and the dynamically built key-value pair for current iteration.

@string(union(json(variables('req')),json(concat('{"',pipeline().parameters.keys[item()],'":"',pipeline().parameters.values[item()],'"}'))))

enter image description here

Finally, I am updating the value of req variable for each iteration with the current temp value i.e., @variables('temp'):

enter image description here

After running the pipeline, it would generate the following output in req variable:

NOTE: Object type variables are not supported in ADF pipelines. Whenever you want to use this object (stored as string), you can convert it to an object type using @json() function

answered Nov 17, 2022 at 6:18

Saideep Arikontham

6,1922 gold badges6 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

xiexieni9527 Over a year ago

Tried it out and got the map, thank you very much!

Collectives™ on Stack Overflow

How to create a key-value map with pipeline expression (not data flow expression) in Azure Data Factory (Synapse Studio)

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related