Triggering a Data Factory Pipeline through javascript

Question

I have made this pipeline in Azure Data Factory that copies data from a Azure Storage Table to a Azure SQL database Table.

The Azure storage table is given data from a Javascript chatbot that records answers and stores them in the table. I would like to trigger the CopyTabletoSQL through my javascript app once all of the answers have been recorded.

This is my CopyTableToSQL pipeline object.

{
"name": "CopyTabletoSQL",
"type": "Copy",
"policy": {
    "timeout": "7.00:00:00",
    "retry": 0,
    "retryIntervalInSeconds": 30,
    "secureOutput": false
},
"typeProperties": {
    "source": {
        "type": "AzureTableSource"
    },
    "sink": {
        "type": "SqlSink",
        "writeBatchSize": 10000
    },
    "enableStaging": false,
    "dataIntegrationUnits": 0
},
"inputs": [
    {
        "referenceName": "tableInputDataset",
        "type": "DatasetReference"
    }
],
"outputs": [
    {
        "referenceName": "OutputSqlTable1",
        "type": "DatasetReference"
    }
]
}

Is there any way to have this execute from a javascript app? The docoumentation (https://learn.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers) only mentions .net, Powershell, REST API and Python SDK but nothing for node.js

I have managed to use learn.microsoft.com/en-us/rest/api/datafactory/pipelines/… to create a run and try it and it was succesfull. POST -URL- Authorization: Bearer -random symbols- Content-type: application/json How would I go about implementing this into my javascript app so that I can call on it at the end of a run of my app? Many thanks! — BergBerg
– BergBerg, Commented Jul 13, 2018 at 8:48

Jay Gong · Accepted Answer · 2018-07-13 08:34:04Z

1

Azure Data Factory nodejs sdk is not supported so far. Based on your description, you already have created pipeline in your adf account.You could use Azure Data Factory Pipelines - Create Run REST api to execute it.

In this process, you need to generate Authorization token in Headers. You could refer to ADAL nodejs sdk to generate token.

Before that, you need to grant adf permission to your ad app.

Hope it helps you.

answered Jul 13, 2018 at 8:34

Jay Gong

23.8k2 gold badges30 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

BergBerg Over a year ago

I have managed to use learn.microsoft.com/en-us/rest/api/datafactory/pipelines/… to create a run and try it and it was succesfull. POST -URL- Authorization: Bearer -random symbols- Content-type: application/json How would I go about implementing this into my javascript app so that I can call on it at the end of a run of my app? Many thanks!

Jay Gong Over a year ago

@BergBerg You could send post request with Authorization token in your js app. About generating token, please refer to the sdk mentioned in my above answer.

BergBerg Over a year ago

Thank you for all your help/trouble! I found this: pastebin.com/urtabdvL from here stackoverflow.com/questions/14873443/… Everything seems to be fine however, I am not sure what to write in var postData = "Some data"; I'm guessing it is somehow related to the JSON body? also would I need to add Authorization: Bearer -random symbols- somehow? I will look at the SDK you mentioned regarding how to generate a token! Many Thanks!

Fang Liu Over a year ago

For the post data, you could just reference what UI sent out. I think an empty object {} should work if you don’t need pass parameter value.

BergBerg Over a year ago

For some added context: This is how my app looks like right now, pastebin.com/fbp3ddzD . I've built it following guides and how to's from the microsoft documents and some googleing/ stackoverflow. It is a simple chatbot that asks 3 questions, records the answers and then I have a pipeline that copies the answers from my table to a SQL database. Everything works fine, the bot asks and records the answers, and the pipeline copies them to my SQL database, I'm looking for a way to trigger the copy from table to database automaticly once a conversation have been finished.

Fang Liu · Accepted Answer · 2018-07-12 14:05:30Z

0

You could call rest api in JavaScript.

answered Jul 12, 2018 at 14:05

Fang Liu

2,3612 gold badges15 silver badges18 bronze badges

Collectives™ on Stack Overflow

Triggering a Data Factory Pipeline through javascript

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related