How to trigger pipeline by Azure create run pipelines API?

Question

I have an Azure data factory subscription, I want to trigger pipeline through my node.js application. The pipeline transfers data from azure cosmos to Azure blob storage. I have triggered the pipeline once manually. And I was trying to do the same with a node.js app, but it throws the following error:

{ Error: tunneling socket could not be established, cause=connect ETIMEDOUT 10.0.0.2:8080
    at ClientRequest.onError (/home/deepti/Desktop/dbmanager/node_modules/tunnel-agent/index.js:177:17)
    at Object.onceWrapper (events.js:286:20)
    at ClientRequest.emit (events.js:198:13)
    at Socket.socketErrorListener (_http_client.js:392:9)
    at Socket.emit (events.js:198:13)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at process._tickCallback (internal/process/next_tick.js:63:19) code: 'ECONNRESET' }

And here's my code in node.js app :

Request.post("POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines/{pipelineName}/createRun?api-version=2018-06-01",(err,res,body)=>{
        if(err) {
            return console.log(err);
        }
        console.dir(JSON.parse(body));
        });

Can anyone please guide me how do I rectify this error and proceed further. Do I need to generate some authorization token too?

Also, I actually want to back up data that is 1 month old from cosmos DB to Azure Blob, is that possible?

Jay Gong · Accepted Answer · 2019-06-28 06:55:25Z

1

1.Can anyone please guide me how do I rectify this error and proceed further. Do I need to generate some authorization token too?

Yes,before process of creating pipeline run , you need to generate Authorization token in Headers. You could refer to ADAL nodejs sdk to generate token. If you don't know what are these parameters:

,then please refer to this tutorial and this blog to get concepts of them. And please note that you also need to grant ADFpermission to your ad app.

Finally,you could call the Pipelines - Create Run REST api and provide auth token in the header.

2.Also, I actually want to back up data that is 1 month old from cosmos DB to Azure Blob, is that possible?

If you want to back up azure blob storage data,please refer to this link.You could use AzCopy in the schedule to back up the data during the stage.

answered Jun 28, 2019 at 6:55

Jay Gong

23.8k2 gold badges30 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to trigger pipeline by Azure create run pipelines API?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related