1

I have to GET data from a REST API and POST it in a sql database.

I know how to GET data from an API and displaying it in a Web-service, and I know how to POST data to a database, from a Web-service, using azure functions.

The problem is that I don't know how to GET the data from an API and directly POST it in a database. NOTE: It's quite a lot of data.

enter image description here

UPDATE: I use C# (.NET?) and SQL. The API provides JSON result.

Improve this question
3
  • Please be more specific about your problem. A code example would help. Commented Mar 13, 2018 at 16:00
  • What programming language do you use to get data from an API? Commented Mar 13, 2018 at 16:02
  • I updated the description, please let me know if I should mention anything else. It would be great if I could get the code to GET the data and POST it to the database, but my main issue is that I don't know conceptually how this operation is done: get data from an API and post it in a database. Commented Mar 14, 2018 at 9:38

1 Answer 1

Reset to default
2

Azure function is basically serverless architecture. It is almost same as if write c# code for consuming json & posting to sql server but instead of hosting it as website/webapi for public consumption you get hosting at azure without having a Virtual machine having IIS/webserver.

  1. For Creating azure function refer https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-serverless-api

  2. you can consume another web API inside function like :

    var client = new HttpClient();

    var response = await client.GetAsync(url);

    var content = await response.Content.ReadAsStringAsync();

  3. Simple ado.net adapter/code for posting/writing data to sql server.

just like consuming any simple api inside c# function in webapp or console application.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thanks. It helped me do what I needed to do, but it's still not addressing the issue of "DIRECTLY" taking data from the api to the database. It still needs to store the data taken from the api, into a variable. So if I have to select for example 50GB of data from the api, I would need at least 50GB of RAM, because the variable will have to store that data, before it's posting it in the database.
That's why broker architecture like queue, message queue etc for that. Azure has many broker architecture pattern(azure service bus) too. Some old software like biztalk,Mule ESB basically all Enterprise Service bus was designed for handling such service data with reliability & real time performance.
Hint is don't try to process 50GB data at same time, it will even not supported by many file system & will be performance hog for memory & CPU both, over service it will block all traffic. You need Service broker like Azure service bus which can take call one by one from Queue & process, also can store all unprocessed call in queue that is coming till it processed so that nothing is lost in queue. You can try n-tiered ESB for multiple distributed processing of data for faster processing.
One more thing just triggered to me, Azure batch has more functionality than Azure functions for such big job with support to High Performance Compute(HPC) if load is very big & can not be broken to Broker Architecture.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.