Multiple http request in the background using angular 6

Question

I have to fetch data on small batches to complete the data I already have in the frontend, So let say I have hundreds of objects of an id, name and the other fields are empty as I have to get them from another endpoint on by passing the id, however, this other endpoint is quite slow and I need to do this process for let say 5 at the time and in the background "without interrupting any other requests".

What is the best approach to do that? and append the fetched data to its corresponding item inside the list of objects?

Picci · Accepted Answer · 2018-09-21 05:35:15Z

1

If I understand right, you start with an array of objects of which you know id and name. You need to populate all these objects with additional data which you can fetch using a slow API, and therefore you do not want to have more than 5 of such requests flying at a time.

Let's assume that the invocation of such API is implemented via the method getDataForId(id) and that the populate of the object with the additional data retrieved from the API is performed by the method populate(obj, data) where obj is the object that has to be populated and data is the raw data to be used to populate the object.

If this is all right, you can consider an approach along these lines

const objArray = [
  {id: 123, name: 'first'},
  {id: 456, name: 'second'},
  ......
  {id: xyz, name: 'last'},
];

    from(objArray).pipe(
       mergeMap(
          obj => getDataForId(objArray.id).pipe(
             map(data => populate(obj, data))
          ), 5
       )
    )
    .subscribe()

ALTERNATIVE IMPLEMENTATION - after the comment

You can consider also to fetch the list from the back end and then get the items in chunks of 5 with full details. You can achieve this with code which would look like

getListOfIds().pipe(
   bufferCount(5),
   map(arrayOf5Ids => getDataForId(objArray.id).pipe(
             map(data => populate(obj, data))
   )),
   switchMap(arrayOf5Observables => forkJoin(arrayOfObservables)),
)
.subscribe(arrayOf5Items => // do what you want}

edited Sep 21, 2018 at 5:35

answered Sep 20, 2018 at 13:47

Picci

17.8k16 gold badges80 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

et3rnal Over a year ago

Perfect, this is exactly my situation. However, as I was investigating this I had another idea. How about in the first call which I get the whole list of objects I do the call to the slow API and return 5 items at the time that has the complete data instead of the long incomplete list? Do you think this is possible and better than showing a list of not complete data waiting for it to populate?

Picci Over a year ago

It is feasible to get first the list and then 5 items at a time with full details. See my updated answer. If this is better, I do not have any idea. I think it depends on the requirement you want to implement.

et3rnal Over a year ago

Thank you for the update, I don't understand it yet, I'll give it a go ASAP. In the main time, doesn't it solves the problem I posted in here stackoverflow.com/questions/52436049/… ?

et3rnal Over a year ago

I had to back to this today as the other approach caused some problems. I just can't figure out how to filter or have some logic in this? because I don't need to get data from the slow API for everyone let say based on the name?

Picci Over a year ago

you can use filter like getListOfIds().pipe(filter(obj => // your filter logic), bufferCount(5), .....)

|

Chunbin Li · Accepted Answer · 2018-09-20 02:25:57Z

1

you can use mergeMap , like this

  this.http.get('/api1').pipe(
      mergeMap(character => this.http.get('/api2'))
    ).subscribe(() => {
       ....  
    });

References - Angular Multiple HTTP Requests with RxJS

answered Sep 20, 2018 at 2:25

Chunbin Li

2,2542 gold badges19 silver badges32 bronze badges

Collectives™ on Stack Overflow

Multiple http request in the background using angular 6

2 Answers 2

7 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related