How to do a async HTTP request inside an Angular2 loop/map and modify the original loop array?

Question

New to Angular2 here and wondering how to do this async request on array pattern (not so sure about the pattern name).

So let's say I use Angular2's Http to get a YouTube Playlist that contains videoId in each return item. Then I need to loop through this item with the videoId and do another request to YouTube to get the individual video information. This can be another common HTTP request, get the list then loop through the list and get the details of every individual item.

let url = [YouTube API V3 list url]
let detailUrl = 'https://www.googleapis.com/youtube/v3/videos?part=contentDetails,statistics';

this.http.get(url)
    .map(res => res.json())
    .subscribe(data => {
        let newArray = data.items.map(entry => {

            let videoUrl = detailUrl + '&id=' + entry.id.videoId + 
                                       '&key=' + this.googleToken;

            this.http.get(videoUrl)
                .map(videoRes => videoRes.json())
                .subscribe(videoData => {
                     console.log(videoData);

                     //Here how to I join videoData into each item inside of data.items. 
                });
    });
});

So the above code does work. But I still have these two questions:

How do I join the videoData back to data.items and make sure the correct item inside the data.items is joined with the correct videoData (the videoData that is using the entry.id.videoId of the related item inside data.items) and create that new newArray?
How do I know when everything is done and do something based on all the data after all the async requests have finished?
The newArray keep the same order of the first HTTP request. As the first HTTP hit the YouTube search list API with order by viewCount. Simply nest observable like above will not keep the original order.

UPDATE

With inoabrian solution, I can successfully achieve the above three points. But when I call the myvideoservice again (like changing the url as new NextPageToken), the Subject - this._videoObject has 2 observers. And load again, it has 3 observers and so on. I need a way to reset the Subject to have only 1 observers so I won't have duplicate videos. Is it a best practice way to clear/reset subject?

inoabrian · Accepted Answer · 2016-07-06 01:09:12Z

2

// You need to set up a subject in your sevice
private _videoObject = new Subject < any > ();
videoDataAnnounced$ = this._videoObject;


let url = [YouTube API V3 list url]
let detailUrl = 'https://www.googleapis.com/youtube/v3/videos?part=contentDetails,statistics';

this.http.get(url)
    .map(res => res.json())
    .subscribe(data => {
        this.cache = data.items;
        data.items.forEach(entry => {
            let videoUrl = detailUrl + '&id=' + entry.id.videoId + '&key=' + this.googleToken;
            this.http.get(videoUrl)
                .map(videoRes => videoRes.json())
                .subscribe(videoData => {
                    // This will announce that the full videoData is loaded to what ever component is listening.
                    this._videoObject.next(videoData);
                });
        });
    });



// In your constructor for your component you can subscribe to wait on the data being loaded.
public newVideos: any[] = [];
constructor(private myServiceToLoadVideos: myvideoservice) {
    let self = this;
    this.myServiceToLoadVideos.videoDataAnnounced$.subscribe((video) => {
        // Here you could loop through that
        self.myServiceToLoadVideos.cache.map(d => {
            if (d.id == video.id) {
                // Here you have the old data set without the details which is (d) from the cached data in the service.
                // And now you can create a new object with both sets of data with the video object
                d.snippet = video.items[0].snippet;
                d.statistics = video.items[0].statistics;
                d.contentDetails = video.items[0].contentDetails;
                this.posts = this.posts.concat(d);
            }
        });
        if(this.posts.length == this.cache.length){
          this.posts.sort(function(l,r){
            // if l is < r then it will return a negative number
            // else it will return a positive number.
            // That is how this sort function works
            return l.viewCount - r.viewCount;
          });

          // Here you will have all of the data and it will be sorted in order.
        }
    });
}

UPDATE -- -- --

The Subject is an observable - http: //reactivex.io/documentation/subject.html

    When you listen to the Subject it is not like q.all it will complete one by one until they complete.

The cache in the service call is just to keep track of the playlist.
Then when you receive the details in the constructor you can match them back up and keep order with the cache.

Q: "And when subscribe to the subject as in your code, the result is when ALL the "
next " in _videoObject finished, right?"
A: No, when each individual finishes you will get once at a time.

JS Sort - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort

edited Jul 6, 2016 at 1:09

answered Jul 2, 2016 at 5:52

inoabrian

3,8101 gold badge22 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

inoabrian Over a year ago

Could you just do this at the end? this.posts.sort((l,r) =>{ return l.viewCount - r.viewCount; })

inoabrian Over a year ago

I added the sorting to where the data would be complete.

inoabrian Over a year ago

well it is completed now when posts and cache are the same size then you sort the complete list

inoabrian Over a year ago

Let us continue this discussion in chat.

Hugh Hou Over a year ago

Found a new issue using your solution: If I wrapped the first part of the code inside a function called fetchData() in the service and the second part of the code inside loadSettings() in the component. And inside the service, I am changing the url with NextPageToken to get the next 10 videos from YT. When I call loadSettings() with a load more btn, which will call this.myServiceToLoadVideos.fetchData() again, the this._videoObject contains 2 observers. If I click load more btn again, it contain 3 observers. So I will have duplicated videos. How do I reset the Subject when I call it again?

|

Ankit Singh · Accepted Answer · 2016-07-02 08:06:16Z

1

Assuming that you only need the modified data, once all the sub-requests are completed

This is what you'll need

this.http.get(url) // original request
      .map(res => res.json())  // extract json from response
      .switchMap(data => {  // take-over the original data and make the subsribers wait for sub-requests to complete

        let modifiedRequest = new Subject(); // observable to to publish modified data, once all sub-requests are completed

        let allSubRequests = data.items.map(entry => {  // array of sub-requests

          let videoUrl = detailUrl + '&id=' + entry.id.videoId +
                         '&key=' + this.googleToken;

          return this.http.get(videoUrl)
            .map(videoRes => videoRes.json())  // extract json for every sub-request
            .map(videoJson => {  // modify the original 'data' object with the data in videoJson (I don't know the format, so can't tell how-to)

              // modify your "data" here

              return videoJson; // let the flow continue
            });
        });

        let mergedSubRequests = Observable.range(0, 1).merge.apply(null, allSubRequests);  // merge all sub-requests to know when will all are completed

        mergedSubRequests.subscribe(  // subscribe to merged observable
          (d) => {}, // data from every sub-request, we don't need it
          (e) => {}, // error from every sub-request, handle it as you want
          () => {   // success, called after all sub-requests are completed
              modifiedRequest.next(data);  // send out the modified data
              modifiedRequest.complete();  // complete the temprary observable
              }
        );

        return modifiedRequest; // provide the observable with modified data, to subscribers of orginal request, they will not know the difference
      });

Required Imports:

import {Observable} from 'rxjs/Observable';
import {Subject} from 'rxjs/Subject';
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/merge';
import 'rxjs/add/observable/range';
import 'rxjs/add/operator/switchMap';

Hope it helps, not tested though :)

edited Jul 2, 2016 at 8:06

answered Jul 2, 2016 at 7:20

Ankit Singh

25k12 gold badges69 silver badges90 bronze badges

3 Comments

Hugh Hou Over a year ago

This part lost me: let mergedSubRequests = Observable.range(0, 1).merge.apply(null, allSubRequests);

Hugh Hou Over a year ago

Also your use of switchmap. Can out explain more why you use switchmap here? As switchmap cancel the pervious observable, right?

Hugh Hou Over a year ago

I just test your code and it is not working for me... I put a console.log(data) after the switchMap and there is no console log out. And I put a console.log(videoJson) before the return videoJson and also no log. The entire code seem block by switchMap...no idea what is going wrong...

Can Nguyen · Accepted Answer · 2016-07-02 08:11:57Z

1

let addDetail = (entry) => {
  let videoUrl = detailUrl + '&id=' + entry.id.videoId + 
                '&key=' + this.googleToken;

  return this.http.get(videoUrl)
    .map(videoRes => Object.assign({videoData: videoRes.json()}, entry)
};

let maxConcurrentRequest = 1;
let source = this.http.get(url)
  // any of mergeMap, concatMap, switchMap... will do
  .mergeMap(res => Observable.from(res.json().items))
  .mergeMap(addDetail, null, maxConcurrentRequest);
source.subscribe( /* Do anything you wish */ );

Now each item that source emits is an entry in the playlist merged with its videoData in videoData field (your first question)

source completes when everything is done (your second question)

you can control the number of maximum concurrent requests to detailUrl by setting maxConcurrentRequest. With maxConcurrentRequest = 1, source will emit items in the same order as they are in the playlist. If you don't care about order, increase maxConcurrentRequest for faster speed

edited Jul 2, 2016 at 8:11

answered Jul 2, 2016 at 8:05

Can Nguyen

1,47010 silver badges11 bronze badges

2 Comments

Hugh Hou Over a year ago

I do need to preserve the original order of the data.items as the video list is sorted based on viewCount. So I want the same order of the video detail. But I don't quite get your code as I am very new to observable. What is the object.assign? And the mergeMap?

Can Nguyen Over a year ago

mergeMap = flatMap, see reactivex.io/documentation/operators/flatmap.html and reactivex.io/rxjs/class/es6/… Object.assign is a function to "merge" objects: developer.mozilla.org/en/docs/Web/JavaScript/Reference/…

Jarod Moser · Accepted Answer · 2016-07-02 03:27:37Z

You already have access within the .map function to entry you should be able to just put another value onto it using dot notation. so do something like this:

let url = [YouTube API V3 list url]
let detailUrl = 'https://www.googleapis.com/youtube/v3/videos?part=contentDetails,statistics';

this.http.get(url)
    .map(res => res.json())
    .subscribe(data => {
        let newArray = data.items.map(entry => {

            let videoUrl = detailUrl + '&id=' + entry.id.videoId + 
                                       '&key=' + this.googleToken;

            this.http.get(videoUrl)
                .map(videoRes => videoRes.json())
                .subscribe(videoData => {
                     console.log(videoData);

                     entry.videoData = videoData;

                     //Here how to I join videoData into each item inside of data.items. 
                });
    });
});

Collectives™ on Stack Overflow

How to do a async HTTP request inside an Angular2 loop/map and modify the original loop array?

4 Answers 4

6 Comments

3 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

6 Comments

3 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related