Angular HttpClient: get data from RESTfull API pages (next page loop)

Question

does anybody has a good example or tutorial where can I find good practices to retrieve data from a paginated RESTfull API using Angular 6 with HttpClient?

Retrieving data from the considered API I use the following code in my service.ts file:

getApiData(term: string): Observable<ApiResponse[]> {
  let apiUri = this.baseUrl;
  let msg = 'data fetched from API';

  if (term.trim()) {
    apiUri = this.baseUrl + this.queryUrl + term;
    msg = msg + ' matching: ' + term;
  }
  return this.http.get<ApiResponse[]>(apiUri)
    .pipe(
      tap(data => {
        this.log(msg);
      }),
      catchError(this.handleError('getApiData', []))
    );
}

However, I am can't find any hint how to loop through all pages (data['next']) of the API endpoint using the observable constraints.

I would appreciate every idea or link to more information to handle this common problem. Obviously I also haven't the right terms so find any suitable answer here. :-/

hjbello · Accepted Answer · 2018-09-12 11:52:27Z

1

First of all be careful with doing .trim() to a variable that could be undefined. I would change your

if (term.trim()) { ... }

with

if (term && term.trim()) { ... }

Regarding how to iterate the data retrieved from the observable, I would do the following. Suppose that your function getApiData belongs to a service my-backend.service.ts. Suppose that you want to invoke in a component somewere in your code. If you have your service added in your app.modules, you can inject it in your component from the constructor like this: first import it:

import {MyBackend} from "/.myBackend.service";

then add it to the constructor (inject it) and a global variable for your data:

data:any;
constructor( public myBackend: MyBackend){}

Now wherever you want to invoke your service you do the following:

this.myBackend.getApiData(term).subscribe( (data) => {
this.data = data;
});

And you will have your data stored in your component. Now you can iterate it or do whatever is necesary (data["next"] as you point out for instance).

Let me know if I am unclear or if you need more detail.

EDIT

If you are dealing with a paged search, your this.data should have the info regarding the number of pages or number of results found versus number of results shown. In that case you will need to decide on-the-go if you have to search again and invoke the service several times (or inform the user).

In this case I believe it would not make sense to iterate calls to http.get in your service, paged searches are not intent to work like that, rather the logical option would be to show the results page by page to the user.

If nevertheless you want to do something like that (automatically iterate searches from your component). You could implement a function like this in your component:

async retrieveAllPages(searchInput){
 let finish = false;
 dataPages = [];

 while (!finish){
  const page = wait this.myBackend.getApiData(searchInput).toPromise()
  dataPages.push(page);
  finish = this.decideIfSearchIsFinished(page);
  searchInput = this.updateSearchTermNextPage (searchInput);
 }
}

You will also need these functions. I do not specify them because I do not know how your backend api works:

decideIfSearchIsFinished(page):boolean{
//looks into the page object response to see if there are more pages or if we are in the last one
//..
}

updateSearchTermNextPage(term):string{
//updates the search term to search for a new page
}

edited Sep 12, 2018 at 11:52

answered Sep 12, 2018 at 9:42

hjbello

6514 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Mike42 Over a year ago

Thanks for your very fast response! Regarding your first recommendation I absolutely go with you. In my case I check it already in the component.ts file before its passed to the service function. I mean all other points are also clear to me. However I would like to interate through all pages of the API endpoint in the service.ts and not in the compontent.ts file (logically separated). Are there any ideas to do the page looping in the service function?

hjbello Over a year ago

I do not understand what you mean by "all pages of the api endpoint". Does the url contain a "page" parameter somewhere? Does the response indicate that there are more resuts? (Iike a with a paged search). If the response indicates that there are more results, and you are getting only an interval of them, you would need to invoke the service again with a different endpoint or parameters.

hjbello Over a year ago

@Mike42 , I updated the code, take a look if it makes sense or if it is what you are looking for.

Mike42 Over a year ago

Thanks again for your point of view. Your are right, I have to decide whether I want to inform the user about ongoing data retrieval from the API endpoint. Especially if there are a huge amount of pages it would be take a long time without response to the user doing it in the service.ts. Right now I was looking for the right approach to concat the observables (gained from a while loop) in the service function. But after your words I will also consider the approach doing the loop in the compontent.ts just to improve the usability for the users. Thanks!

JoelFan Over a year ago

Did you mean await instead of wait?

|

fix · Accepted Answer · 2022-12-07 13:41:12Z

0

Updated answer for latest Angular + rxjs version.

subscription.toPromise() is deprecated. See change here

Using the example from the accepted answer:

import { firstValueFrom, Observable } from 'rxjs';

async retrieveAllPages(searchInput){
 let finish = false;
 dataPages = [];

 while (!finish){
  const page = await firstValueFrom(this.myBackend.getApiData(searchInput));
  dataPages.push(page);
  finish = this.decideIfSearchIsFinished(page);
  searchInput = this.updateSearchTermNextPage (searchInput);
 }
}

answered Dec 7, 2022 at 13:41

fix

2,2752 gold badges21 silver badges30 bronze badges

Collectives™ on Stack Overflow

Angular HttpClient: get data from RESTfull API pages (next page loop)

2 Answers 2

6 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related