How do you fetch multiple pages of data into an array?

Question

Here is my code

let url;
let planetCount;
let planetData = [];

//turn through the pages
for (let p = 1; p < 7; p++) {
  url = `https://swapi.boom.dev/api/planets?page=${p}`;

  //fetch data
  for (let j = 0; j < 1; j++) {
    fetch(url).then(res => res.json())
      .then(data => {

        //push data to array
        for (let i = 0; i < data.results.length; i++) {
          planetData.push(data.results[i]);
        }
      })

    console.log(planetData)

  }
}

And here is the output: https://i.sstatic.net/Hivwr.png

Problem: How do I make it so it all goes into one array, instead of 6?

your console.log is still in a for loop so you display 6 times same array, but each time this array is empty — ekans
– ekans, Commented Sep 13, 2021 at 14:36
You have a console.log in your first for loop that runs 6 times. I think the code looks correct otherwise :) — Wojtek322
– Wojtek322, Commented Sep 13, 2021 at 14:37

score 1 · Accepted Answer · 2021-09-13 14:46:43Z

1

You do have only one array. You just get an output six times because you run console.log() within a loop.

I expanded your code with the proper way to handle several async calls and wait until they are all finished:

let url;
let planetCount;
let planetData = []
let promises = [];

//turn through the pages
for (let p = 1; p < 7; p++) {
url = `https://swapi.boom.dev/api/planets?page=${p}`;

//fetch data
for (let j = 0; j < 1; j++) {
    promises.push(fetch(url).then(res => res.json())
        .then(data => {

            //push data to array
            for (let i = 0; i < data.results.length; i++) {
                planetData = planetData.concat(data.results[i]);
            }

        }));
    }
}

Promise.all(promises)
.then(() => {
    console.log(planetData.length, '=>', planetData);
})

edited Sep 13, 2021 at 14:46

answered Sep 13, 2021 at 14:37

user56725

Sign up to request clarification or add additional context in comments.

Comments

David Biderman · Accepted Answer · 2021-09-13 14:45:25Z

0

Does this answer your question? How can I fetch an array of URLs with Promise.all?

Promise all should be the right tool for you to use so that you wait until all promises are resolved before handling responses

answered Sep 13, 2021 at 14:45

David Biderman

1988 bronze badges

Comments

Randy Casburn · Accepted Answer · 2021-09-13 14:48:01Z

The issue is related to how async code works.

the fetch() function will eventually return a result, but that result is not available at the time you are trying to log it.

You must wait for the result to be retrieved. Use Aysnc/Await to accomplish that:

async function turnThroughThePges() {
  let url;
  let planetCount;
  let planetData = [];


  //turn through the pages
  for (let p = 1; p < 7; p++) {
    url = `https://swapi.boom.dev/api/planets?page=${p}`;
    //fetch data
    for (let j = 0; j < 1; j++) {
      await fetch(url).then(res => res.json())
        .then(data => {
          for (let i = 0; i < data.results.length; i++) {
            planetData.push(data.results[i]);
          }
        })
    }
  }
  return planetData;
}

turnThroughThePges().then(console.log);

Terry Lennox · Accepted Answer · 2021-09-13 14:49:44Z

0

You can use async/await and Promise.all() to return an array of the pages returned. Once we have this, we can use Array.flat() to create a contiguous array of planet data:

async function getPlanets() {
    const urls = Array.from( { length: 7 }, (v,i) => `https://swapi.boom.dev/api/planets?page=${i + 1}` );
    const promises = urls.map(url => fetch(url).then(res => res.json()).then(data => data.results));
    const planetData = (await Promise.all(promises)).flat();
    console.log(`Results for ${planetData.length} planets downloaded...`);
    console.log('Results:', planetData);
}

getPlanets()

answered Sep 13, 2021 at 14:49

Terry Lennox

30.9k5 gold badges36 silver badges45 bronze badges

1 Comment

user56725 Over a year ago

This is extremely beautiful! I tried to stay as close as possible to the original code to not add additional reading-complexities.

Vitaliy Rayets · Accepted Answer · 2021-09-13 15:33:26Z

0

let url;
let promises = [];

//turn through the pages
for (let p = 1; p < 7; p++) {
  url = `https://swapi.boom.dev/api/planets?page=${p}`;

  //fetch data
  promises.push(
    fetch(url)
      .then((res) => res.json())
      .then((data) => data.results)
  );
}

function handleRejection(p) {
  return p.catch((err) => ({ error: err }));
}
async function requests() {
  return await Promise.all(promises.map(handleRejection));
}
requests().then((planetData) => console.log(planetData.flat()));

edited Sep 13, 2021 at 15:33

answered Sep 13, 2021 at 15:20

Vitaliy Rayets

2,4041 gold badge8 silver badges13 bronze badges

Collectives™ on Stack Overflow

How do you fetch multiple pages of data into an array?

5 Answers 5

Comments

Comments

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related