Javascript: nested fetch with foreach loop..?

Question

this is my problem: I need to make some fetch POST requests to my server. The server holds a list of names. Each name corresponds to a list of elements which I need to browse, in order to find a specific element from all the lists. I have to work with this structure, can't change it.

My approach was to write a fetch request, to get the all these names (of the lists I need to browse), then I would loop through each name and fetch to get the according lists from the server. Once I get these lists I can loop through the elements till I find the correct element.

This is not an easy task for me as I am new to javascript.

This is how I implemented this:

function browse(){
      fetch(TARGET_IP, {
            method: "POST",
            headers: {
                "Content-type": "application/json",
                "X-Auth-Token": AUTH_TOKEN
            },
            body: JSON.stringify({
                "id": messageId++,
                "jsonrpc": "2.0",
                "method": "Browse",
                "params": {
                    "mode": "children"
                }
            })
        })
            .then(response => response.json())
            .then((data) =>  {

                data.results = Object.assign(Array.prototype, data.result);

                data.results.forEach(element => {
                    if (element.datatype == "datablock") {

                        datablocks.push(element.name)
                    }
                })

                return datablocks;

            })
            .then((datablocks) => {
                datablocks.forEach(element => {
                    
                    browseDatablocks(element)
                })
            })

            .catch(e => console.error(e))
}


function browseDatablocks(dbname) {

    fetch(TARGET_IP, {
        method: "POST",
        headers: {
            "Content-type": "application/json",
            "X-Auth-Token": AUTH_TOKEN
        },
        body: JSON.stringify({
            "id": messageId++,
            "jsonrpc": "2.0",
            "method": "Browse",
            "params": {
                "var": "\"" + dbname + "\"",
                "mode": "children"
            }
        })
    })
        .then(response => response.json())
        .then((data) => {

            data.results = Object.assign(Array.prototype, data.result);

            data.results.forEach(element => {
                    
                if (element.name == "P_Index")
                {
                    console.warn(dbname);
                    // This is the element I need and would save it
                }
            })

        })

        .catch(e => console.error(e))
}

Now all sorts of strange things happen with this approach.

The console outputs elements that are just not the elements I search for, which is just wrong as I checked on server side.

Some requests are not even handled.

I assume these problems occur because I send too many requests at the same time (the nested ones in browseDatablocks). How can I handle one request and then the next one? Or are there other ways to implement this?

Thanks.

Where do you get messageId from? You use it twice, both in browse() and in browseDatablocks(). Probably that's the problem. It's hard to debug from the code that you posted, because you should debug the network calls, not the code itself. — Rickard Elimää
– Rickard Elimää, Commented Apr 6, 2022 at 9:29
That is a global variable, which I increase for every request. Why can that be a problem? I looked at the network call, as I stated my assumption is that the API calls I make are in wrong order. Like I should first make the request in browseDatablocks and directly handling it. What currently happens is, all the fetch requests are done at the same time and after that the "then" statement is handled... How can I structure my code differently to first fetch, then handle, then the next fetch, and so on...? — CodeStallion
– CodeStallion, Commented Apr 6, 2022 at 9:38
What is Object.assign(Array.prototype, data.result); supposed to achieve? There lies madness. — Bergi
– Bergi, Commented Apr 6, 2022 at 9:49
"all the fetch requests are done at the same time and after that the "then" statement is handled..." - yes. Don't use forEach if you don't want that. However it's not clear what wrong with doing the requests in parallel, they really should be independent? — Bergi
– Bergi, Commented Apr 6, 2022 at 9:53
I think my server can not handle so many requests, but I am not sure... I just assume this could be an error, as the output of my code is different every time, like sometimes I get the correct element, but I sometimes get elements which are just not correct or I get ERR_EMPTY_RESPONSE . — CodeStallion
– CodeStallion, Commented Apr 6, 2022 at 10:02

eamanola · Accepted Answer · 2022-04-06 10:15:23Z

1

looks like what you need is Promise.all.

if i understand the problem correctly:

first return the fetch result in function browseDatablocks(dbname)

...
function browseDatablocks(dbname) {

  return fetch(TARGET_IP, {
  ....

And return the value, in last then

.then((data) => {
  data.results = Object.assign(Array.prototype, data.result);

  const filtered = data.results.filter((element) => element.name == "P_Index");

  return filtered;
});

Then in function browse(), last then:

.then((datablocks) => {
  const results = datablocks.map(element => browseDatablocks(element));

  return Promise.all(results);
})
.then((listOfResults) => {
  // Some magic here
})

// Edit or to do them in order:

In function browseDatablocks(dbname), last then:

do the handling, and return a bool instead of filtered eg true = all done, false fetch the next. Something like:

.then((data) => {
  data.results = Object.assign(Array.prototype, data.result);

  const found = data.results.find(element => element.name == "P_Index");
  if (found) {
    // Some handling
    return true;
  }
  return false;
})

And in function browse(), last then

.then(async (datablocks) => {
  let i = 0;
  while(i < datablocks.length) {
    const handled = await browseDatablocks(datablocks[i]);

    if (handled === true) {
      break;
    }

    i += 1;
  }
});

edited Apr 6, 2022 at 10:15

answered Apr 6, 2022 at 9:38

eamanola

4442 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

CodeStallion Over a year ago

My assumption is that I should first make the request in browseDatablocks and directly handle it without already making the next request, which is made async in the foreach loop in browse() . What currently happens is, all the fetch requests are done at the same time and after that the "then" statement of each fetch is handled... How can I structure my code differently to first fetch, then handle, then the next fetch, next handle and so on...?

eamanola Over a year ago

i see, so you are trying to do them consecutively? is async await an option for you?

eamanola Over a year ago

forEach does not wait for promise to resolve. it starts the promise and jumps to starting the next promise.

CodeStallion Over a year ago

Yes, at least I would like to try and see if it solves the problem if I fetch consecutively. How would I do that with async and await?

CodeStallion Over a year ago

Handling the requests sequentially solved the problem for me. I think my server was not able to handle so many requests once at a time or at least in such a short time frame. Thanks.

|

Collectives™ on Stack Overflow

Javascript: nested fetch with foreach loop..?

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related