I am crawling 5 different sites for data using node-fetch and cheerio. everything checks out but I need to collect the returned data from these 5 separate functions in an array.
First I store function name and url for each site in an object like so
url: 'https://sampleurl.com',
crawl: FirstLinkCrawl
}
const secondLink = {
url: 'https://sampleurl.com',
crawl: secondLinkCrawl
}
}```
Then I write the function to crawl each site like so, I ran this function with and without promise, both check out
```const secondLinkCrawl = (body) =>{
return new Promise((res, rej)=>{
"this crawl function is ran here, everything works fine, Data is an object"
const error = false
if(!error){
res(Data)
}else{
rej()
}
})
}```
This here is my fetch function that takes the url and a callback, which is the crawl function
```async function Fetch(url, callback){
const response = await fetch(url)
const html = await response.text()
callback(html)
}
Then I call the fetch and crawl using promise.all() in an attempt to have it return in an array, but in an array,
const promises = [
Fetch(firstLink.url, firstLink.crawl),
Fetch(secondLink.url, secondLink.crawl)]
Promise.all(promises)
.then(values => console.log(values))
.catch(err => console.log(err))
}
When I run this, I get [ undefined, undefined ]
But when I run it without promises and simply log the result, both of them run successfully.
My aim is to get my results in a single array. what can I do?
I also tried declaring an array a the top of the page and then pushing each result into the array, after which I log the array at the bottom of the functions call. But the array returns empty