1

I'm working on web scraping a news website to show top headlines and URLs. In my backend, I put each title and URL in an object within an array, which works fine. When I fetch this information in React, it is not working as expected.

I used the useState hook to initialize an empty array, where I would put each object containing the title and URL. Then, I map through that array and render each title and URL to the page.

When I refresh the webpage, it takes several seconds for each title and URL to pop up, and additionally, they are all the same title and URL.

It seems that my array is not being updated properly by putting in the same article information each time I set the state. However, I do not understand why it is taking so long for all the information to show up on the webpage, and do not know how to make it a faster process. I want all the information to be on the page after the user hits refresh, not appear after several seconds at a time. Could anyone help me see where I'm going wrong?

Here is my code:

import {useState} from 'react';

const News = () => {
  const [articles, setArticles] = useState([])

  fetch('http://localhost:8000/news')
    .then(response => {return response.json()})
    .then(data => {
        data.forEach(article => {
            setArticles([...articles, {
              title: article.article_title,
              url: article.article_url}])
    })
    })
    .catch(err => console.log(err))

  return (
    <div>
      {articles.map(article => {
        return (
        <div className="indv-article">
          <h1 className="article-title" key={article.title}>{article.title}</h1>
          <p className='article-url' key={article.url}>{article.url}</p>
        </div>);
      })}
    </div>
  )
}

export default News
5
  • Always run ferch inside useEffect hook otherwise it will fetch multiple times for no reason. Check network tab in the dev tools to see if the request takes long time to get Commented Aug 1, 2022 at 0:35
  • Besides the information provided by the other users in the comments and answers, why are you looping through your data to set the state? Wouldn't it be better to set it only once since you are retrieving an array of objects with all the articles anyways? Commented Aug 1, 2022 at 0:49
  • @ivanatias honestly, it was a dumb mistake- setting it once didn't cross my mind for some reason Commented Aug 1, 2022 at 17:16
  • I see. Did any of the answers provided help you solve your issue or do you need further assistance? Commented Aug 1, 2022 at 20:03
  • @ivanatias yes they definitely helped and my issue is now resolved. Thanks so much! Commented Aug 2, 2022 at 19:16

2 Answers 2

3

Couple of things that may solve your issues.

First: In React, all side-effects (such as data fetching, for example) should be handled inside a useEffect hook.

Second: As stated in @Khorne07's answer, the key attribute should be on the root DOM node that is being returned from the map.

Third: I don't really know the purpose of looping through your data to set the state. If the reason you are doing this is because the response contains other information that you are not interested to display and you just want the title and url for each article, I suggest you to create an adapter function that will receive this data as a parameter and return just the information that you are interested in.

Additional: You can use a loading state to show a loading indicator while the data is being fetched and improve user experience.

Putting it all together:

const News = () => {
  const [articles, setArticles] = useState([])
  const [loading, setLoading] = useState(false)

  const adaptArticles = (data) => {
    return data.map(({ article_title, article_url }) => ({
      title: article_title,
      url: article_url
    }))
  }

  useEffect(() => {
    setLoading(true)
    fetch('http://localhost:8000/news')
      .then(response => {return response.json()})
      .then(data => setArticles((prevArticles) => prevArticles.concat(adaptArticles(data))))
      .catch(err => console.log(err))
      .finally(() => {
        setLoading(false)
      })
  }, []) //Insert the corresponding dependencies (if any) in the dependencies array so the useEffect hook gets executed when any of these dependencies change.

  if(loading) //Return some loading indicator, like a Spinner for example.

  return (
    <div>
      {articles.map(article => {
        return (
        <div className="indv-article" key={article.title}>
          <h1 className="article-title">{article.title}</h1>
          <p className='article-url'>{article.url}</p>
        </div>);
      })}
    </div>
  )
}
Sign up to request clarification or add additional context in comments.

Comments

1

Edited:

You have some errors on your current code:

First of all, and, as mentioned by the accepted answer, all side effects like fetch calls should be inside a useEffect hook.

The second error is related to the way you are updating your state array. When your new state depends on the previous state value, you should use the callback function inside your setState function, in order to have your data correctly synchronized with the previous value. And in this particular example you are also calling a setState function inside a loop, which is a bad idea and can potentially drive your app into unexpected behavior. The best approach is described in the code snippet bellow.

fetch('http://localhost:8000/news')
    .then(response => {return response.json()})
    .then(data => {
        const articlesArray = []
        data.forEach(article => {
            articlesArray.push({
                title: article.article_title,
                url: article.article_url
            })
        setArticles(currentArticles => [...currentArticles, ...articlesArray])
     })
    })
    .catch(err => console.log(err))

And on the map function, the key attribute should be on the root node you are returning:

{articles.map(article => {
        return (
        <div className="indv-article" key={article.title}>{article.title}>
          <h1 className="article-title"</h1>
          <p className='article-url'</p>
        </div>);
      })}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.