0

I have A table that has two columns, the first containing the URLs, and the second containing numbers.

What I need to do is parsing all URLs at the same time, then wait for how much in column 2, Then back again to parse each URL in column 1.
Not one by one but at the same time.

URls time for wait
url1 2
url2 4
url3 5

Thanks For helping,

3
  • 2
    What, exactly, do you mean by "parsing all URLs at the same time"? Commented Dec 8, 2022 at 2:23
  • 2
    What have you tried so far? Have you tried simple multiprocessing? Commented Dec 8, 2022 at 2:26
  • I mean access all urls in the same time not one by One. Commented Dec 8, 2022 at 14:31

1 Answer 1

1

If you know the size of the lists will be same and the time to wait values will match with the url you want you can do this to combine them.

urls = ['https://www.google.com', 'https://www.apple.com', 'https://www.duckduckgo.com']
time_to_wait = [2, 4, 5]

for url, wait_time in zip(urls, time_to_wait):
    get(url)
    sleep(wait_time)

I would checkout asyncio event loops to make them run at the same time.

https://docs.python.org/3/library/asyncio-eventloop.html

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.