0

I had started to build a program for my personal work use of web scraping and input for mortgage rates. Essentially what i wanted to do was have my program log into each website, enter the mortgage data necessary, and it would return rates and compare each site so that i wouldnt have to manually do this on each site.

The problem i didnt think of is the login portion. i would have to store tokens and a few other items in order for me to navigate from page to page within each website.

my question is, is this even possible since i dont know the credentials/tokens to send to each page within a site? (i have the login info but unsure if i need more than just the credentials and the tokens)

Improve this question

1 Answer 1

Reset to default
1

This is complicated with just the request module.

Note that this approach requires more system resources

You can use PlayWright to control a chromium instance.

Chromium saves the credentials and tokens like nearly every other browser and you just have to program the browser to login and scrape.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.