cURL script in PHP or Python, perl

Question

I have in PHP a script that use multi curl to connect to sites doing the following actions:

login to the site

get a page

verify the page content

if the content is what I need I do 5.

post some data to the site.

log out.

My script can login to <20 sites in parallel, but is not too fast and is use a lot of server resources.

Can I convert the script to Python or other language to make it faster? I don't know if is possible to use a browser like firefox or other to send commands and get the content? Example:

system("firefox http://stackoverflow.com/users/login");
system("firefox post-data:username=stack&password=some");
system("firefox getcontent > mypage.html");

It may looks a crazy solution :) but I need to improve the performance of PHP script.

Thank you very much, Adrian

ryanblak · Accepted Answer · 2011-02-02 11:37:19Z

2

I'd imagine it has less to do with the language, and more to do with waiting for HTTP requests. It doesn't sound like you are doing anything that would cause PHP to be the bottleneck.

I'd start by testing the time it takes for the HTTP requests. I'll bet that's where your bottleneck is.

answered Feb 2, 2011 at 11:37

ryanblak

212 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

keatch · Accepted Answer · 2011-02-02 11:28:13Z

1

For python, try to use Mechanize

answered Feb 2, 2011 at 11:28

keatch

2,2172 gold badges18 silver badges22 bronze badges

1 Comment

varu Over a year ago

Thanks for help. I think I can use another server to connect remotely to the sites and the result will be sent to server 1. I think will reduce a lot the server load from main server.

Collectives™ on Stack Overflow

cURL script in PHP or Python, perl

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related