long time running python script

Question

I have application of following parts:

client->nginx->uwsgi(python)

and some python scripts can be running long time (2-6 minutes). After execution of script I should give to client content, but connection break with error "gateway timeout 504". What can I use for my case to avoid this error?

6 minutes for request is very very slow, your clients dont be enjoy about your program. At the first to try setup caching system and optimize db (like denormalize and slicing if you have huge tables). — Denis
– Denis, Commented Jan 31, 2012 at 6:29
No, I can't divide the script. And script don't makes query to DB and dont sends many content. It is just algorithm for image processing, but algorithm need some time (2-6 minuts) for to make result. And my clients understand and willing to wait. — monax
– monax, Commented Jan 31, 2012 at 8:30

Corbin · Accepted Answer · 2012-01-31 06:13:18Z

2

So is your goal to reduce the run time of the scripts, or to not have them time out? Browsers are going to give up on a 6 minute request no matter what you try.

Perhaps try doing the work on the server, and then polling for progress with AJAX requests?

Or, if possible, try optimizing the scripts. For example, if you have some horribly slow SQL stuff going on, try cleaning that up.

Otherwise, without more information, a more specific answer is hard to give.

answered Jan 31, 2012 at 6:13

Corbin

33.6k7 gold badges73 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

monax Over a year ago

I'd like to avoid time out. I can't make valuable optimization because script makes processing a image and this algorithm can't work faster.

monax Over a year ago

maybe I can change source code of nginx for increase time of connection between nginx and uwsgi (or nginx and client)?

Lawrence D'Oliveiro Over a year ago

AJAX would seem to be the only reasonable solution: have the processing continue in the background on the server while a “status: processing” page is immediately returned to the user, which contains periodic polling JavaScript code to retrieve the result once it’s available.

Aaron Watters · Accepted Answer · 2012-02-07 18:57:59Z

0

I once set up a system where the "main page" contained an Iframe which showed the output of the long running program as text/plain. I think the the handler for the the Iframe content was a Python CGI script which emitted all headers and then the program output line by line under an Apache server.

I don't know whether this would work under your configuration.

answered Feb 7, 2012 at 18:57

Aaron Watters

2,8464 gold badges24 silver badges37 bronze badges

Comments

tzot · Accepted Answer · 2012-02-15 00:10:20Z

0

This heavily depends on your server setup (i.e. how easy it is to push data back to the client), but is it possible while running your lengthy application to periodically send some “null” content (e.g plain newlines assuming your output is html) so that the browser thinks this is just a slow connection and not a stalled one?

answered Feb 15, 2012 at 0:10

tzot

96.6k30 gold badges151 silver badges210 bronze badges

Collectives™ on Stack Overflow

long time running python script

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related