I've to develop a server that has to make a lot of connections to receive and send small files. The question is if the increment of performance with C++ worth the time to spend on develop the code or if is better to use Python and debug the code time to time to speed it up. Maybe is a little abstract question without giving a number of connections but I don't really know. At least 10,000 connections/minute to update clients status.
6
-
2Optimisations should really come near the end. Get to code, and up and running as fast as possible, You may find that IO is the bottleneck factor.Michael Brown– Michael Brown2012-09-29 20:13:26 +00:00Commented Sep 29, 2012 at 20:13
-
2For questions like this, you should probably build an example in Python and measure the performance. If it's not up to snuff, then you see what's the problem. You might be limited by using a single server first, but you never know.wkl– wkl2012-09-29 20:13:39 +00:00Commented Sep 29, 2012 at 20:13
-
Good advises guys. I think that I was afraid to choose Python and have to rewrite all the code because Python can't manage too many connections. But as you don't say anything about this I think that I'll write a prototype using Python.David Moreno García– David Moreno García2012-09-29 20:15:34 +00:00Commented Sep 29, 2012 at 20:15
-
1@MichaelBrown In general yes, but some decision much be made early because they vastly influence almost the entire code base. Language choice is one that much be made early -- provided it matters, of course.user395760– user3957602012-09-29 20:17:30 +00:00Commented Sep 29, 2012 at 20:17
-
@delnan I was (unclearly) referring to prototyping, to aid in the decision of requirements.Michael Brown– Michael Brown2012-09-29 20:23:29 +00:00Commented Sep 29, 2012 at 20:23
|
Show 1 more comment
2 Answers
With that many connections, your server will be I/O bound. The frequently cited speed differences between languages like C and C++ and languages like Python and (say) Ruby lie in the interpreter and boxing overhead which slow down computation, not in the realm of I/O.
Not only can use make good and reasonably use of concurrency (both via processes and threads, the GIL is released during I/O and thus does not matter much for I/O-bound programs), there is also a wealth of asynchronous servers. In addition, web servers in general have much better Python integration (e.g. mod_wsgi for Apache) than C and C++. This frees you from writing your own server loop, socket management, etc. which you likely won't do as well as the major servers anyway. This is assuming we're talking about a web service, and not something more arcane which Apache etc. cannot do out of the box.
3 Comments
David Moreno García
Really nice comment. This server must be connected with a web server so the integration is really important. About the I/O problem, do you have any advise to this apart of use concurrency? Thanks.
David Moreno García
Around 100mb at the beginning.
I'd expect that the server time would be dominated by I/O- network, disk, etc. You'd want to prove that the CPU consumption of the Python program is problematic and that you've grasped all the low-hanging CPU fruit before considering a change.
