Application performance metrics for Python

Question

We're writing a Python web-application, where we'd like to know general-ballpark-figures about what our users are doing.

As an example, we have a bunch of legacy-modes for setting the used locale; url-parameter, using different cookies in different formats, Accpet-Language-headers &c. We would really like to measure this in some way (but logging it is simply too much data).

I've tried looking for libraries like Metrics for Node.js, but I can't seem to find anything for Python. Right now, I think we can make do with a bunch of UNIX-load-like counters. (But more is always better, right?)

Have I missed some obvious library or some smart technique somewhere?

Morten Siebuhr · Accepted Answer · 2012-04-20 09:49:23Z

3

Just found the PyCounters project, which looks just like what I needed to begin with.

answered Apr 20, 2012 at 9:49

Morten Siebuhr

6,1284 gold badges33 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Chris Zelenak · Accepted Answer · 2012-04-20 21:52:36Z

1

A Python project called Graphite coupled with a Node.js project called Statsd can do exactly the thing you want. Many times people use these projects for software performance metrics and business metrics as well.

Incidentally there are hosted app versions that provide a lot of the same functionality as Graphite and Statsd. My company's product, Instrumental, will do the metric hosting frontend and graph generation for you; in Python, you'd use a Python statsd client in concert w/ our Statsd to Instrumental proxy to report your metrics to our service.

Since the accepted answer was doing file-based logging (by default at least:) ), I'll note the transportation layer is interesting among these different things: A statsd server (whether the official or our proxy) uses UDP to send your metric information to the server. You can also contact Graphite or Instrumental directly using a line based TCP protocol; other commercial services out there like librato Metrics or Stathat take in your metrics using HTTP messages. Depending on the scale of messages you're sending, any one of these tools may be more or less attractive to you.

answered Apr 20, 2012 at 21:52

Chris Zelenak

2,17817 silver badges21 bronze badges

2 Comments

Morten Siebuhr Over a year ago

I wasn't looking for something to store data, I was rather looking for something pythoninc to gather them in the application itself.

Chris Zelenak Over a year ago

Ahh, I misunderstood - my apologies.

S.Lott · Accepted Answer · 2011-09-08 13:26:36Z

0

Look into multiple logs.

We have many separate logs (separate handles, separate formats) and use the Python logging to log these events into separate logs.

We have lots of functions with things like this.

def some_feature( ... ):
    customer_log.info( "Requested some_feature" )
    ...
    vendor_x_log.info( "Got foo" )
    ...
    vendor_y_log.info( "Got bar" )

Now we have detailed logs. Since the logging module can write to the database (or do any other thing that seems appropriate) we have all the details through a simple, easily-configured interface.

answered Sep 8, 2011 at 13:26

S.Lott

393k83 gold badges520 silver badges791 bronze badges

1 Comment

Morten Siebuhr Over a year ago

I've gone for this approach for the job at hand (and it works quite well), but I still feel there should be more ... obvious ... way of doing it, somehow.

user3752700 · Accepted Answer · 2014-08-23 15:34:56Z

0

You can try a hosting metrics service Metrics At. You can log the data you want to metric, e.g. url-parameter, Accpet-Language-headers ...etc. Then you can input a regular express in Metrics At. Metrics At will automatically extract data and metric it for you.

answered Aug 23, 2014 at 15:34

user3752700

1036 bronze badges

Comments

turtlemonvh · Accepted Answer · 2015-08-06 14:24:21Z

0

One newer alternative to PyCounters is Scales: https://github.com/Cue/scales/

It has a few cool extra features like launching a server to serve current stats or sending to graphite.

answered Aug 6, 2015 at 14:24

turtlemonvh

9,9296 gold badges50 silver badges55 bronze badges

Collectives™ on Stack Overflow

Application performance metrics for Python

5 Answers 5

Comments

2 Comments

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

2 Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related