10

Background

For a REST api service, I'd like to provide more logging context in a way that I don't have to rewrite the log statements for my entire application. I'm using the python logging lib within flask and the eventlet runner type with gunicorn.

Use Case

Imagine a future where all requests through this system have a unique(enough) transaction ID passed as a header from some upstream source (reverse proxy maybe). I'd like to log this transaction id with each log statement to make it easy to trace a given request through my system even during peak load.

Approach

Write a custom logging context filter class that pulls desired information from flask. It is my understanding that I should be able to pull this info (namely the request object) from thread local context variables. Upon initialization of the global root logger, then I simply set this custom context filter and all should be good in the land of debugging!

I discovered this approach from the following cookbook documentation... https://docs.python.org/2/howto/logging-cookbook.html#using-filters-to-impart-contextual-information

Questions

  • Do you foresee any scaling issues with this approach?
  • Thoughts on propagate this transaction id downstream to other requests across my network?
  • Will the use of the eventlet worker type get in the way of this behaving as expected (i.e. mixed context as a result of concurrency issues)?
  • Just because you can, doesn't mean you should. Any other reason why I shouldn't do it this way?
Improve this question
1
  • 3
    On the off chance you're still looking, this blog post might help. Commented Nov 1, 2016 at 22:11

1 Answer 1

Reset to default
2

It sounds like you may be considering implementing an X-Trace like system. Disclaimer: I work with a commercial product which operates like this.

Regarding propagation it can become a concern as the complexity of the app grows. As you require more off the shelf solutions or polyglot components in your back end these will require support for context logging or be burdened by a lack of visibility. Same goes for alternate messaging mechanisms, if you need to pass context over something besides HTTP-RPC like JBoss or Thrift or a message queue of some kind implementation complexity goes up.

For async requests you will definitely need to ensure the unique id is passed correctly from the blocking code path into the evented code path or risk mixing / re-using id's. This can be the source of subtle red herrings when analyzing call propagation.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.