Statement-driven documentation for CPython internals

The documents inside the InternalDocs are sufficient to help you gain a general understanding of the internal components of CPython. However, I suggest adding a statement-driven tutorial, let’s say, a “statement_tutorial.md” file, inside InternalDocs, explaining how to add a new statement to Python in a step-by-step tutorial. In that case, we will help the maintainers and contributors understand the internal components straightforwardly, using one example. The “statement_tutorial.md” file will build an imaginary statement and use it to explain, for example:

  1. Tokens

  2. ASDL

  3. PEG Grammar

  4. Parsing

  5. AST

  6. Compilation

  7. Validation

  8. Symbol Table

  9. Code Generation

  10. Bytecode

  11. Execution

By contributing to that document from time to time, we can build a clean, robust, and elegant tutorial that helps developers grasp the big picture and understand CPython’s internals easily.

Note: My point is about the idea itself, regardless of how it will be applied.

2 Likes

Could this be part of the grammar document, if deemed useful? I’m not sure we add new statements with sufficient frequency to have a dedicated document (which would itself need updating as details of the internal implementation change).

A

2 Likes