diff --git a/internals/compiler.rst b/internals/compiler.rst index 7b5df7b87e..2785aab51d 100644 --- a/internals/compiler.rst +++ b/internals/compiler.rst @@ -524,140 +524,112 @@ statement in ``_PyEval_EvalFrameDefault()``. Important Files =============== -+ Parser/ +* :cpy-file:`Parser/` + * :cpy-file:`Parser/Python.asdl`: ASDL syntax file. - Python.asdl - ASDL syntax file + * :cpy-file:`Parser/asdl.py`: Parser for ASDL definition files. + Reads in an ASDL description and parses it into an AST that describes it. - asdl.py - Parser for ASDL definition files. Reads in an ASDL description - and parses it into an AST that describes it. + * :cpy-file:`Parser/asdl_c.py`: Generate C code from an ASDL description. + Generates :cpy-file:`Python/Python-ast.c` and + :cpy-file:`Include/internal/pycore_ast.h`. - asdl_c.py - "Generate C code from an ASDL description." Generates - :cpy-file:`Python/Python-ast.c` and :cpy-file:`Include/internal/pycore_ast.h`. + * :cpy-file:`Parser/parser.c`: The new PEG parser introduced in Python 3.9. + Generated by :cpy-file:`Tools/peg_generator/pegen/c_generator.py` + from the grammar :cpy-file:`Grammar/python.gram`. Creates the AST from + source code. Rule functions for their corresponding production rules + are found here. - parser.c - The new PEG parser introduced in Python 3.9. - Generated by :cpy-file:`Tools/peg_generator/pegen/c_generator.py` - from the grammar :cpy-file:`Grammar/python.gram`. Creates the AST from - source code. Rule functions for their corresponding production rules - are found here. + * :cpy-file:`Parser/peg_api.c`: Contains high-level functions which are + used by the interpreter to create an AST from source code. - peg_api.c - Contains high-level functions which are used by the interpreter to - create an AST from source code . + * :cpy-file:`Parser/pegen.c`: Contains helper functions which are used + by functions in :cpy-file:`Parser/parser.c` to construct the AST. + Also contains helper functions which help raise better error messages + when parsing source code. - pegen.c - Contains helper functions which are used by functions in - :cpy-file:`Parser/parser.c` to construct the AST. Also contains helper - functions which help raise better error messages when parsing source - code. + * :cpy-file:`Parser/pegen.h`: Header file for the corresponding + :cpy-file:`Parser/pegen.c`. Also contains definitions of the ``Parser`` + and ``Token`` structs. - pegen.h - Header file for the corresponding :cpy-file:`Parser/pegen.c`. Also contains - definitions of the ``Parser`` and ``Token`` structs. +* :cpy-file:`Python/` + * :cpy-file:`Python/Python-ast.c`: Creates C structs corresponding to + the ASDL types. Also contains code for marshalling AST nodes (core + ASDL types have marshalling code in :cpy-file:`Python/asdl.c`). + "File automatically generated by :cpy-file:`Parser/asdl_c.py`". + This file must be committed separately after every grammar change + is committed since the ``__version__`` value is set to the latest + grammar change revision number. -+ Python/ + * :cpy-file:`Python/asdl.c`: Contains code to handle the ASDL sequence type. + Also has code to handle marshalling the core ASDL types, such as number + and identifier. Used by :cpy-file:`Python/Python-ast.c` for marshalling + AST nodes. - Python-ast.c - Creates C structs corresponding to the ASDL types. Also - contains code for marshalling AST nodes (core ASDL types have - marshalling code in :cpy-file:`Python/asdl.c`). "File automatically - generated by :cpy-file:`Parser/asdl_c.py`". This file must be - committed separately after every grammar change is committed since - the ``__version__`` value is set to the latest grammar change - revision number. + * :cpy-file:`Python/ast.c`: Used for validating the AST. - asdl.c - Contains code to handle the ASDL sequence type. Also has code - to handle marshalling the core ASDL types, such as number and - identifier. Used by :cpy-file:`Python/Python-ast.c` for marshalling - AST nodes. + * :cpy-file:`Python/ast_opt.c`: Optimizes the AST. - ast.c - Used for validating the AST. + * :cpy-file:`Python/ast_unparse.c`: Converts the AST expression node + back into a string (for string annotations). - ast_opt.c - Optimizes the AST. + * :cpy-file:`Python/ceval.c`: Executes byte code (aka, eval loop). - ast_unparse.c - Converts the AST expression node back into a string - (for string annotations). + * :cpy-file:`Python/compile.c`: Emits bytecode based on the AST. - ceval.c - Executes byte code (aka, eval loop). + * :cpy-file:`Python/symtable.c`: Generates a symbol table from AST. - compile.c - Emits bytecode based on the AST. + * :cpy-file:`Python/peephole.c`: Optimizes the bytecode. - symtable.c - Generates a symbol table from AST. + * :cpy-file:`Python/pyarena.c`: Implementation of the arena memory manager. - peephole.c - Optimizes the bytecode. + * :cpy-file:`Python/wordcode_helpers.h`: Helpers for generating bytecode. - pyarena.c - Implementation of the arena memory manager. + * :cpy-file:`Python/opcode_targets.h`: One of the files that must be + modified if :cpy-file:`Lib/opcode.py` is. - wordcode_helpers.h - Helpers for generating bytecode. +* :cpy-file:`Include/` + * :cpy-file:`Include/code.h`: Header file for :cpy-file:`Objects/codeobject.c`; + contains definition of ``PyCodeObject``. - opcode_targets.h - One of the files that must be modified if :cpy-file:`Lib/opcode.py` is. + * :cpy-file:`Include/opcode.h`: One of the files that must be modified if + :cpy-file:`Lib/opcode.py` is. -+ Include/ + * :cpy-file:`Include/internal/pycore_ast.h`: Contains the actual definitions + of the C structs as generated by :cpy-file:`Python/Python-ast.c`. + "Automatically generated by :cpy-file:`Parser/asdl_c.py`". - code.h - Header file for :cpy-file:`Objects/codeobject.c`; contains definition of - ``PyCodeObject``. + * :cpy-file:`Include/internal/pycore_asdl.h`: Header for the corresponding + :cpy-file:`Python/ast.c`. - opcode.h - One of the files that must be modified if :cpy-file:`Lib/opcode.py` is. + * :cpy-file:`Include/internal/pycore_ast.h`: Declares ``_PyAST_Validate()`` + external (from :cpy-file:`Python/ast.c`). - + internal/ + * :cpy-file:`Include/internal/pycore_symtable.h`: Header for + :cpy-file:`Python/symtable.c`. ``struct symtable`` and ``PySTEntryObject`` + are defined here. - pycore_ast.h - Contains the actual definitions of the C structs as generated by - :cpy-file:`Python/Python-ast.c`. - "Automatically generated by :cpy-file:`Parser/asdl_c.py`". + * :cpy-file:`Include/internal/pycore_parser.h`: Header for the + corresponding :cpy-file:`Parser/peg_api.c`. - pycore_asdl.h - Header for the corresponding :cpy-file:`Python/ast.c` + * :cpy-file:`Include/internal/pycore_pyarena.h`: Header file for the + corresponding :cpy-file:`Python/pyarena.c`. - pycore_ast.h - Declares ``_PyAST_Validate()`` external (from :cpy-file:`Python/ast.c`). +* :cpy-file:`Objects/` + * :cpy-file:`Objects/codeobject.c`: Contains PyCodeObject-related code + (originally in :cpy-file:`Python/compile.c`). - pycore_symtable.h - Header for :cpy-file:`Python/symtable.c`. ``struct symtable`` and - ``PySTEntryObject`` are defined here. + * :cpy-file:`Objects/frameobject.c`: Contains the ``frame_setlineno()`` + function which should determine whether it is allowed to make a jump + between two points in a bytecode. - pycore_parser.h - Header for the corresponding :cpy-file:`Parser/peg_api.c`. +* :cpy-file:`Lib/` + * :cpy-file:`Lib/opcode.py`: Master list of bytecode; if this file is + modified you must modify several other files accordingly + (see "`Introducing New Bytecode`_") - pycore_pyarena.h - Header file for the corresponding :cpy-file:`Python/pyarena.c`. - - -+ Objects/ - - codeobject.c - Contains PyCodeObject-related code (originally in - :cpy-file:`Python/compile.c`). - - frameobject.c - Contains the ``frame_setlineno()`` function which should determine - whether it is allowed to make a jump between two points in a bytecode. - -+ Lib/ - - opcode.py - Master list of bytecode; if this file is modified you must modify - several other files accordingly (see "`Introducing New Bytecode`_") - - importlib/_bootstrap_external.py - Home of the magic number (named ``MAGIC_NUMBER``) for bytecode - versioning. + * :cpy-file:`Lib/importlib/_bootstrap_external.py`: Home of the magic number + (named ``MAGIC_NUMBER``) for bytecode versioning. Known Compiler-related Experiments