Skip to content

gh-125063: marshal: Add version 5, improve documentation #126829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 15, 2024

Conversation

encukou
Copy link
Member

@encukou encukou commented Nov 14, 2024

This builds on #126804, but I followed the rabbit hole much deeper.

In Python 3.14, slice objects can be marshalled.
This calls for a new version of the marshal format, so that backwards-incompatible
objects are rejected at serialization time.

  • Bump marshal version to 5
  • Reject slices in earlier versions
  • Reorganize the TYPE_ defines to make it easier to contribute
  • Adjust and improve the documentation.

📚 Documentation preview 📚: https://cpython-previews--126829.org.readthedocs.build/

Comment on lines +124 to 125
assert(Py_MARSHAL_VERSION >= 5);
PyObject *marshalled = PyMarshal_WriteObjectToString(code, Py_MARSHAL_VERSION);
Copy link
Member Author

@encukou encukou Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Makefile, _freeze_module doesn't depend on marshal.h, but it needs to be rebuilt when Py_MARSHAL_VERSION switches to 5.
Instead of properly changing the Makefile, which would be another rabbit hole, I decided to touch _freeze_module.c instead.

@encukou encukou added docs Documentation in the Doc dir and removed docs Documentation in the Doc dir labels Nov 14, 2024
Copy link
Contributor

@mdboom mdboom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I had no idea changing marshal required all this, so I appreciate the backfill.

@encukou encukou merged commit d00f7b1 into python:main Nov 15, 2024
43 checks passed
@encukou encukou deleted the marshal-v branch November 15, 2024 12:49
@encukou
Copy link
Member Author

encukou commented Nov 15, 2024

I also learned a lot.
Most of it's not required. I hope these changes will make it easier for the next person to notice what we missed :)

picnixz pushed a commit to picnixz/cpython that referenced this pull request Dec 8, 2024
…nGH-126829)

* Document that slices can be marshalled
* Deduplicate and organize the list of supported types
  in docs
* Organize the type code list in marshal.c, to make
  it more obvious that this is a versioned format
* Back-fill some historical info

Co-authored-by: Michael Droettboom <[email protected]>
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…nGH-126829)

* Document that slices can be marshalled
* Deduplicate and organize the list of supported types
  in docs
* Organize the type code list in marshal.c, to make
  it more obvious that this is a versioned format
* Back-fill some historical info

Co-authored-by: Michael Droettboom <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants