Skip to content

PyLong_FromString() does not always set *pend on error #110865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
serhiy-storchaka opened this issue Oct 14, 2023 · 1 comment
Open

PyLong_FromString() does not always set *pend on error #110865

serhiy-storchaka opened this issue Oct 14, 2023 · 1 comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-C-API type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

According to the documentation, PyLong_FromString() sets *pend (if pend is not NULL) to the first character that could not be processed on error. But it does not do this in the following cases:

  1. base is invalid (not 0 or in range from 2 to 36).
  2. Binary base number exceeds sys.maxsize bits.
  3. Decimal number exceeds the limit of digits sys.get_int_max_str_digits().
  4. Other base numbers also have limits.
  5. Memory allocation error.
  6. User presses Ctrl-C when _pylong.int_from_string() is used.

The workaround is to initialize the pointer to some known value, for example the start of the string or NULL before using PyLong_FromString(). If initialize it to NULL, we can distinguish ValueError raised for invalid base or exceeding the sys.get_int_max_str_digits() limit from ValueError raised for invalid string content.

We should either document this and require initialization of the end pointer, or always set it on error in PyLong_FromString(). The latter is a potentially breaking change, because it can override the sentinel value used in the user code. But the former looks like bad design.

@serhiy-storchaka serhiy-storchaka added 3.11 only security fixes topic-C-API 3.12 only security fixes 3.13 bugs and security fixes labels Oct 14, 2023
@encukou
Copy link
Member

encukou commented Oct 17, 2023

Setting it to NULL might make existing code crash, and does not follow the current documentation. IMO, that makes it a no-go.
So, IMO it should be set to the start of the string.

Distinguishing e.g. ValueError from invalid base from ValueError from string contents doesn't seem that useful to me.
If it is useful, IMO we should add specialized API with a richer set of errors states it can report.

@skirpichev skirpichev removed 3.11 only security fixes 3.12 only security fixes labels Apr 19, 2025
@picnixz picnixz added interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error and removed 3.13 bugs and security fixes labels Apr 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-C-API type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants