gh-129173: simplify PyCodec_XMLCharRefReplaceErrors logic

[picnixz]

• Issue: Refactor codecs error handlers to use _PyUnicodeError_GetParams and extract complex logic into separate functions #129173

[picnixz]

Huh, I probably messed up my index somewhere. Will fix it later or tomorrow.

[encukou]

I filed my suggestion as https://github.com/picnixz/cpython/pull/1/files

[picnixz]

I've applied your suggestion and tweaked it a bit. By the way, I observe that I forgot to remove a "object is ready" comment. Do you mind that after this PR and the following one for backslashreplace, I skim through the code base to remove the un-necessary related comments? there are a few occurrences of saying that some Unicode object is "ready"

[encukou]

That's a style change, which we generally only do when touching neighboring code.
On the other hand, the comments will be getting increasingly confusing as PyUnicode_READY/PyUnicode_WCHAR_KIND is forgotten and people don't associate “ready” with them.

How many are there? I'd need to review each of those removals in context.

[picnixz]

How many are there? I'd need to review each of those removals in context.

Apart from those already in codecs.c that I forgot to remove, not many:

cpython/Modules/unicodedata.c

Lines 594 to 596 in a85eeb9

	/* result is guaranteed to be ready, as it is compact. */
	kind = PyUnicode_KIND(result);
	data = PyUnicode_DATA(result);

cpython/Modules/unicodedata.c

Lines 655 to 661 in a85eeb9

	result = nfd_nfkd(self, input, k);
	if (!result)
	return NULL;
	/* result will be "ready". */
	kind = PyUnicode_KIND(result);
	data = PyUnicode_DATA(result);
	len = PyUnicode_GET_LENGTH(result);

cpython/Modules/_io/textio.c

Lines 357 to 363 in a85eeb9

	kind = PyUnicode_KIND(modified);
	out = PyUnicode_DATA(modified);
	PyUnicode_WRITE(kind, out, 0, '\r');
	memcpy(out + kind, PyUnicode_DATA(output), kind * output_len);
	Py_SETREF(output, modified); /* output remains ready */
	self->pendingcr = 0;
	output_len++;

cpython/Modules/_io/textio.c

Lines 1821 to 1824 in a85eeb9

	/* decoded_chars is guaranteed to be "ready". */
	avail = (PyUnicode_GET_LENGTH(self->decoded_chars)
	- self->decoded_chars_used);

cpython/Parser/lexer/lexer.c

Lines 311 to 315 in a85eeb9

	/* Verify that the identifier follows PEP 3131.
	All identifier strings are guaranteed to be "ready" unicode objects.
	*/
	static int
	verify_identifier(struct tok_state *tok)

cpython/Parser/pegen.c

Lines 505 to 513 in a85eeb9

	PyObject *
	_PyPegen_new_identifier(Parser *p, const char *n)
	{
	PyObject *id = PyUnicode_DecodeUTF8(n, (Py_ssize_t)strlen(n), NULL);
	if (!id) {
	goto error;
	}
	/* PyUnicode_DecodeUTF8 should always return a ready string. */
	assert(PyUnicode_IS_READY(id));

cpython/Python/tracemalloc.c

Lines 252 to 259 in a85eeb9

	if (!PyUnicode_IS_READY(filename)) {
	/* Don't make a Unicode string ready to avoid reentrant calls
	to tracemalloc_alloc() or tracemalloc_realloc() */
	#ifdef TRACE_DEBUG
	tracemalloc_error("filename is not a ready unicode string");
	#endif
	return;
	}

The tracemalloc one seems to be dead code.

[encukou]

Thanks!

[encukou]

You might want to split the comment removals to 5 PRs (each to be seen by different subject experts), and combine them with removing the remaining calls to PyUnicode_IS_READY.

Careful, the one in unicodeobject.c refers to a different kind of "ready", PyUnicode_Type.tp_flags & Py_TPFLAGS_READY.

[picnixz]

Careful, the one in unicodeobject.c refers to a different kind of "ready", PyUnicode_Type.tp_flags & Py_TPFLAGS_READY.

Oups, you're right.

[picnixz]

I really want to be able to preview the commit message when I'm enabling auto-merge...