CSV reader should support QUOTE_NOTNULL and QUOTE_STRINGS

[serhiy-storchaka]

Feature or enhancement

New quoting rules QUOTE_NOTNULL and QUOTE_STRINGS were introduced in #67230. But they only affect CSV writer, not CSV reader. I think that they should affect CSV reader in the same way as QUOTE_NONNUMERIC does.

• QUOTE_NOTNULL -- unquoted empty strings are returned as None.
• QUOTE_STRINGS -- unquoted empty strings are returned as None and unquoted numeric strings are returned as float.

It is perhaps too late to change this in 3.12, so it can be considered as a new feature in 3.13.

Linked PRs

• gh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in csv.reader #113738

[serhiy-storchaka]

On other hand, it is the documented behavior. So this can be considered as a bugfix.

• It is a bug fix in 3.12.2.
• It is a new feature in 3.13. 3.12.2 needs a documentation fix, and the change between 3.12 and 3.13 is breaking.

[serhiy-storchaka]

I think that it is less disruptive than alternatives. For now, not much people use 3.12, and even less use new quoting options. But the mass adaptation of 3.12 will start before releasing 3.13, and at that time more code can be broken by 3.13 changes.

1. Fix this in 3.12.2. It is only disruptive for these who use new quoting options in 3.12.0 and 3.12.1.
2. Change the behavior in 3.13 without warning. It is disruptive for all users of 3.12, and also for users of 3.13 who occasionally run their code on the contemporary supported 3.12.
3. Emit a FutureWarning in 3.13 and 3.14 and change the behavior in 3.15. It is how such changes should be done (when change an established behavior), but it increases the number of affected people and delays the proper solution.
4. Emit a FutureWarning in 3.12.2+ and 3.13 and change the behavior in 3.14.
5. Emit a FutureWarning in 3.12.2+ and change the behavior in 3.13.

It was better to do this before 3.12.0, but it was not done, and now we have no good options. The longer we delay the solution, the worse options we have. cc @Yhg1s

[merwok]

Ah, I missed that the options were introduced in 3.12! Option 1 to fix in 3.12.2 seems great then. Great that you tagged the release manager to confirm.

[Yhg1s]

We should consider observable behaviour, rather than documentation, to be leading. Changing the behaviour in 3.12.2 is too disruptive. Patch releases also should not add new warnings. Given that it was documented to behave this way, I don't think it's necessary to add a new warning for this. That it doesn't work as documented is a bug we can't fix in 3.12, but we can in 3.13.

[merwok]

That it doesn't work as documented is a bug we can't fix in 3.12, but we can in 3.13.

Would it be useful to add a note in the 3.12 docs?

[Prometheus3375]

Adding a note to the documentation will be useful indeed. Was just writing some .csv involved code and because of the documentation of new quoting constants increased python version. Then I realized that these constants does not work for reading as documentation suggests. Hopefully, I found this issue.

Actually, instead of the note of bugged behavior maybe it will be better to simply remove reader instruction in 3.12 documentation and then add in 3.13 as an addition.

Also, one of the None is not properly formatted.

[merwok]

instead of the note of bugged behavior maybe it will be better to simply remove reader instruction in 3.12 documentation

I don’t think so: it is common to read a newer version of the docs than the Python version you’re using. It’s best to be clear about the behaviour and the change here.

Does someone want to open a pull request for this?
If not, we should open a doc ticket to make sure this doesn’t get forgotten.