Skip to content

bpo-31770: Prevent a crash and refleaks when calling sqlite3.Cursor.__init__() more than once #3968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 7, 2017

Conversation

orenmn
Copy link
Contributor

@orenmn orenmn commented Oct 12, 2017

In addition, add a test to test_sqlite to make sure that the crash is no more.

https://bugs.python.org/issue31770

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except of "del ref".

@serhiy-storchaka: Would you mind to double-check this PR please?

It seems like "self->in_weakreflist" is already set to NULL by the memory allocator, so it's safe to remove "self->in_weakreflist = NULL;".

ref = weakref.ref(cur, callback)
cur.__init__(con)
del cur
del ref # shouldn't crash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the purpose of removing explicitly the last reference to a weak reference object. Just remove "del ref" ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least on my Windows 10, when the test fails, the del ref causes the crash info that is printed to include the line (and test) that caused the crash
(i.e. File "C:\Users\orenm\cpython\lib\sqlite3\test\regression.py", line 392 in CheckBpo31770). If i remove the del ref, it doesn't show info about the line and test that caused the crash.
If you still think that i should remove it, i wouldn't mind, of course.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add test.support.gc_collect() for ensuring that objects are collected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When i use test.support.gc_collect(), the crash info also doesn't include the line and the test that crashed.
Also, when i add 1/0 after the call to gc_collect(), and run the test (on cpython without the patch in C code), the code doesn't crash in gc_collect(), and a ZeroDivisionError is raised...
(of course, without the 1/0, it does crash, but after the gc_collect(), i guess.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should i replace the del ref with a call to test.support.gc_collect() anyway?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you call test.support.gc_collect() instead of del ref? Or adding test.support.gc_collect() after del ref breaks the test?

I meant that you should keep del ref, but call test.support.gc_collect() after it for sure. Of course, if this breaks the test, don't add it.

@vstinner
Copy link
Member

I didn't approve the PR yet because of my question on "del ref".

@vstinner
Copy link
Member

vstinner commented Oct 15, 2017 via email

@orenmn
Copy link
Contributor Author

orenmn commented Oct 20, 2017

IIUC, calling gc_collect() without doing del ref wouldn't really cause the collection of ref, because there is still one reference to it until the method is over. The call to gc_collect() is to make sure ref is collected, as Serhiy advised.
Is it ok, Victor, or am i missing something?

@miss-islington
Copy link
Contributor

Thanks @orenmn for the PR, and @Haypo for merging it 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 7, 2017
@bedevere-bot
Copy link

GH-4301 is a backport of this pull request to the 3.6 branch.

@miss-islington
Copy link
Contributor

Thanks @orenmn for the PR, and @Haypo for merging it 🌮🎉.. I'm working now to backport this PR to: 2.7.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-4302 is a backport of this pull request to the 2.7 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 7, 2017
vstinner pushed a commit that referenced this pull request Nov 7, 2017
…_init__() more than once (GH-3968) (#4302)

(cherry picked from commit e56ab74)
vstinner pushed a commit that referenced this pull request Nov 7, 2017
…_init__() more than once (GH-3968) (#4301)

(cherry picked from commit e56ab74)
embray pushed a commit to embray/cpython that referenced this pull request Nov 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants