Segfault in PythonQtPrivate::createNewPythonQtInstanceWrapper #33
Comments
|
The stacktrace alone is insufficient to guess the cause of this crash, I'm afraid you need to look at the state of the used variables in a debugger to determine what went wrong, probably starting at frame 2 (since the PyObject_Call is the first thing done in createNewPythonQtInstanceWrapper). |
|
I will check in wrapPtr what the state of variables are. The scripting interface is working in many other places so its not a generic issue. This issue originates by a signal which is evident in the stacktrace as well so perhaps PythonQt became more strict (or does things differently now) and something in that QVariantList is missing a meta type declaration. I'll report back. |
|
So the segfault happens on a bespoke data type but it has been declared with Q_DECLARE_METATYPE. It also works for the same data type on two previous calls and for another bespoke datatype which has been declared with Q_DECLARE_METATYPE. The difference between the three calls for this data type is that in the first call to wrapPtr for this data type wrap is null, info is not null after call to getClassInfo, a downcasting takes place as it is found is not a QObject, it is registered as a CPP class, a possible alive wrapper is not found and there is a call to createNewPythonQtInstanceWrapper which succeeds. The exactly same thing happens the second and third time except it is not registered as a CPP class in that call which would imply that info->pythonQtClassWrapper() returns non-NULL. In the third call for this data type it segfaults in the call to createNewPythonQtInstanceWrapper. In call three calls to createNewPythonQtInstanceWrapper, wrapper argument is NULL, info argument is a pointer with the same address in each case, and ptr argument is a pointer with a different address in each case. The bespoke data type is question is a struct and a member of a class. Hence it cannot be a QObject as Qt meta object compiler (moc) doesn't allowed nested QObject. It's not possible to declare a QObject class inside a QObject class. I also do not subclass QObject for these bespoke data types as I need to be able to copy them. |
|
I also checked the return value of info->pythonQtClassWrapper() in createNewPythonQtInstanceWrapper and it returns the same pointer address for all three calls. On the third call for this data type it segfaults in the call to PyObject_Call. |
|
So I have done a day of debugging and looking at the use of the the bespoke data type. I could not identify any internal issue in the struct attributes such as invalid or null QVariant between the three calls. I then tried disabling use of the data type and various callers and users of it. What I could reproduce is that the segfault always happens on the third call to wrapPtr for the data type. So to be that would seem to mean that the initial wrapping of it using the cpp wrapper works, and does the second call using of the already wrapped one, while the third call trying to use the already wrapped one fails. I will try next and see what happens if I disable the caching of the wrapped data type. |
|
This is just a hunch, but you could try to revert the changes done by Gregor Anich to fix some memory leaks and see if this helps. Perhaps one of the changes was a little bit to eager or requires further changes at another place. |
|
That's quite a lot of changes. I will try and see if I can rebase without all his commits and see what happens. I rather have memory leaks than segfaults. |
|
I did a rebase and removed the following commits: pick 06860d4 Fix reference/memory leaks It now works fine. So the issue is probably too aggressive "garbage collection" so the a pointer to the CPP wrapper is accessed after being freed. I would need to help of @gregor-anich-uibk to figure out what in all of his changes introduced this regression. |
|
You could try to use valgrind to find out more about what's going on, if you could provide a short piece of code that triggers the error I can also look into it to see if your use-case is strange or if a bug in PythonQt needs to be fixed. |
|
Well, my use case is that I use two CPP classes/structs which are declared as meta type with Q_DECLARE_METATYPE and have worked fine for the last three years using trunk-r502 and official releases before that, and now also work fine after removing the five commits listed above. My segfault only occurs for my two wrapped CPP objects and never for the thousands of wrapped QObject objects - so there is something fishy when dealing with CPP wrapped data types/classes which was introduced in one of the five above listed commits. I don't think I can get funding to develop the short piece of code to reproduce this as the problem with Python 3.8 support is now resolved using the latest code with the five commits listed above removed and we probably can use the new baseline of PythonQt for the next 10 years until it is time for RHEL 9 or Ubuntu 30.04 LTS :-) Works fine now on CentOS 6.x (Python 2.6, Qt 5.6), CentOS 7.x (Python 2.7, Qt 5.9), CentOS 8.x (Python 3.6, Qt 5.12), and Ubuntu 20.04 LTS (Python 3.8, Qt 5.12). But I can try to test on my own personal time if there are potential fixes to this in order to assist the community as there might be other users of wrapped CPP objects who do not want to remove the five commits listed above which might also become more complicated to do cleanly in the future as newer changes and made on top of them. I have attached six sample errors from valgrind below: |
|
Can confirm I'm seeing this same issue working from the same December 2 commit 9ea0fcf. In my case, there are no custom types involved. I'm just passing some heavily nested VariantMaps and VariantLists through to Python. The crash is in the same spot and I suspect for the same reason. And it also occurs randomly indicating likely use after free issues. I can do the same operation one time and it works and the next time it fails. My stacktrace looks like this (Windows): I'm going to attempt reverting those same commits (thank you @he-hesce!). |
|
Confirmed. Branching off of 9ea0fcf and reverting these commits: pick 06860d4 Fix reference/memory leaks seems to have stopped the invalid memory accesses. |
|
I have created a branch "revert-refcount-fixes" in which the relevant commits are reverted (but I spared the changes to python.prf, since I don't believe that they caused this problem). I would like to have the refcount fixes, but don't have the time to look into this. |
|
@davidkhess: Great and thanks to see a confirmation on this issue. @usiems: Great and thanks, this will allow us to upgrade to later PythonQt revisions in the future. |
|
@he-hesce - the problematic commits have been reverted some time ago, so I guess this not an issue anymore? |
|
@mrbean-bremen : Indeed, this is not an issue after the commits were reverted. Current tagged release is solid as a rock. |
Dear PythonQt community,
I recent upgraded from the old trunk r502 (sourceforge) to latest code on GitHub master branch, committish 9ea0fcf of 2 December 2020. I did this in order to get compatibility with Python 3.8. Using Python 2.7 and 3.6 have both worked fine with "r502".
Now when I try to use the latest PythonQt with code which previously worked fine, I get a segmentation fault. Looking at some (somewhat) recent commits I see there has been some work going on in the area where the segfault happens. Though it's hard for me to identify which commit introduced the issue.
The backtrace is here:
I am using CentOS 7.9.2009 which has Python 2.7.5 and Qt 5.9.7.
I would very much appreciate the community's and the developers' insight. Thank you.
The text was updated successfully, but these errors were encountered: