Racing condition in compiling queries after upgrading from 5.0.x to 5.2.x
|
marius.ropotica@...
Hello,
We have recently upgraded Datanucleus from 5.0 to 5.2 and we started to see broken queries, here are some examples: # this query is cut after the WHERE cause SELECT 'com.somepackage.EntityOne' AS `dn_type`, `a0`.`creationdate`, `a0`.`encodedkey`, `a0`.`NAME`, `a0`.`entityone_encodedkey_own` FROM `entityone` `a0` WHERE # this query have multiple issues: some parts are duplicated (see the first line), some nulls appear in the field list SELECT 'com.somepackage.AnotherEntity''com.somepackage.AnotherEntity' AS `dn_type`,, `a0`.`active`, `a0`.`amount`, nullnullnullnullnullnull, `a0`.`id`, `a0`.`lastmodifieddate`, `a0`.`NAME`, `a0`.`TRIGGER`, `a0`.`anotherentity_encodedkey_own`, `a0`.`anotherentity_integer_idx` AS `nucorder0` FROM `anotherentity` `a0`
|
|
|
|
Hi Marius,
no, never seen that situation. If you want to track it down, find the stack trace of that method call; as the commit says, any locking should be further out in the call chain. PS 5.x is no longer maintained. |
|
|
|
marius.ropotica@...
Hi,
Thanks for your quick response. I already tried what you have suggested buth with no luck so far. I'll have a closer look. Marius |
|
|
|
Define how you are using PMF, PM with respect to threads. FWIW "Multithreaded" PMs has always been a recipe for pain.
I would expect each query to be used by a single thread, and hence such locking be unnecessary. |
|
|
|
marius.ropotica@...
Hi,
PMs or query objects are not reused, each time a query is executed a new PM object is created. The PMF is created at the application startup. I tried to see what happens if we reuse PMs or queries, I got all sorts of exceptions like NPE, class cast exception, and many more. But nothing like I mentioned in the first post. Currenlty I'm trying to reproduce the issue in a dev / local environment, no luck so far. Could you suggest anything else I should check? Thanks, Marius |
|
|
|
marius.ropotica@...
Hi,
I think we have found the issue. Please have a look here https://github.com/datanucleus/datanucleus-rdbms/blob/master/src/main/java/org/datanucleus/store/rdbms/query/JDOQLQuery.java#L708 At this line, getSQLText() is called on a select statement that comes from the query compilation cache. From what I understand, the query compilation cache comes from the PMF, so it's shared by all PMs and query objects. I don't see anything that would prevent another thread to access the same select statement and call the getSQLText() method. Can somebody have a look an confirm our findings ? Thanks, Marius |
|
|
|
You could easily enough make a change to such as "SQLText.toSQL" to make the setting of "sql" be within a synchronised block and see what effect it has on your application, since that is the only thing that is generated by an SQLText.
|
|
|
|
marius.ropotica@...
That's exactly what we intend to do. Will let you know once we make the change and run some tests.
Thanks, Marius |
|
|
|
marius.ropotica@...
Hi,
It was faster for us to just revert this commit https://github.com/datanucleus/datanucleus-rdbms/commit/548e18d6babcddf2271a2bba312139eb58f2d742 . No more broken queries after this change. Marius |
|
|