Skip to content

memory issue with where conditions #1642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
mustafauysal opened this issue Apr 26, 2016 · 9 comments
Closed
3 tasks done

memory issue with where conditions #1642

mustafauysal opened this issue Apr 26, 2016 · 9 comments

Comments

@mustafauysal
Copy link

Environment Setup

Ubuntu 14.04
node v5.10.1
npm 3.8.3
mongod 3.0.11
parse-server 2.2.7

Steps to reproduce

When I try to get installation objects with where conditions, parse server fails.

example request:

curl -XGET \
  -H "X-Parse-Application-Id: ****************************************" \
  -H "X-Parse-Master-Key: ****************************************" \
  -G \
  --data-urlencode 'where={"userId":"244862"}' \
  https://example.com/parse/installations

Logs/Trace

verbose: GET /parse/installations?where=%7B%22userId%22%3A%22244862%22%7D { 'x-real-ip': '78.183.50.33',
  'x-forwarded-for': '78.183.50.33',
  'x-nginx-proxy': 'true',
  host: 'example.com',
  connection: 'close',
  'user-agent': 'curl/7.43.0',
  accept: '*/*',
  'x-parse-application-id': '****************************************',
  'x-parse-master-key': '****************************************' } {}

<--- Last few GCs --->

   72055 ms: Scavenge 1401.3 (1458.2) -> 1401.3 (1458.2) MB, 6.5 / 0 ms (+ 3.1 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
   74474 ms: Mark-sweep 1401.3 (1458.2) -> 1401.3 (1457.2) MB, 2418.8 / 0 ms (+ 4.6 ms in 2 steps since start of marking, biggest step 3.1 ms) [last resort gc].
   76873 ms: Mark-sweep 1401.3 (1457.2) -> 1401.0 (1458.2) MB, 2398.5 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x64ae50e3ac1 <JS Object>
    1: /* anonymous */(aka /* anonymous */) [/usr/lib/node_modules/parse-server/node_modules/bson/lib/bson/parser/deserializer.js:~37] [pc=0xacaf5591e07] (this=0x64ae5004189 <undefined>,buffer=0x110c7fbb9809 <an Uint8Array with map 0x29897e8051d9>,index=0,options=0x110c7fbb9859 <an Object with map 0x2ab942a8e5b9>,isArray=0x64ae5004189 <undefined>)
    2: /* anonymous */(aka /* anonymous */) [/us...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Aborted (core dumped)
@hramos
Copy link
Contributor

hramos commented Apr 26, 2016

How much memory does your server have?

@mustafauysal
Copy link
Author

My server has 2G memory (1.3G available at the time)

@steven-supersolid
Copy link
Contributor

Some more ideas to try and identify the problem:

  • How many installations are returned by that query? (I think there is no default query limit on parse-server. Can check by running the same query on parse.com)
  • How many installations do you have?
  • Do you get the same issue with other classes? I.e. is this Installation class specific?
  • Are you running mongod on a separate instance?
  • Is there an index on userId?
  • What is executed on mongod? E.g. Use db.setProfilingLevel(2)

@mustafauysal
Copy link
Author

How many installations are returned by that query? (I think there is no default query limit on parse-server. Can check by running the same query on parse.com)

Parse.com has own limit, and returns various results, depends on user's installation.

How many installations do you have?

~2.65 M

Do you get the same issue with other classes? I.e. is this Installation class specific?

We use parse for push messages only and we don't have another class.
(schemas end point returns _Installation class only)

Are you running mongod on a separate instance?

Nope, single instance only.

Is there an index on userId?

Imported from parse database, it should have index.

What is executed on mongod? E.g. Use db.setProfilingLevel(2)

Somehow, parse server could not apply where clause (I guess)

{ 
    "op" : "query", 
    "ns" : "prodtest._SCHEMA", 
    "query" : {

    }, 
    "ntoreturn" : NumberInt(1000), 
    "ntoskip" : NumberInt(0), 
    "nscanned" : NumberInt(0), 
    "nscannedObjects" : NumberInt(3), 
    "keyUpdates" : NumberInt(0), 
    "writeConflicts" : NumberInt(0), 
    "numYield" : NumberInt(0), 
    "locks" : {
        "Global" : {
            "acquireCount" : {
                "r" : NumberLong(2)
            }
        }, 
        "MMAPV1Journal" : {
            "acquireCount" : {
                "r" : NumberLong(1)
            }
        }, 
        "Database" : {
            "acquireCount" : {
                "r" : NumberLong(1)
            }
        }, 
        "Collection" : {
            "acquireCount" : {
                "R" : NumberLong(1)
            }
        }
    }, 
    "nreturned" : NumberInt(3), 
    "responseLength" : NumberInt(1100), 
    "millis" : NumberInt(0), 
    "execStats" : {
        "stage" : "COLLSCAN", 
        "filter" : {
            "$and" : [

            ]
        }, 
        "nReturned" : NumberInt(3), 
        "executionTimeMillisEstimate" : NumberInt(0), 
        "works" : NumberInt(5), 
        "advanced" : NumberInt(3), 
        "needTime" : NumberInt(1), 
        "needFetch" : NumberInt(0), 
        "saveState" : NumberInt(0), 
        "restoreState" : NumberInt(0), 
        "isEOF" : NumberInt(1), 
        "invalidates" : NumberInt(0), 
        "direction" : "forward", 
        "docsExamined" : NumberInt(3)
    }, 
    "ts" : ISODate("2016-04-30T20:14:04.171+0000"), 
    "client" : "188.166.x.x", 
    "allUsers" : [
        {
            "user" : "username", 
            "db" : "databasename"
        }
    ], 
    "user" : "username@databasename"
}

@steven-supersolid
Copy link
Contributor

I just put some extra debugging in parse-server and ran locally with the same query. this.restWhere is {} in RestQuery in this case, so would return all documents.

I removed the -G option from the curl and this.restWhere becomes the correct {"userId":"244862"} which works correctly. The verbose output is:

verbose: GET /parse/installations { host: 'localhost:1337',
  'user-agent': 'curl/7.45.0',
  accept: '*/*',
  'x-parse-application-id': 'appId',
  'x-parse-master-key': 'masterKey',
  'content-length': '39',
  'content-type': 'application/x-www-form-urlencoded' } {
  "where": "{\"userId\":\"244862\"}"
}

So it seems there is a problem with extracting the query from URL parameters.

@steven-supersolid
Copy link
Contributor

I think I found the problem and solution. Will attempt a PR

@mustafauysal
Copy link
Author

@steven-supersolid thanks for that.
@hramos Any thought on this?

steven-supersolid added a commit to supersolid/parse-server that referenced this issue May 2, 2016
drew-gross pushed a commit that referenced this issue May 6, 2016
* Fix for #1642 - copy query parameters to request body

* Add missing request.query to pass test
@flovilmart
Copy link
Contributor

It seems to be fixed by #1687, please reopen if the error still occurs

@mustafauysal
Copy link
Author

Yeah, it fixed. I forgot to close this issue. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants