Parse.Object.saveAll() get slower after each call

[rasool1994]

Issue Description

I have a cloud function that acts as an API for getting data from another server.
let suppose server x read some data from an MSSQL database and send every 1000 record in one request to my parse server API.(ex: 13 calls for 13000 records)
every time that API called, Parse.Object.saveAll() would work slower.

cloud code

console.log("A");
  let A = new Date();
  console.log(A);

  for (let i = 0; i < 1000; i++) {
    const ScoreLog = Parse.Object.extend("ScoreLog");
    let scoreLog = new ScoreLog();
    scoreLog.set("type", 0);
    scoreLog.set("cId", PARAMS.scoreLogs[i].CustomerID.toString());
    scoreLog.set("score", PARAMS.scoreLogs[i].Credit);
    scoreLog.set("logAt", new Date(PARAMS.scoreLogs[i].TRBusinessDayDate));
    scoreLog.set("sNum", PARAMS.scoreLogs[i].SequenceNumer);
    scoreLogs.push(scoreLog);
  }

  console.log("B");
  let B = new Date();
  console.log(B - A);
  await Parse.Object.saveAll(scoreLogs);
  console.log("C");
  let C = new Date();
  console.log(C - B);

Expected Results

save each 1000 records at the same time

Actual Outcome

increase save time as request go along

Environment Setup

• Server

  • parse-server version : 3.9.0
  • Operating System: ubuntu 18.04
  • Hardware: CPU intel corei7, RAM 16 g DDR4
  • Localhost or remote server? : localhost

• Database

  • MongoDB version: v4.2.0
  • Storage engine: wiredTiger
  • Hardware: CPU intel corei7, RAM 16 g DDR4
  • Localhost or remote server? : localhost

Logs/Trace

2019-12-26T18:00:32.004Z
B
59
C
941
A
2019-12-26T18:00:33.016Z
B
101
C
3317
A
2019-12-26T18:00:36.443Z
B
143
C
5790
A
2019-12-26T18:00:42.382Z
B
302
C
8169
A
2019-12-26T18:00:50.859Z
B
258
C
8542
A
2019-12-26T18:00:59.666Z
B
271
C
12515
A
2019-12-26T18:01:12.458Z
B
315
C
13761
A
2019-12-26T18:01:26.541Z
B
494
C
17816

Update

I tried replacing mongoose API with parse object handling system and the result was amazing
100000 record saved in 8 seconds.why parse API is so slow in MongoDB case?

[dplewis]

You could try directAccess and enableSingleSchemaCache server options.

We are currently looking into improving the caching as well.

[omkar-tenkale]

@rasool1994 Can you share details on how you achieved the 100000 performance improvement?

[mtrezza]

There are many possible causes for the described issue here, the bottleneck could be the database instance, the server instance, the parse server or the custom app logic.

I'm closing this as it lacks essential information required to investigate the issue. Please follow up on any questions asked and further steps suggested in the thread and we can re-open this issue.

[bobert600]

I would also like to know how rasool1994 resolved this -- anyone have more details on how this was resolved? I will play around with mongoose in the meantime to see if that'll solve it instead of using Parse.Object.saveAll().