readPreference secondaryPreferred causes "Unable to create case insensitive username index: not master" #7048

sunshineo · 2020-12-07T23:35:27Z

New Issue Checklist

I am not disclosing a vulnerability.
I am not just asking a question.
I have searched through existing issues.
I can reproduce the issue with the latest version of Parse Server.

Issue Description

When Mongo URI has readPreference set to secondaryPreferred (and connecting to a Mongo Atlas cluster) we get the following error:

[13973] parse-server running on http://localhost:1337/parse
warn: Unable to create case insensitive username index:  not master {"operationTime":"6903658739996295169","ok":0,"code":10107,"codeName":"NotMaster","$clusterTime":{"clusterTime":"6903658739996295169","signature":{"hash":"/wM8w3Ym3vb446s04YZqDHx5TTw=","keyId":"6847700517334810656"}},"name":"MongoError","stack":"MongoError: not master\n    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)\n    at MessageStream.emit (events.js:314:20)\n    at MessageStream.EventEmitter.emit (domain.js:486:12)\n    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)\n    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)\n    at writeOrBuffer (_stream_writable.js:352:12)\n    at MessageStream.Writable.write (_stream_writable.js:303:10)\n    at TLSSocket.ondata (_stream_readable.js:717:22)\n    at TLSSocket.emit (events.js:314:20)\n    at TLSSocket.EventEmitter.emit (domain.js:486:12)"}
MongoError: not master
    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at MessageStream.EventEmitter.emit (domain.js:486:12)
    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (_stream_writable.js:352:12)
    at MessageStream.Writable.write (_stream_writable.js:303:10)
    at TLSSocket.ondata (_stream_readable.js:717:22)
    at TLSSocket.emit (events.js:314:20)
    at TLSSocket.EventEmitter.emit (domain.js:486:12) {
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
  ok: 0,
  code: 10107,
  codeName: 'NotMaster',
  '$clusterTime': {
    clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
    signature: { hash: [Binary], keyId: [Long] }
  },
  [Symbol(errorLabels)]: Set(1) { 'RetryableWriteError' }
}

Steps to reproduce

Create a cluster with 3 nodes on MongoDB Atlas. I feel a cluster anywhere can reproduce

// This works
parse-server --appId myappid --masterKey masterkey --databaseURI "mongodb+srv:/<username>:<password>@cluster0.b2mxc.mongodb.net/<db-name>?retryWrites=true&w=3"

// Add &readPreference=secondaryPreferred&slaveOk=true to the end of the URI fails
parse-server --appId myappid --masterKey masterkey --databaseURI "mongodb+srv:/<username>:<password>@cluster0.b2mxc.mongodb.net/<db-name>?retryWrites=true&w=3&readPreference=secondaryPreferred&slaveOk=true"

Environment

Server

Parse Server version: 4.4.0
Operating system: Mac

Database

System (MongoDB or Postgres): MongoDB
Database version: 4.2.10
Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): MongoDB Atlas

Logs

[13973] parse-server running on http://localhost:1337/parse
warn: Unable to create case insensitive username index:  not master {"operationTime":"6903658739996295169","ok":0,"code":10107,"codeName":"NotMaster","$clusterTime":{"clusterTime":"6903658739996295169","signature":{"hash":"/wM8w3Ym3vb446s04YZqDHx5TTw=","keyId":"6847700517334810656"}},"name":"MongoError","stack":"MongoError: not master\n    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)\n    at MessageStream.emit (events.js:314:20)\n    at MessageStream.EventEmitter.emit (domain.js:486:12)\n    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)\n    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)\n    at writeOrBuffer (_stream_writable.js:352:12)\n    at MessageStream.Writable.write (_stream_writable.js:303:10)\n    at TLSSocket.ondata (_stream_readable.js:717:22)\n    at TLSSocket.emit (events.js:314:20)\n    at TLSSocket.EventEmitter.emit (domain.js:486:12)"}
MongoError: not master
    at MessageStream.messageHandler (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at MessageStream.EventEmitter.emit (domain.js:486:12)
    at processIncomingData (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/usr/local/lib/node_modules/parse-server/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (_stream_writable.js:352:12)
    at MessageStream.Writable.write (_stream_writable.js:303:10)
    at TLSSocket.ondata (_stream_readable.js:717:22)
    at TLSSocket.emit (events.js:314:20)
    at TLSSocket.EventEmitter.emit (domain.js:486:12) {
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
  ok: 0,
  code: 10107,
  codeName: 'NotMaster',
  '$clusterTime': {
    clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_: 1607383308 },
    signature: { hash: [Binary], keyId: [Long] }
  },
  [Symbol(errorLabels)]: Set(1) { 'RetryableWriteError' }
}

The text was updated successfully, but these errors were encountered:

dplewis · 2020-12-09T17:57:32Z

I believe this has been fixed in the latest version of the mongo driver.

https://jira.mongodb.org/browse/NODE-2784

We are looking into updating the driver.

mtrezza · 2020-12-09T18:08:40Z

Here are some workaround suggestions:

Remove the readPreference from the connection string and instead set it in the Parse Server configuration as driver option. Untested.
Set the readPreference as option of the Parse Query where you want to override the driver / connection string read preference. Untested.
Manually create the index in the DB.

sunshineo · 2020-12-09T18:16:30Z

I tried that. Same problem.

…

On Wed, Dec 9, 2020 at 10:08 AM Manuel ***@***.***> wrote: A workaround may be - I haven't tested it - to remove the readPreference from the connection string and instead set it in the Parse Server configuration as driver option. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7048 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIF47TTJGNMSGM5HCVEBL3ST64LRANCNFSM4URFMBNQ> .

sunshineo · 2020-12-09T22:49:19Z

@mtrezza manually create the index in the DB does not work. As a matter of fact, when I test removed the readPrefrence settings, it worked and created the index. But when add back the readPreference, code probably is trying to ensure index exists and still fails even the index is already there

parse-server/src/Controllers/DatabaseController.js

Line 1628 in ca1b782

this.adapter.ensureIndex(

mtrezza · 2020-12-09T23:57:47Z

manually create the index in the DB does not work.

I cannot reproduce this with an existing Parse Server deployment by adding readPreference=secondaryPreferred to the connection string. The server works just fine.

sunshineo · 2020-12-09T23:59:54Z

@mtrezza is your DB a cluster with 3 nodes (mine on MongoDB Atlas)?

mtrezza · 2020-12-10T00:00:53Z

Yes, I tried it with a 3 node (PSS) replica set. Just to be sure I also tried to use the exactly same connection string parameters as yours, still works. Maybe you want to post the created index?

You could also try to upgrade the Parse Server mongodb dependency to 3.6.3.

sunshineo · 2020-12-10T00:34:34Z

Upgrade to 3.6.3 definitely works. I guess I have to install from my repo for a while. I'm creating a PR

mtrezza · 2020-12-10T01:05:23Z

I reopened the issue because it still exists and the PR has not been merged yet.

dplewis · 2020-12-10T18:06:13Z

Closing via #7026

mtrezza added the needs more info label Dec 8, 2020

mtrezza added type:bug Impaired feature or lacking behavior that is likely assumed on hold and removed needs more info labels Dec 9, 2020

sunshineo closed this as completed Dec 10, 2020

sunshineo mentioned this issue Dec 10, 2020

Upgrade mongodb to 3.6.3 #7055

Closed

mtrezza reopened this Dec 10, 2020

sunshineo mentioned this issue Dec 10, 2020

[Snyk] Upgrade mongodb from 3.6.2 to 3.6.3 #7026

Merged

dplewis closed this as completed Dec 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

readPreference secondaryPreferred causes "Unable to create case insensitive username index: not master" #7048

readPreference secondaryPreferred causes "Unable to create case insensitive username index: not master" #7048

sunshineo commented Dec 7, 2020

dplewis commented Dec 9, 2020

mtrezza commented Dec 9, 2020 •

edited

Loading

sunshineo commented Dec 9, 2020 via email

sunshineo commented Dec 9, 2020

mtrezza commented Dec 9, 2020

sunshineo commented Dec 9, 2020

mtrezza commented Dec 10, 2020 •

edited

Loading

sunshineo commented Dec 10, 2020

mtrezza commented Dec 10, 2020 •

edited

Loading

dplewis commented Dec 10, 2020

readPreference secondaryPreferred causes "Unable to create case insensitive username index: not master" #7048

readPreference secondaryPreferred causes "Unable to create case insensitive username index: not master" #7048

Comments

sunshineo commented Dec 7, 2020

New Issue Checklist

Issue Description

Steps to reproduce

Environment

Logs

dplewis commented Dec 9, 2020

mtrezza commented Dec 9, 2020 • edited Loading

sunshineo commented Dec 9, 2020 via email

sunshineo commented Dec 9, 2020

mtrezza commented Dec 9, 2020

sunshineo commented Dec 9, 2020

mtrezza commented Dec 10, 2020 • edited Loading

sunshineo commented Dec 10, 2020

mtrezza commented Dec 10, 2020 • edited Loading

dplewis commented Dec 10, 2020

mtrezza commented Dec 9, 2020 •

edited

Loading

mtrezza commented Dec 10, 2020 •

edited

Loading

mtrezza commented Dec 10, 2020 •

edited

Loading