Skip to content

DOCSP-47023 Connection pools #505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
240 changes: 239 additions & 1 deletion source/connect/connection-options/connection-pools.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,242 @@
Connection Pools
================

.. TODO
.. contents:: On this page
:local:
:backlinks: none
:depth: 2
:class: singlecol

.. facet::
:name: genre
:values: reference

Overview
--------

In this guide, you can learn about how the {+driver-short+} uses connection
pools to manage connections to a MongoDB deployment and how you can configure
connection pool settings in your application.

A connection pool is a cache of open database connections maintained by the
{+driver-short+}. When your application requests a connection to MongoDB, the
{+driver-short+} seamlessly gets a connection from the pool, performs
operations, and returns the connection to the pool for reuse.

Connection pools help reduce application latency and the number of times new
connections are created by {+driver-short+}.

.. _golang-faq-connection-pool:

Create a Connection Pool
------------------------

Every ``Client`` instance has a built-in connection pool for each server in your
MongoDB topology. If you do not configure the ``minPoolSize`` option, connection
pools open sockets on demand. These sockets support concurrent MongoDB
operations, or `goroutines <https://go.dev/tour/concurrency/1>`__, in
your application.

When a new ``Client`` instance is instatiated, it opens two sockets per server
in your MongoDB topology for monitoring the server's state.

For example, a client connected to a three-node replica set opens six monitoring
sockets. If the application uses the default setting for ``maxPoolSize`` and
only queries the primary (default) node, then there can be at most ``106`` open
sockets and ``100`` connections in the connection pool. If the application uses
a :ref:`read preference <golang-read-pref>` to query the secondary nodes, their
pools also grow and there can be ``306`` total connections.

For efficiency, create a client once for each process, and reuse it for all
operations. Avoid creating a new client for each request because this will
increase latency.

Configure a Connection Pool
---------------------------

You can specify settings for your connection pool either by using a connection
string or by using the ``options.Client`` methods.

Select the :guilabel:`Connection String` or :guilabel:`MongoClientSettings` tab to
see the corresponding syntax:

.. tabs::

.. tab:: Connection String
:tabid: uri

The following are connection string settings you can use to configure your
connection pool:

.. list-table::
:widths: 25,75
:header-rows: 1

* - Setting
- Description

* - ``maxPoolSize``

- Maximum number of connections opened in the pool. If an operation needs a
new connection while the connection pool has ``maxPoolSize`` connections
open, the new operation waits for a new connection to open. To limit this
waiting time, use the single timeout setting.

*Default:* ``100``

* - ``minPoolSize``

- Minimum number of connections opened in the pool. The value of
``minPoolSize`` must be less than the value of ``maxPoolSize``.

*Default*: ``0``

* - ``maxConnecting``

- Maximum number of connections a pool may establish concurrently.

*Default:* ``2``

* - ``maxIdleTimeMS``

- The maximum number of milliseconds that a connection can remain idle in
the pool before being removed and closed.

*Default:* ``None`` (no limit)

* - ``waitQueueTimeoutMS```

- Specifies the maximum wait time in milliseconds that a thread can wait
for a connection to become available.

*Default*: ``0`` (no limit)

.. tab:: ClientOptions
:tabid: ClientOptions

The following table describes the methods you can chain to your settings
to modify the driver's behavior:

.. list-table::
:widths: 25,75
:header-rows: 1

* - Setting
- Description

* - ``SetMaxPoolSize()``

- Maximum number of connections opened in the pool. If an operation needs a
new connection while the connection pool has the configured maximum connections
open, the new operation waits for a new connection to open. To limit this
waiting time, use the single timeout setting.

*Default:* ``100``

* - ``SetMinPoolSize()``

- Minimum number of connections opened in the pool. The value of
``MinPoolSize`` must be less than the value of ``MaxPoolSize``.

*Default*: ``0``

* - ``SetMaxConnecting()``

- Maximum number of connections a pool may establish concurrently.

*Default:* ``2``

* - ``SetMaxConnIdleTime()``

- The maximum number of milliseconds that a connection can remain idle in
the pool before being removed and closed.

*Default:* ``0`` (no limit)

Example
~~~~~~~

Select the :guilabel:`Connection String` or :guilabel:`MongoClientSettings` tab to
see the corresponding example:

.. tabs::

.. tab:: Connection String
:tabid: uriExample

The following code uses the connection string to configure the maximum
connection pool size of ``50``, a minimum pool size of ``10``, and a
maximum idle time of ``30000`` milliseconds (30 seconds):

.. literalinclude:: /includes/connect/connection-pools-uri.go
:language: go
:start-after: start-uri-variable
:end-before: end-uri-variable
:dedent:

The following code creates a client and passes the connection string to the
``ApplyURI()`` method:

.. literalinclude:: /includes/connect/connection-pools-uri.go
:language: go
:start-after: start-apply-uri
:end-before: end-apply-uri
:dedent:

.. tab:: ClientOptions
:tabid: optionsExample

The following code creates a client and sets the connection pool options
with a maximum connection pool size of ``50``, a minimum pool size of
``10``, and a maximum idle time of ``30000`` milliseconds (30 seconds):

.. literalinclude:: /includes/connect/connection-pools-client-options.go
:language: go
:start-after: start-client-options
:end-before: end-client-options
:dedent:

Optimize Connection Pools
-------------------------

Connection pools are rate-limited such that each connection pool can only create,
at maximum, the value of ``maxConnecting`` connections in parallel at any time.
Any new goroutine stops waiting in the following cases:

- One of the existing goroutines finishes creating a connection, or an existing
connection is checked back into the pool.
- The driver's ability to reuse existing connections improves due to rate-limits
on connection creation.

The driver does not limit the number of operations that can wait for sockets to
become available, so it is the application's responsibility to manage its
operation queue. Operations can wait for any length of time unless you define
the ``waitQueueTimeoutMS`` option.

An operation that waits more than the length of time defined by
``waitQueueTimeoutMS`` for a socket raises a connection error. Use this option
if it is more important to bound the duration of operations during a load spike
than it is to complete every operation.

Disconnecting
-------------

When you call ``Client.Disconnect()`` from any goroutine, the driver closes all
idle sockets, and then closes all sockets as they are returned to the pool.

Additional Information
----------------------

For more information on using a connection pool, see the :manual:`Connection
Pool </administration/connection-pool-overview>` documentation in the Server
manual.

API Documentation
~~~~~~~~~~~~~~~~~

To learn more about any of the methods or types discussed in this
guide, see the following API documentation:

- `Client <{+api+}/mongo#Client>`__
- `ClientOptions <{+api+}/mongo/options#ClientOptions>`__
- `Client.Diconnect() <{+api+}/mongo#Client.Disconnect>`__
39 changes: 39 additions & 0 deletions source/includes/connect/connection-pools-client-options.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package main

import (
"context"
"log"
"os"
"time"

"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)

func main() {
uri := os.Getenv("MONGODB_URI")
if uri == "" {
log.Fatal("Set your 'MONGODB_URI' environment variable.")
}
// start-client-options
// Sets client options with connection pool settings
clientOptions := options.Client().
ApplyURI(uri).
SetMaxPoolSize(50).
SetMinPoolSize(10).
SetMaxConnIdleTime(30 * time.Second)

// Creates a new client and connects to the server
client, err := mongo.Connect(clientOptions)
if err != nil {
log.Fatal(err)
}
// end-client-options

// Ensures the client is disconnected when the function exits
defer func() {
if err = client.Disconnect(context.TODO()); err != nil {
log.Fatal(err)
}
}()
}
33 changes: 33 additions & 0 deletions source/includes/connect/connection-pools-uri.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package main

import (
"context"
"log"

"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)

// start-uri-variable
// Connection string with connection pool options
const (
uri = "mongodb://localhost:27017/?maxPoolSize=50&minPoolSize=10&maxIdleTimeMS=30000"
)

// end-uri-variable
func main() {
// start-apply-uri
// Creates a new client and connect to the server
client, err := mongo.Connect(options.Client().ApplyURI(uri))
if err != nil {
log.Fatal(err)
}
// end-apply-uri

fmt.log("Connected to MongoDB with connection pool options")
defer func() {
if err = client.Disconnect(context.TODO()); err != nil {
log.Fatal(err)
}
}()
}
Loading