Skip to content

Support for pymongo 3 #214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Jul 12, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
568dd07
differentiate updates from inserts to allow for updates to use the s…
Jun 9, 2015
e257266
Merge branch 'feature/atomic-updates'
Jun 9, 2015
075608d
added folders to gitignore
Jun 9, 2015
f3a2215
fixed pymongo version to be > 3.0
Jun 16, 2015
9fa43d4
removed deprecated MongoReplicaSetClient and all now unused connectio…
Jun 16, 2015
681b712
removed the use of the fields parameter which has now been changed to…
Jun 16, 2015
292e74c
removed use of the safe option which is now no longer used. All conne…
Jun 16, 2015
5b4e11d
added fixture teardown override to wipe out all collections. The djan…
Jun 17, 2015
69e028f
changed the flush command to not try and delete/remove records from c…
Jun 17, 2015
b02729c
removed the fixture teardown override
Jun 17, 2015
236b152
clear out any existing objects in the setup
Jun 17, 2015
8bc14d4
removed use of slave_okay option when creating connection
Jun 17, 2015
495b82d
removed use of slave_okay
Jun 17, 2015
1573e06
added some clean up in the setup to remove any old docs between tests
Jun 17, 2015
57cc51d
removed unique constraint test as this will now fail due to safe read…
Jun 17, 2015
2f259b4
Merge branch 'safe-writes'
Jun 17, 2015
30a6acf
Merge branch 'master' into feature/pymongo_3
Jun 23, 2015
801d7bf
updated link to new github user (django-nonrel)
Jun 23, 2015
5edbaef
merged changes from ajdavis:pymongo-3 branch
Jun 23, 2015
c0dcc66
removed distinguishes_insert_from_update flag in DatabaseFeatures (wr…
Jun 23, 2015
fef0185
reverted change to use tarballs instead of cloning from git
Jun 26, 2015
c1bb7dc
Merge remote-tracking branch 'upstream/master' into feature/pymongo_3
Jun 26, 2015
af3d08b
removed 1.7 from test options
Jun 26, 2015
7395f5f
added 1.7 back in, but allow to fail for now
Jun 26, 2015
122e62b
added can_rollback_ddl=True in DatabaseFeatures to fix issues with at…
Jun 26, 2015
e6d2b38
support pymongo 2.x
Jun 26, 2015
7faa4a4
added environment variables for testing against pymongo 2.8.1 and and…
Jun 26, 2015
605dfcb
cleanup based on @aburgel comments
Jun 29, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,7 @@ dist/
.tox

MANIFEST


.env
.idea
18 changes: 14 additions & 4 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,28 @@ python:
- 2.7

env:
- DJANGO_VERSION=1.4
- DJANGO_VERSION=1.5
- DJANGO_VERSION=1.6
- DJANGO_VERSION=1.4 PYMONGO_VERSION=2.8.1
- DJANGO_VERSION=1.4 PYMONGO_VERSION=3.0.2
- DJANGO_VERSION=1.5 PYMONGO_VERSION=2.8.1
- DJANGO_VERSION=1.5 PYMONGO_VERSION=3.0.2
- DJANGO_VERSION=1.6 PYMONGO_VERSION=2.8.1
- DJANGO_VERSION=1.6 PYMONGO_VERSION=3.0.2
- DJANGO_VERSION=1.7 PYMONGO_VERSION=2.8.1
- DJANGO_VERSION=1.7 PYMONGO_VERSION=3.0.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does django 1.7 actually work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet (am planning to look at that next). I've added it to the allowable failures for now



matrix:
allow_failures:
- env: DJANGO_VERSION=1.4
- env: DJANGO_VERSION=1.4 PYMONGO_VERSION=2.8.1
- env: DJANGO_VERSION=1.4 PYMONGO_VERSION=3.0.2
- env: DJANGO_VERSION=1.7 PYMONGO_VERSION=2.8.1
- env: DJANGO_VERSION=1.7 PYMONGO_VERSION=3.0.2

install:
- pip install git+http://github.com/django-nonrel/django@nonrel-$DJANGO_VERSION
- pip install git+http://github.com/django-nonrel/django-dbindexer@master
- pip install git+http://github.com/django-nonrel/djangotoolbox@master
- pip install pymongo==$PYMONGO_VERSION --upgrade
- python setup.py install

script: cd tests && python runtests.py
2 changes: 1 addition & 1 deletion AUTHORS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ Contributions by
* Brandon Pedersen (https://github.com/bpedman)

(For an up-to-date list of contributors, see
https://github.com/django-mongodb-engine/mongodb-engine/contributors.)
https://github.com/django-nonrel/mongodb-engine/contributors.)
15 changes: 10 additions & 5 deletions django_mongodb_engine/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ class DatabaseFeatures(NonrelDatabaseFeatures):
supports_microsecond_precision = False
supports_long_model_names = False

can_rollback_ddl = True


class DatabaseOperations(NonrelDatabaseOperations):
compiler_module = __name__.rsplit('.', 1)[0] + '.compiler'
Expand All @@ -60,6 +62,9 @@ def sql_flush(self, style, tables, sequence_list, allow_cascade=False):
drop all `tables`. No SQL in MongoDB, so just clear all tables
here and return an empty list.
"""



for table in tables:
if table.startswith('system.'):
# Do not try to drop system collections.
Expand Down Expand Up @@ -246,11 +251,6 @@ def pop(name, default=None):
warnings.warn("slave_okay has been deprecated. "
"Please use read_preference instead.")

if replicaset:
connection_class = MongoReplicaSetClient
else:
connection_class = MongoClient

conn_options = dict(
host=host,
port=int(port),
Expand All @@ -259,6 +259,11 @@ def pop(name, default=None):
)
conn_options.update(options)

if replicaset:
connection_class = MongoReplicaSetClient
else:
connection_class = MongoClient

try:
self.connection = connection_class(**conn_options)
self.database = self.connection[db_name]
Expand Down
2 changes: 1 addition & 1 deletion django_mongodb_engine/compiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ def get_cursor(self):
return []

fields = get_selected_fields(self.query)
cursor = self.collection.find(self.mongo_query, fields=fields)
cursor = self.collection.find(self.mongo_query, fields)
if self.ordering:
cursor.sort(self.ordering)
if self.query.low_mark > 0:
Expand Down
2 changes: 0 additions & 2 deletions django_mongodb_engine/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,6 @@ def log(self, op, duration, args, kwargs=None):
logger.debug(msg, extra={'duration': duration})

def find(self, *args, **kwargs):
if not 'slave_okay' in kwargs and self.collection.slave_okay:
kwargs['slave_okay'] = True
return DebugCursor(self, self.collection, *args, **kwargs)

def logging_wrapper(method):
Expand Down
36 changes: 15 additions & 21 deletions docs/source/reference/settings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ Settings

.. TODO fix highlighting

Connection Settings
Client Settings
-------------------
Additional flags may be passed to :class:`pymongo.Connection` using the
Additional flags may be passed to :class:`pymongo.MongoClient` using the
``OPTIONS`` dictionary::

DATABASES = {
Expand All @@ -14,27 +14,25 @@ Additional flags may be passed to :class:`pymongo.Connection` using the
'NAME' : 'my_database',
...
'OPTIONS' : {
'slave_okay' : True,
'tz_aware' : True,
'network_timeout' : 42,
'socketTimeoutMS' : 500,
...
}
}
}

All of these settings directly mirror PyMongo settings. In fact, all Django
MongoDB Engine does is lower-casing the names before passing the flags to
:class:`~pymongo.Connection`. For a list of possible options head over to the
`PyMongo documentation on connection options`_.
:class:`~pymongo.MongoClient`. For a list of possible options head over to the
`PyMongo documentation on client options`_.

.. _operations-setting:

Safe Operations (``getLastError``)
----------------------------------
Acknowledged Operations
-----------------------
Use the ``OPERATIONS`` dict to specify extra flags passed to
:meth:`Collection.save <pymongo.collection.Collection.save>`,
:meth:`~pymongo.collection.Collection.update` or
:meth:`~pymongo.collection.Collection.remove` (and thus to ``getLastError``):
:meth:`~pymongo.collection.Collection.remove` (and thus included in the write concern):

.. code-block:: python

Expand All @@ -43,21 +41,17 @@ Use the ``OPERATIONS`` dict to specify extra flags passed to
...
}

Since any options to ``getLastError`` imply ``safe=True``,
this configuration passes ``safe=True, w=3`` as keyword arguments to each of
:meth:`~pymongo.collection.Collection.save`,
:meth:`~pymongo.collection.Collection.update` and
:meth:`~pymongo.collection.Collection.remove`.


Get a more fine-grained setup by introducing another layer to this dict:

.. code-block:: python

'OPTIONS' : {
'OPERATIONS' : {
'save' : {'safe' : True},
'save' : {'w' : 3},
'update' : {},
'delete' : {'fsync' : True}
'delete' : {'j' : True}
},
...
}
Expand All @@ -69,10 +63,10 @@ Get a more fine-grained setup by introducing another layer to this dict:
"`insert vs. update`" into `save`.


A full list of ``getLastError`` flags may be found in the
`MongoDB documentation <http://www.mongodb.org/display/DOCS/getLastError+Command>`_.
A full list of write concern flags may be found in the
`MongoDB documentation <http://docs.mongodb.org/manual/core/write-concern/>`_.

.. _Similar to Django's built-in backends:
http://docs.djangoproject.com/en/dev/ref/settings/#std:setting-OPTIONS
.. _PyMongo documentation on connection options:
http://api.mongodb.org/python/current/api/pymongo/connection.html
.. _PyMongo documentation on client options:
http://api.mongodb.org/python/current/api/pymongo/mongo_client.html
4 changes: 3 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
pymongo<3.0

pymongo>=2.8

https://github.com/django-nonrel/djangotoolbox/tarball/master
https://github.com/django-nonrel/django/tarball/nonrel-1.5
https://github.com/django-nonrel/django-dbindexer/tarball/master
4 changes: 3 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@
license='2-clause BSD',
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
install_requires=['pymongo<3.0', 'djangotoolbox>=1.6.0'],

install_requires=['pymongo>=2.8', 'djangotoolbox>=1.6.0'],

packages=find_packages(exclude=['tests', 'tests.*']),
zip_safe=False,
classifiers=[
Expand Down
5 changes: 4 additions & 1 deletion tests/contrib/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,6 @@ def test_map_reduce_with_custom_primary_key(self, inline=False):
class RawQueryTests(TestCase):

def setUp(self):
MapReduceModel.objects.all().delete()

for i in xrange(10):
MapReduceModel.objects.create(n=i, m=i * 2)
Expand Down Expand Up @@ -158,6 +157,10 @@ def test_raw_update(self):
# TODO: Line breaks.
class FullTextTest(TestCase):

def setUp(self):
Post.objects.all().delete()


def test_simple_fulltext(self):
blog = Post(content="simple, full text.... search? test")
blog.save()
Expand Down
5 changes: 0 additions & 5 deletions tests/lookup/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from operator import attrgetter

from django.core.exceptions import FieldError
from django.db import connection
from django.db.utils import DatabaseError
from django.test import TestCase, skipUnlessDBFeature

Expand All @@ -18,10 +17,6 @@
class LookupTests(TestCase):

def setUp(self):
# Create a few Authors.
Author.objects.all().delete()
Article.objects.all().delete()
Tag.objects.all().delete()

self.au1 = Author(name='Author 1')
self.au1.save()
Expand Down
Loading