DumpError when running test with -n flag

[sigiesec]

When I try to run my tests with xdist and the -n flag, I get a DumpError:

13:33:10 ============================= test session starts =============================
13:33:10 platform win32 -- Python 3.5.4, pytest-3.6.2, py-1.5.3, pluggy-0.6.0 -- D:\slave\venv\0\conan\Scripts\python.exe
13:33:10 cachedir: .pytest_cache
13:33:10 rootdir: D:\slave\bfsvidl0, inifile:
13:33:10 plugins: xdist-1.22.2, forked-0.2
13:33:10 gw0 I / gw1 I / gw2 I / gw3 I
13:33:11 
[gw0] win32 Python 3.5.4 cwd: D:\slave\bfsvidl0
13:33:11 INTERNALERROR> Traceback (most recent call last):
13:33:11 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1383, in _save
13:33:11 INTERNALERROR>     dispatch = self._dispatch[tp]
13:33:11 INTERNALERROR> KeyError: <enum 'TargetVersions'>
13:33:11 INTERNALERROR> 
13:33:11 INTERNALERROR> During handling of the above exception, another exception occurred:
13:33:11 INTERNALERROR> 
13:33:11 INTERNALERROR> Traceback (most recent call last):
13:33:12 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\_pytest\main.py", line 176, in wrap_session
13:33:12 INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
13:33:12 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\pluggy\__init__.py", line 617, in __call__
13:33:12 INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
13:33:12 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\pluggy\__init__.py", line 222, in _hookexec
13:33:12 INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
13:33:12 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\pluggy\__init__.py", line 216, in <lambda>
13:33:12 INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
13:33:13 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\pluggy\callers.py", line 201, in _multicall
13:33:13 INTERNALERROR>     return outcome.get_result()
13:33:13 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\pluggy\callers.py", line 76, in get_result
13:33:13 INTERNALERROR>     raise ex[1].with_traceback(ex[2])
13:33:13 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\pluggy\callers.py", line 180, in _multicall
13:33:13 INTERNALERROR>     res = hook_impl.function(*args)
13:33:13 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\xdist\dsession.py", line 81, in pytest_sessionstart
13:33:13 INTERNALERROR>     nodes = self.nodemanager.setup_nodes(putevent=self.queue.put)
13:33:13 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\xdist\workermanage.py", line 64, in setup_nodes
13:33:13 INTERNALERROR>     nodes.append(self.setup_node(spec, putevent))
13:33:14 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\xdist\workermanage.py", line 73, in setup_node
13:33:14 INTERNALERROR>     node.setup()
13:33:14 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\xdist\workermanage.py", line 246, in setup
13:33:14 INTERNALERROR>     self.channel.send((self.workerinput, args, option_dict))
13:33:14 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 717, in send
13:33:14 INTERNALERROR>     self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
13:33:14 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1354, in dumps_internal
13:33:14 INTERNALERROR>     return _Serializer().save(obj)
13:33:14 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1372, in save
13:33:14 INTERNALERROR>     self._save(obj)
13:33:15 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1390, in _save
13:33:15 INTERNALERROR>     dispatch(self, obj)
13:33:15 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1475, in save_tuple
13:33:15 INTERNALERROR>     self._save(item)
13:33:15 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1390, in _save
13:33:15 INTERNALERROR>     dispatch(self, obj)
13:33:15 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1471, in save_dict
13:33:15 INTERNALERROR>     self._write_setitem(key, value)
13:33:15 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1465, in _write_setitem
13:33:15 INTERNALERROR>     self._save(value)
13:33:15 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1390, in _save
13:33:16 INTERNALERROR>     dispatch(self, obj)
13:33:16 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1461, in save_list
13:33:16 INTERNALERROR>     self._write_setitem(i, item)
13:33:16 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1465, in _write_setitem
13:33:16 INTERNALERROR>     self._save(value)
13:33:16 INTERNALERROR>   File "D:\slave\venv\0\conan\lib\site-packages\execnet\gateway_base.py", line 1388, in _save
13:33:16 INTERNALERROR>     raise DumpError("can't serialize %s" % (tp,))
13:33:16 INTERNALERROR> execnet.gateway_base.DumpError: can't serialize <enum 'TargetVersions'>

I am using a custom conftest.py, which includes another module that defines the mentioned enum. Do I need to register it somehow? If I understand it correctly, execnet uses pickle to serialize the objects, and an enum should be serializable out of the box.

[RonnyPfannschmidt]

execnet uses a custom serializer and enum is not supported

i plan to work on a non-execnet setup

[sigiesec]

Ok, too sad it can't be fixed easily.

I just read on the execnet mailing list you are planning to migrate xdist to mitogen.

[RonnyPfannschmidt]

i dont plan a migration - i plan to open up more ways to parallelism and run on other systems

one of those will be multiprocessing #302

[youtux]

This happened to me when trying to upgrade from pytest_xdist 1.22.0 to 1.23.2 (I'm using pytest 3.9.2):

INTERNALERROR> DumpError: can't serialize <class 'sqlalchemy.util.langhelpers._hash_limit_string'>