Commit 000c565
authored
Merge pull request #57 from lincc-frameworks/issue/56/orbital-death-ray
* Support for multiple connections via 'concurrent_connections' config
* Replaced FailedChunkCollector and old resume system in downloadCutout.py with manifest-based resume system
* Added hash and eq to downloadCutout Rects so they can be dictionary keys
* Pulled downloader functions into a class so class constants could be namespaced
* Removed pwd contextmanager in favor of absolute path filename representation in Rects.
pwd context manager was not compatible with threading.Thread
* Stats system now outputs every 30s (controllable with stats_print_interval config)
* Stats system creates sensible metrics for a multithreaded system including how
efficiently its multiple connections are being used (this allows us to tell on average how many connections are live with the server at once)
* Several termination issues have been fixed by making all non-main threads
daemon threads. The result is that everything shuts down dirty on
KeyboardInterrupt. This is okay because the only thing the main thread does
on shutdown is read append-only manifest dicts, so additional download activity
during the shutdown will be presumed to have failed and be overwritten on resume.File tree
4 files changed
+582
-470
lines changed- src/fibad
- downloadCutout
4 files changed
+582
-470
lines changed
0 commit comments