encodingFormat should be checked by CLI before uploading

[satra]

in dandiset 108 a TSV file metadata did not have encoding format set. encoding formats should be set by cli before uploading. in this dandiset we are saying use any path, but i think the objects should still be recognizable from a format perspective.

[jwodder]

When was the file uploaded? encodingFormat only started to be set for all files in commit 0322f50.

[satra]

looks like yesterday. https://api.dandiarchive.org/api/dandisets/000108/versions/draft/assets/e84b5900-bef1-494a-9399-7eaf537390bf/

i'm pinging @dchiquito as well on this thread.

i suspect someone used an older dandi-cli to do this. because the schemaVersion is 0.3.0. we should also set a minimum version of the schema on the server as well and reject posts/puts that have missing schemaVersion or use a schema lower than the minimum. this also means that the cli did not error locally for this person.

[yarikoptic]

yeap, we do always provide encodingFormat as of 0.21.0-1-g0322f50 (although no validation is done - just that code does provide it), so nothing to fix for this particular issue probably. But IMHO #722 should be addressed to prevent this and similar cases where required by API server metadata is not provided in the future.

[satra]

we have a mechanism to prevent earlier clients through ET, but it seems the ET bad versions is not being used to prevent this.

[satra]

do we know if ET bad version exception is overridden in the CLI? the version that was used is listed as being bad.

[yarikoptic]

yeah, I think we just issue a warning on any exception from etelemetry: https://github.com/dandi/dandi-cli/blob/master/dandi/cli/command.py#L132

[satra]

so we aren't at all protecting against bad versions then? also that line isn't asking etelemetry to check for bad versions: https://github.com/sensein/etelemetry-client/blob/fc36b7202494c6733526bf582f9516d2c8e17e1c/etelemetry/client.py#L72

i think the exception is mostly to track if for some reason etelemetry could not be imported. perhaps this should be addressed properly.

[yarikoptic]

ref: #725 on PR to error out on bad versions, and #726 for making ET check used also when used as a Python library