Should changes to field order be considered breaking changes?

[edgarrmondragon]

Recently a question came up in Slack regarding target-s3-csv causing problems with downstream COPY/import operations due to the column order changing from one invocation to another.

For better or worse, some targets force alpha sorting of the fields (e.g. target-snowflake).

Some users may prefer that targets respect the order communicated by tap via the SCHEMA messages, but that requires taps to emit a consistent order and that they try to preserve the order between revisions/versions.

The SDK could help with SCHEMA messages emitted, but also perhaps with best practices/recommendations around handling schema updates.

[tayloramurphy]

@edgarrmondragon can you update with a link to the slack convo?

And when you say breaking change, are you talking about on the version on the connector or something else?

[edgarrmondragon]

can you update with a link to the slack convo?

Good catch, done!

And when you say breaking change, are you talking about on the version on the connector or something else?

Yeah, I'm talking about the version of the connector. Changing the tap's field order may break the target if it relies on a stable order.

[tayloramurphy]

So my gut on this is that having it as a major version change is probably too much. But a minor update makes sense to me. We should have some better features around handling schema changes and what action to take when a change is detected, but I don't think we need a major increment on a tap/target if something changes. @pnadolny13 do you agree?