-
Notifications
You must be signed in to change notification settings - Fork 14
Conversation
What happens if the schema is explicitly not null, e.g.:
and then import your test CSV file. |
I'm not sure we really check for NULL values when importing into a non-nullable column. Worth testing. |
0280fb3
to
5c49e20
Compare
4da8746
to
6d89706
Compare
Added check, that should cover this case, also test "ImportCsv_KnownSchema_NullsColumn_NullableTypeHeader" describes similar scenario. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall the patch looks good. I have several minor comments about unrelated changes, styling, etc.
b835219
to
3099194
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR looks good to go! You might want to rebase it first, CI should be green if rebased on the current main branch.
Hm, looks like there was Null on non-nullable type.
|
That's probably a sign to leave the null-check part for another PR. I don't think that in this test we actually have imported NULL value. The test is for an empty string and the reference value has no NULLs. Also, changing the type to allow NULLS in the test table causes the test to fail. So, in this case, NULL is probably interpreted as an empty string on import/execution/conversion. |
This commit support empty column from csv parsing. Signed-off-by: Dmitrii Makarenko <[email protected]>
4046f8f
to
c280c3a
Compare
This commit adds tests for null columns and inserts verification that non-nullable types will not have nulls. Signed-off-by: Dmitrii Makarenko <[email protected]>
c280c3a
to
585ec8d
Compare
@alexbaden PTAL your notice (verification for nulls depending on schema) not covered fully. String conversion to null looks a bit difficult to support in current PR. Should we crate an issue with this and remove this WA in further PR? Or is current WA fine? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's create an issue and handle later. The empty handling here looks good.
This commit support empty column from csv parsing.
Signed-off-by: Dmitrii Makarenko [email protected]