[iceberg] When using hadoop catalog and query tables created by Spark the column name is incorrect

[PingLiuPing]

Your Environment

• Presto version used:
• Storage (HDFS/S3/GCS..):
• Data source and connector used:
• Deployment (Cloud or On-prem):
• Pastebin link to the complete debug logs:

Use Spark to create a iceberg table as following:

spark-sql (default)>
> show create table iceberg.emp_mix;
CREATE TABLE iceberg.emp_mix (
emp_name STRING,
emp_LOCATION STRING,
EMP_ID INT,
dep_id STRING,
u_ID INT)
USING iceberg
LOCATION 'file:///work/opensource/presto/presto-native-execution/data/iceberg_data/HADOOP/iceberg/emp_mix'
TBLPROPERTIES (
'current-snapshot-id' = '6820088206506591358',
'format' = 'iceberg/parquet',
'format-version' = '2',
'write.parquet.compression-codec' = 'zstd')

Note that the column names are mixed case.

Next using Prestissimo to query the table:

presto:iceberg> show create table emp_mix;
Create Table

CREATE TABLE iceberg.iceberg.emp_mix (
"emp_name" varchar,
"emp_location" varchar,
"emp_id" integer,
"dep_id" varchar,
"u_id" integer
)
WITH (
"format-version" = '2',
location = 'file:///Users/pingliu/work/opensource/presto/presto-native-execution/data/iceberg_data/HADOOP/iceberg/emp_mix',
"read.split.target-size" = 134217728,
"write.delete.mode" = 'copy-on-write',
"write.format.default" = 'PARQUET',
"write.metadata.delete-after-commit.enabled" = false,
"write.metadata.metrics.max-inferred-column-defaults" = 100,
"write.metadata.previous-versions-max" = 100,
"write.update.mode" = 'copy-on-write'
)
(1 row)

The column names are all lower case.
The iceberg table metadata file indicates the column names are mixed case.

And also from the parquet data file metadata, the column names are mixed case too.

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce

Screenshots (if appropriate)

Context

[imjalpreet]

@agrawalreetika, this is something to look into from a mixed case support perspective, as the dependency on Hive metastore is minimal or absent for the Iceberg connector. Maybe we could support it.

@PingLiuPing, in Spark SQL, if we select emp_location rather than emp_LOCATION, does Spark throw an error that the column does not exist?

[PingLiuPing]

@imjalpreet The column names have been converted to lower case in Presto and I think this is unexpected. I'm not sure where this is done.

in Spark SQL, if we select emp_location rather than emp_LOCATION, does Spark throw an error that the column does not exist?

No, when query lower case emp_location spark returns correct result.

And Presto also returns correct result (both emp_location, emp_LOCATION).