Skip to content

Conversation

geruh
Copy link
Contributor

@geruh geruh commented Feb 24, 2024

This PR Fixes #455 by adding ability to use custom headers in PyIceberg's RESTCatalog, making it consistent with the Java implementation of the RESTCatalog. In the Java implementation, configs have the flexibility to be specified at two points: initially, through catalog properties provided at creation, and dynamically, based on the properties included in the server's response to the getConfig endpoint call. Which ensures that all the subsequent requests are made with these headers. One thing to call out is headers are also case insensitive, and this change respects that.

Catalog Properties

catalog = RestCatalog("rest", uri=TEST_URI, warehouse="s3://some-bucket", **{"header.Content-Type": "application/vnd.api+json"})

getConfig Response

{
  "defaults": {
    "header.cache-control": "no-cache"
  },
  "overrides": {}
}

Header logic in Java RESTCatalog: https://github.com/apache/iceberg/blob/4090a8860061f58748e3faa6804094f90d3575f3/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L165

@Fokko @HonahX

Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, can you add this to the docs as well? :)

Copy link
Contributor

@HonahX HonahX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@HonahX
Copy link
Contributor

HonahX commented Feb 27, 2024

Merging. Thanks @geruh for working on this and @Fokko for reviewing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for custom header configs in RESTCatalog
3 participants