Multiple Catalog Connection

[asheeshgarg]

Question

From pyiceberg if I want to connect to multiple catalogs we need to create a seperate instance per catalog or there is option to specify multiple catalogs as part of the config?

[Fokko]

Hey @asheeshgarg You can already do this.

The configuration in ~/.pyiceberg.yaml:

catalog:
    hive:
        uri: thrift://127.0.0.1:9083
        s3.endpoint: http://127.0.0.1:9000
        s3.access-key-id: admin
        s3.secret-access-key: password

    rest:
        uri: https://rest-server:8181/
        warehouse: Fokko

In Python you can load the different catalogs like this:

from pyiceberg.catalog import load_catalog

hive = load_catalog('hive')
rest = load_catalog('rest')

Probably we should make this more explicit in the documentation.

[jayceslesar]

Im happy to add this to the docs, wondering the best place for that? With mkdocs it seems to be the best place to add docs would be in docstrings. Thoughts? Ideally there are multiple examples for each type of major class (IE Table, Catalog, io)

[Fokko]

Thanks @jayceslesar for updating the docs here! 👍