Airtable Connector Discovery Fails with AirbyteTracedException

[matthew-petty]

Description

Encountered an error when attempting to create a new Airtable connector. The error was reproducible on subsequent attempts.

Environment

• Account: New account
• OS: macOS
• Browser: Firefox
• Auth Method: api_key (Personal Access Token)

Error Message

Generating Specification Failed

There was an issue attempting to discover your endpoint.

flow://capture/Pattern_Zones_Co/airtable/source-airtable
Traceback (most recent call last):
  File "/opt/estuary-cdk/estuary_cdk/capture/base_capture_connector.py", line 83, in handle
    self._emit(Response(discovered=await self.discover(log, discover)))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/estuary-cdk/estuary_cdk/shim_airbyte_cdk.py", line 264, in discover
    resources = await self._all_resources(log, discover.config)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/estuary-cdk/estuary_cdk/shim_airbyte_cdk.py", line 186, in _all_resources
    catalog = self.delegate.discover(log, config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/source-airtable/source_airtable/source.py", line 77, in discover
    for base in AirtableBases(authenticator=auth).read_records(sync_mode=None):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/airbyte_cdk/sources/streams/http/http.py", line 471, in read_records
    yield from self._read_pages(
  File "/opt/venv/lib/python3.12/site-packages/airbyte_cdk/sources/streams/http/http.py", line 487, in _read_pages
    request, response = self._fetch_next_page(stream_slice, stream_state, next_page_token)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/airbyte_cdk/sources/streams/http/http.py", line 513, in _fetch_next_page
    response = self._send_request(request, request_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/airbyte_cdk/sources/streams/http/http.py", line 411, in _send_request
    return backoff_handler(user_backoff_handler)(request, request_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/backoff/_sync.py", line 105, in retry
    ret = target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/backoff/_sync.py", line 105, in retry
    ret = target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/airbyte_cdk/sources/streams/http/http.py", line 355, in _send
    if self.should_retry(response):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/source-airtable/source_airtable/streams.py", line 45, in should_retry
    raise AirbyteTracedException(message=error_message, failure_type=FailureType.config_error)
airbyte_cdk.utils.traced_exception.AirbyteTracedException: None

Connector Details

• Image: ghcr.io/estuary/source-airtable:v1
• Image Created: 2025-09-30T20:40:20.428358242Z
• Runtime Protocol: capture
• Python Version: 3.12

Configuration

The Personal Access Token was configured via api_key auth method. See screenshot in issue for API key configuration interface.

Steps to Reproduce

1. Create new Airtable connector
2. Configure with api_key auth method
3. Enter Personal Access Token
4. Attempt to generate specification
5. Error occurs during discovery phase

Expected Behavior

Connector should successfully discover the Airtable endpoint and generate specification.

Actual Behavior

Discovery fails with AirbyteTracedException during should_retry check in streams.py line 45.

[dyaffe]

Hey @matthew-petty -- thanks for reporting this. Our current Airtable connector is based on an Airbyte OSS one and we're going to be rewriting it on our CDK in the next month or so. At that time we will greatly improve it by doing things like making it incremental and low-latency. We'll also make sure to address the issue you're seeing here then.

[matthew-petty]

Thanks for sharing a little about the roadmap. I look forward to testing it again.