Skip to content

Conversation

@krivard
Copy link
Contributor

@krivard krivard commented Nov 21, 2025

Overview

Closes #78

What did you change in this PR?

  • Fetch datapackage from viewer bucket (__init__.py)
  • Run preview queries against viewer bucket (index.ts)
  • If the user wants the whole parquet for offline analysis, use the nightly bucket (search_results partial; unchanged in this PR)

Testing

How did you make sure this worked? How can a reviewer verify this?

  • Ran locally
  • Verified server log included {"event": "Getting datapackage from https://s3.us-west-2.amazonaws.com/pudl.catalyst.coop/eel-hole/pudl_parquet_datapackage.json"
  • In a browser, turned on network monitoring dev tools
  • Loaded http://127.0.0.1:8080/search and previewed a table
  • Examined headers for parquet requests (the ones with 206 response status); verified URL started with https://s3.us-west-2.amazonaws.com/pudl.catalyst.coop/eel-hole/

To-do list

  • Review the PR yourself and call out any questions or issues you have

…till use nightlies if the user wants the whole parquet)
@krivard krivard self-assigned this Nov 21, 2025
@cmgosnell cmgosnell moved this from New to In progress in Catalyst Megaproject Nov 24, 2025
@cmgosnell cmgosnell moved this from In progress to In review in Catalyst Megaproject Nov 24, 2025
@e-belfer e-belfer self-requested a review November 24, 2025 20:19
Copy link
Member

@e-belfer e-belfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This worked as expected for me! I was able to reproduce your testing workflow locally. I've updated the uv lockfile to resolve failed builds: https://github.com/catalyst-cooperative/eel-hole/actions/runs/19572591605

and to handle an npm package that was triggering a dependabot warning. After doing this I reran the docker build and all seemed to work as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

Migrate from nightly to new PUDL Viewer bucket for DuckDB queries in S3

4 participants