How to structure multi-location projects for devcontainer development? #32520
Unanswered
philiporlando
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm working on designing a project structure and development workflow for an OSS Dagster deployment using Docker Compose for production. I'm trying to integrate devcontainers for local development but running into some organizational challenges.
Current Approach
I'm organizing my team's work into separate code locations, with each having its own Dockerfile. The challenge is that developing on any single code location still requires Dagster's core services (webserver, daemon, PostgreSQL, etc.).
My current thinking is to:
docker-compose.yamlin each code location directory that spins up only that location plus the necessary Dagster servicesdocker-compose.yamlat the root directory that orchestrates all code locations.However, this approach seems to require maintaining separate development versions of
dagster.yamlandworkspace.yamlfor each code location's devcontainer. As we scale, managing these files (multipledocker-compose.yaml,Dockerfile,dagster.yaml, andworkspace.yamlper location) feels like significant overhead.Questions
I appreciate Dagster's flexibility, but I want to make sure I'm setting up a structure that will scale well. Has anyone tackled similar challenges or have insights to share?
Beta Was this translation helpful? Give feedback.
All reactions