[RFC 0002] MVP

rfcs/0002-mvp.md

@@ -0,0 +1,160 @@
+---
+feature: mvp
+start-date: 2020-01-28
+author: kiwicopple
+co-authors: steve-chavez
+related-issues: (will contain links to implementation PRs)
+---
+
+# Summary
+[summary]: #summary
+
+We would like to create a basic CLI for Supabase. At this stage we can assume that the user won't need to create orgs/projects from the CLI, but they will need to:
+
+- develop locally
+- push database migrations to their Postgres database
+
+# Motivation
+[motivation]: #motivation
+
+- Make it easy to run Supabase locally
+- Make it easy to manage database migrations
+- Make it easy to do "database branching", where each git branch can have it's own data to prevent conflicts. This is especially important for working with platforms like Vercel/Netlify
+
+
+# Detailed design
+[design]: #detailed-design
+
+
+### Initializing
+
+```sh
+supa init     # creates a .supabase folder with relevant config, connection strings etc. Also scaffolds out a database folder.
+supa start    # Starts Supabase (by running docker-compose), and pulls up the database
+supa eject    # See the .supabase/docker-compose.yml details below
+```
+This will likely need: 
+
+- `supabase.js` - a config file with the Supabase Project details
+- `.supabase/config.txt` - CLI config and details
+- `.supabase/docker-compose.yml` - for running Supabase locally. We may not even need this, as we can just run it directly from the `npm_modules` folder, which would prevent conflicts when we add upgrades or make changes. If we do generate a file, this should be stored in the `.supabase` folder so that it doesn't conflict with the user's docker-compose.yml file. When the user runs `supa eject`, it can move the file out to their root folder (similar to a `cra eject` model), and then the user can manage this themselves.
+
+
+For the database scaffold, [this](https://gitlab.com/delibrium/delibrium-postgrest/-/tree/master/database) format is very nice, although I think we can make it a bit clearer. Here is a suggested format:
+
+```sh
+├── _init.sql
+├── types.sql # Types: custom database types all in a single file
+│   # Data: database fixed/seed data.
+├── data
+│   ├── some_data.{sql|csv|json}        
+│   # Models: Tables and Views
+├── models
+│   ├── public.table                    # Personal preference to explicitly add the schema, and we should prefer dot notation over nesting.
+│   │   ├── data.{sql|csv|json}         # For convenience, table specific data can be stored here too.   
+│   │   ├── model.sql                   # This is the core. We should also figure out a nice, readable format for SQL files. Should include constraints and indexes and table/column comments.
+│   │   ├── permissions.sql             # Grants and Policies.
+│   │   ├── test.sql                    # PGTAP tests. For MVP: generate some simple pgtap has_column tests by default. https://pgtap.org/documentation.html#has_column
+│   │   └── triggers.sql                # table triggers 
+│   # Functions/RPC: SQL only for MVP
+├── functions                                
+│   ├── public.func1.sql                
+│   ├── public.func2.sql│   
+│   # Postgres roles and grants.
+└── roles                                
+    ├── role_name.sql    
+    └── role_name.sql
+```
+
+
+### Branching
+
+This is going to be a key part of Supabase development. Since so many of our users are on Vercel/Netlify, we should have the ability to support [branch/preview-development](https://vercel.com/docs/platform/deployments).
+
+We can do this by connecting to a different database, an idea that I got from [pgsh](https://github.com/sastraxi/pgsh). For example:
+
+- `postgresql://postgres@localhost:5432/master`
+- `postgresql://postgres@localhost:5432/develop`
+- `postgresql://postgres@localhost:5432/chore_add_types`
+
+This is the same from CI (like github actions), except that instead of running against a local database, it is running against the supabase database. We will need a way inside our config to match branch names to database names too. For example 
+
+
+### Dump
+
+```sh
+supa db dump                # dumps the currently active (local) database to disk
+supa db restore             # restores the (local) database from disk
+supa migrate                # generate the migrations file
+supa up                     # runs the "up" migrations on Prod
+supa down                   # runs the "down" migrations on Prod
+```
+
+The `supa migrate` task is the trickiest part here. See [#migration-questions](#migration-questions) below.
+
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+- Supabase is about "time to value", and making things as simple/easy as possible. A CLI can be complicated, so we should make sure that the experience is as amazing as our hosted platform
+
+# Alternatives
+[alternatives]: #alternatives
+
+The hosted platform provides much of this functionality. We could continue to build this functionality into the platform however it will be a great dev experience if they can develop locally and push their changes later.
+
+# Unresolved questions
+[unresolved]: #unresolved-questions
+
+
+### Migration questions
+[migration-questions]: #migration-questions
+
+The `supa migrate` task is the trickiest part. How do we determine what migrations should run on our database, since the database is created declaratively? We would need to compare the databases somehow, and figure out the `up/down` migrations. Alternatively we could do "stateless" migrations, where we just run `restore` on a brand new database, then re-route traffic, but then how do we get the user's prod data into the new database. Bear in mind that enterprise users might have terrabytes of data, and so copying it across will be slow. 
+
+
+### Other questions
+- what is the prefix? should it be `supabase` (clearer) or `supa` (might have a few people calling it "supper/suppa")
+- Which CLI tool/framework/libraries should we use?
+- `supa start` requires docker-compose. Is it reasonable to expect that everyone will have docker set up on their computer? Can we make this easier? 
+- How much functionality should we push to postgres-meta? 
+
+
+
+# Future work
+[future]: #future-work
+
+
+- Functions: Add support for `plv8` and other languages
+
+### Supabase platform 
+
+```sh
+supa deploy   # Deploy a local app to prod. On the first time this is run, it should generate a `./supabase.js` with relevant Supabase config.
+supa orgs     # Manage orgs
+supa project  # Manage projects
+supa logs     # Fetch logs
+supa scale    # Scale the current database
+...etc
+```
+
+
+### Generators (bonus)
+
+Since I have been using typescript in one of our repos, I have discovered a lot of limitations around our [Typescript Generators](https://supabase.io/docs/client/generating-types). This is because Open API spec doesn't have all the tables (if they aren't exposed to `postgres` role), and sometimes it expects some inputs where the database already has defaults (for example our OpenAPI for `created_at` expects a string, when it is actually auto-generated).
+
+I propose here a few ideas for generators. While they are exposed through the CLI, I think we should actually build these into [pg-api](https://github.com/supabase/pg-api), so that anyone can use them without the supabase CLI. (I also think this should be the case with the "dump" functionality).
+
+```sh
+supa gen json api # Dumps the JSON Schema for the API
+supa gen json db # generate AJV schemas for input validation
+
+supa gen typescript api  # generate typescript interfaces for api
+supa gen typescript db # generate typescript interfaces for database. Note that these types should have an interface for each table with "insert types" and "select types". These are actually different because the database can generate default values, so some fields aren't "required" 
+
+
+# Models: 
+# This is my suggestion for a "local first" database which has all of the rules/relationships we need to build a a realtime user store with automatic synchronization via our Realtime engine.
+# A bit like https://github.com/mobxjs/mst-gql (but hopefully much simpler)
+supa gen typescript store 
+```