diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 788f9d0bb..0f9c8809d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -78,11 +78,13 @@ Ready to code? Check our [open issues](https://github.com/Merit-Systems/echo/iss - **Node.js**: 18.0.0 or higher - **pnpm**: 10.0.0 or higher -- **PostgreSQL**: Required for running Echo Control locally +- **Docker & Docker Compose**: Required for running PostgreSQL locally - **Git**: For version control ### Initial Setup +The fastest way to get started is using the automated setup script: + 1. **Fork and clone the repository** ```bash @@ -90,38 +92,52 @@ Ready to code? Check our [open issues](https://github.com/Merit-Systems/echo/iss cd echo ``` -2. **Install dependencies** +2. **Run the setup script** (recommended) ```bash - pnpm install + bash scripts/setup-local.sh ``` -3. **Set up environment variables** + This script will: + - Check all prerequisites + - Install dependencies + - Set up environment variables + - Start PostgreSQL in Docker + - Run database migrations + - Provide next steps + +3. **Start development servers** ```bash - pnpm local-setup + pnpm dev ``` -4. **Set up the database** (for Echo Control) + This starts both Echo Control (localhost:3000) and Echo Server simultaneously. + + Visit http://localhost:3000 to see Echo running! + +### Manual Setup (Alternative) + +If you prefer to set up manually or the script encounters issues: +1. **Install dependencies** ```bash - cd packages/app/control - ./setup-db.sh - # Or manually: - npx prisma generate - npx prisma db push + pnpm install ``` -5. **Start development servers** - - From the root directory: +2. **Set up Echo Control** + ```bash + cd packages/app/control + pnpm local-setup + # This generates .env file, AUTH_SECRET, and runs migrations + cd ../.. + ``` +3. **Start development** ```bash pnpm dev ``` - This starts both Echo Control (localhost:3000) and Echo Server simultaneously. - ## Development Workflow ### Creating a Branch diff --git a/README.md b/README.md index bba3874ae..188565552 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,101 @@ Or run `npx echo-start my-app` to choose interactively. # Development -Fill out `packages/app/control/.env` and `packages/app/server/.env`. Then... +## Prerequisites -- `pnpm i` -- `pnpm dev` +Before starting, ensure you have: + +- **Node.js 18+** - Download from [nodejs.org](https://nodejs.org/) +- **pnpm 10+** - Install with `npm install -g pnpm` +- **Docker & Docker Compose** - Download from [docker.com](https://www.docker.com/products/docker-desktop) + +## Quick Start + +### Automated Setup (Recommended) + +```bash +# Run the setup script (handles dependencies, database, and configuration) +bash scripts/setup-local.sh +``` + +Then start development: + +```bash +pnpm dev +``` + +Visit [http://localhost:3000](http://localhost:3000) - you should see Echo running! + +### Manual Setup + +If you prefer manual setup or the script fails: + +1. **Install dependencies**: + ```bash + pnpm install + ``` + +2. **Set up Echo Control (frontend/API)**: + ```bash + cd packages/app/control + cp .env.example .env + pnpm local-setup + ``` + +3. **Start the development servers**: + ```bash + cd ../.. # Back to root + pnpm dev + ``` + +## Configuration + +### Required Environment Variables + +**Echo Control** (`packages/app/control/.env`): +- `AUTH_SECRET` - Generated automatically by the setup script +- `DATABASE_URL` - PostgreSQL connection string (set to `postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public`) + +**Echo Server** (`packages/app/server/.env`): +- `DATABASE_URL` - PostgreSQL connection string (set to `postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public`) +- `ECHO_CONTROL_BASE_URL` - Set to `http://localhost:3000/` + +For local development, most other variables (Stripe, OAuth providers, API keys) are optional and can use mock values. + +## Database Management + +PostgreSQL runs in Docker. Useful commands: + +```bash +# View database in Prisma Studio +pnpm -C packages/app/control prisma studio + +# View database logs +docker compose -f packages/app/control/docker-local-db.yml logs postgres + +# Stop the database +docker compose -f packages/app/control/docker-local-db.yml down + +# Reset database (WARNING: deletes all data) +pnpm -C packages/app/control prisma migrate reset +``` + +## Troubleshooting + +### Docker Connection Error +Make sure Docker Desktop is running. On Mac/Windows, launch Docker Desktop. On Linux, ensure the Docker daemon is running. + +### Port Already in Use +If port 3000 or 3070 is already in use: +- Find the process: `lsof -i :3000` +- Kill it: `kill -9 ` +- Or change the port in the `.env` files + +### Database Connection Failed +1. Check Docker is running: `docker ps` +2. Check PostgreSQL is running: `docker ps | grep postgres` +3. View logs: `docker logs echo-control-postgres-v2` +4. Reset: `docker compose -f packages/app/control/docker-local-db.yml down -v` + +### pnpm command not found +Install pnpm globally: `npm install -g pnpm@10.11.0` diff --git a/packages/app/control/README.md b/packages/app/control/README.md index 99d3dce93..0db8b4257 100644 --- a/packages/app/control/README.md +++ b/packages/app/control/README.md @@ -46,44 +46,47 @@ A comprehensive Next.js application for managing Echo applications, API keys, an ### Prerequisites - Node.js 18+ -- PostgreSQL database -- pnpm +- Docker & Docker Compose (PostgreSQL runs in Docker) +- pnpm 10+ -### Installation +### Quick Start (From Root) -1. **Clone and navigate to the project**: +The easiest way is to use the root setup script: - ```bash - cd echo-control - ``` - -2. **Install dependencies**: +```bash +bash scripts/setup-local.sh +``` - ```bash - pnpm install - ``` +Then: +```bash +pnpm dev +``` -3. **Create .env file**: +### Manual Setup (From echo-control directory) +1. **Navigate to this directory**: ```bash - # Generate .env file - pnpm local-setup + cd packages/app/control ``` -4. **Run database migrations**: - +2. **Create .env file**: ```bash - npx prisma generate - npx prisma db push + cp .env.example .env + pnpm local-setup ``` -5. **Start the development server**: + This script will: + - Generate an AUTH_SECRET if not present + - Set database connection string + - Start PostgreSQL in Docker + - Run Prisma migrations +3. **Start the development server**: ```bash - pnpm run dev + pnpm dev ``` -6. **Open the application**: +4. **Open the application**: Visit [http://localhost:3000](http://localhost:3000) ## Features Overview diff --git a/packages/app/control/scripts/setup.sh b/packages/app/control/scripts/setup.sh index c7e3e89b7..7d0a8e83a 100755 --- a/packages/app/control/scripts/setup.sh +++ b/packages/app/control/scripts/setup.sh @@ -1,20 +1,86 @@ #!/bin/bash -# Check if .env file exists and if AUTH_SECRET is already set -if [ -f .env ] && grep -q "^AUTH_SECRET=" .env; then - echo "AUTH_SECRET already exists in .env file. Skipping generation." -else +set -e + +echo "Setting up Echo Control local development..." +echo "" + +# Create .env file if it doesn't exist +if [ ! -f .env ]; then + echo "Creating .env file from template..." + cp .env.example .env +fi + +# Generate AUTH_SECRET if not present +if ! grep -q "^AUTH_SECRET=" .env || grep -q "^AUTH_SECRET=$" .env; then echo "Generating AUTH_SECRET..." - AUTH_SECRET=$(node -e "console.log('AUTH_SECRET=' + require('crypto').randomBytes(32).toString('base64'))") - echo $AUTH_SECRET - echo $AUTH_SECRET >> .env + AUTH_SECRET=$(node -e "console.log(require('crypto').randomBytes(32).toString('base64'))") + # Use grep and mv to safely update AUTH_SECRET, avoiding sed issues with special chars and improving portability. + grep -v "^AUTH_SECRET=" .env > .env.tmp + echo "AUTH_SECRET=$AUTH_SECRET" >> .env.tmp + mv .env.tmp .env + echo "OK: AUTH_SECRET generated" +fi + +# Ensure DATABASE_URL is set for local dev +if ! grep -q "^DATABASE_URL=" .env || grep -q "^DATABASE_URL=$" .env; then + echo "Setting DATABASE_URL..." + DB_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" + grep -v "^DATABASE_URL=" .env > .env.tmp + echo "DATABASE_URL=\"$DB_URL\"" >> .env.tmp + mv .env.tmp .env + echo "OK: DATABASE_URL configured" fi -# Check if DATABASE_URL is already set -if [ -f .env ] && grep -q "^DATABASE_URL=" .env; then - echo "DATABASE_URL already exists in .env file. Skipping." -else - echo "DATABASE_URL='postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public'" >> .env +# Ensure ECHO_CONTROL_APP_BASE_URL is set +if ! grep -q "^ECHO_CONTROL_APP_BASE_URL=" .env || grep -q "^ECHO_CONTROL_APP_BASE_URL=$" .env; then + echo "Setting ECHO_CONTROL_APP_BASE_URL..." + grep -v "^ECHO_CONTROL_APP_BASE_URL=" .env > .env.tmp + echo "ECHO_CONTROL_APP_BASE_URL=http://localhost:3000" >> .env.tmp + mv .env.tmp .env fi -echo "Setup complete! You can now run 'pnpm dev' to start the development server." +echo "" +echo "Starting PostgreSQL Docker container..." + +# Start PostgreSQL container +docker compose -f docker-local-db.yml up -d postgres + +# Wait for PostgreSQL to be ready with health check +echo "Waiting for PostgreSQL to be ready..." +max_attempts=30 +attempt=0 +while [ $attempt -lt $max_attempts ]; do + if docker compose -f docker-local-db.yml exec -T postgres pg_isready -U echo_user -d echo_control_v2 &> /dev/null; then + echo "OK: PostgreSQL is ready" + break + fi + attempt=$((attempt + 1)) + if [ $((attempt % 5)) -eq 0 ]; then + echo " Waiting... ($attempt/$max_attempts)" + fi + sleep 1 +done + +if [ $attempt -eq $max_attempts ]; then + echo "ERROR: PostgreSQL failed to start after 30 seconds" + echo " Check Docker logs: docker compose -f docker-local-db.yml logs postgres" + exit 1 +fi + +echo "" +echo "Running Prisma migrations..." +pnpm prisma generate +pnpm prisma db push --skip-generate + +echo "" +echo "Setup complete!" +echo "" +echo "Next steps:" +echo " 1. Return to the root directory: cd ../.." +echo " 2. Start development: pnpm dev" +echo " 3. Open http://localhost:3000 in your browser" +echo "" +echo "Useful commands:" +echo " pnpm prisma studio # View the database GUI" +echo " docker logs # View Docker logs" diff --git a/packages/app/server/setup-env.sh b/packages/app/server/setup-env.sh new file mode 100755 index 000000000..6527c8c7d --- /dev/null +++ b/packages/app/server/setup-env.sh @@ -0,0 +1,17 @@ +#!/bin/bash + +# Create .env file if it doesn't exist +if [ ! -f .env ]; then + echo "Creating .env file for echo-server..." + cp .env.example .env + + echo "OK: Created .env file" + echo "" + echo "NOTE: The following environment variables need to be configured:" + echo " - ECHO_API_KEY: API key from echo-control (get from dashboard after setup)" + echo " - Provider keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)" + echo "" + echo "See the .env file for more information." +else + echo "INFO: .env file already exists, skipping creation" +fi \ No newline at end of file diff --git a/scripts/setup-local.sh b/scripts/setup-local.sh new file mode 100755 index 000000000..5d059ff64 --- /dev/null +++ b/scripts/setup-local.sh @@ -0,0 +1,126 @@ +#!/bin/bash + +set -e + +echo "Welcome to Echo! Let's set up your local development environment." +echo "" + +# Check prerequisites +echo "Checking prerequisites..." +echo "" + +# Check Node.js +if ! command -v node &> /dev/null; then + echo "ERROR: Node.js is not installed. Please install Node.js 18+ from https://nodejs.org/" + exit 1 +fi + +NODE_VERSION=$(node -v | cut -d'v' -f2 | cut -d'.' -f1) +if [ "$NODE_VERSION" -lt 18 ]; then + echo "ERROR: Node.js version 18+ is required. You have version $(node -v)" + exit 1 +fi +echo "OK: Node.js $(node -v) detected" + +# Check pnpm +if ! command -v pnpm &> /dev/null; then + echo "ERROR: pnpm is not installed. Installing pnpm..." + npm install -g pnpm +fi +echo "OK: pnpm $(pnpm -v) detected" + +# Check Docker +if ! command -v docker &> /dev/null; then + echo "ERROR: Docker is not installed. Please install Docker from https://www.docker.com/products/docker-desktop" + exit 1 +fi +echo "OK: Docker detected" + +# Check Docker Compose +if ! docker compose version &> /dev/null; then + echo "ERROR: Docker Compose is not installed. Please install Docker Compose from https://docs.docker.com/compose/install/" + exit 1 +fi +echo "OK: Docker Compose detected" + +echo "" +echo "Installing dependencies..." +pnpm install + +echo "" +echo "Setting up databases..." + +# Setup control package database +echo "Setting up Echo Control database..." +cd packages/app/control + +# Create .env file if it doesn't exist +if [ ! -f .env ]; then + echo "Creating .env file for echo-control..." + cp .env.example .env +fi + +# Generate AUTH_SECRET if not present +if ! grep -q "^AUTH_SECRET=" .env || grep -q "^AUTH_SECRET=$" .env; then + echo "Generating AUTH_SECRET..." + AUTH_SECRET=$(node -e "console.log(require('crypto').randomBytes(32).toString('base64'))") + grep -v "^AUTH_SECRET=" .env > .env.tmp + echo "AUTH_SECRET=$AUTH_SECRET" >> .env.tmp + mv .env.tmp .env + echo "OK: AUTH_SECRET generated" +fi + +# Ensure DATABASE_URL is set for local dev +if ! grep -q "^DATABASE_URL=" .env || grep -q "^DATABASE_URL=$" .env; then + echo "Setting DATABASE_URL..." + DB_URL="postgresql://echo_user:echo_password@localhost:5469/echo_control_v2?schema=public" + grep -v "^DATABASE_URL=" .env > .env.tmp + echo "DATABASE_URL=\"$DB_URL\"" >> .env.tmp + mv .env.tmp .env + echo "OK: DATABASE_URL configured" +fi + +# Start PostgreSQL container +echo "Starting PostgreSQL container..." +docker compose -f docker-local-db.yml up -d postgres + +# Wait for PostgreSQL to be ready +echo "Waiting for PostgreSQL to be ready..." +max_attempts=30 +attempt=0 +while [ $attempt -lt $max_attempts ]; do + if docker compose -f docker-local-db.yml exec -T postgres pg_isready -U echo_user -d echo_control_v2 &> /dev/null; then + echo "OK: PostgreSQL is ready" + break + fi + attempt=$((attempt + 1)) + sleep 1 +done + +if [ $attempt -eq $max_attempts ]; then + echo "ERROR: PostgreSQL failed to start. Please check Docker logs:" + echo " docker compose -f docker-local-db.yml logs postgres" + exit 1 +fi + +# Run Prisma migrations +echo "Setting up database schema..." +pnpm prisma generate +pnpm prisma db push --skip-generate + +cd ../.. + +echo "" +echo "Setup complete!" +echo "" +echo "You're ready to go! Start development with:" +echo "" +echo " pnpm dev" +echo "" +echo "Then open http://localhost:3000 in your browser." +echo "" +echo "Useful commands:" +echo " pnpm dev - Start both control and server" +echo " pnpm -C packages/app/control prisma studio - View database GUI" +echo " docker compose -f packages/app/control/docker-local-db.yml logs postgres - View database logs" +echo ""