🩺 AI Healthcare Voice Assistant

A comprehensive full-stack healthcare application that leverages AI-powered voice interactions for patient triage, symptom analysis, specialist mapping, and appointment booking. Built with modern technologies including FastAPI, PostgreSQL, React, and integrated with leading AI models.

🔄 System Architecture & User Flow

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                           🎤 VOICE-ENABLED USER INTERFACE                           │
│                                  (React Frontend)                                   │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                              🔐 USER AUTHENTICATION                                 │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   User Login    │───▶│  FastAPI Backend │───▶│     PostgreSQL Database         │ │
│  │ (Email/Password)│    │   sp_login_user  │    │      (users table)             │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          🎙️ VOICE SYMPTOM COLLECTION                               │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │  Web Speech API │───▶│ Speech-to-Text   │───▶│    Symptom Phrases Array       │ │
│  │  (Microphone)   │    │   Conversion     │    │   ["headache", "fever", ...]   │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                        🤖 AI-POWERED SYMPTOM ANALYSIS                              │
│                              (LangGraph Workflow)                                  │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Raw Symptoms  │───▶│  Gemini/GPT-4    │───▶│   Normalized Symptoms           │ │
│  │   Processing    │    │   AI Analysis    │    │ ["migraine", "pyrexia", ...]    │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
│                                  │                                                  │
│                                  ▼                                                  │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Specialist    │◀───│   AI Mapping     │◀───│     Symptom Analysis            │ │
│  │ Recommendation  │    │    Algorithm     │    │      & Classification          │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          🏥 DOCTOR MATCHING & RETRIEVAL                            │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Specialist    │───▶│     FastAPI      │───▶│      PostgreSQL Query          │ │
│  │     Types       │    │sp_get_doctors_by │    │    (doctors table filter)      │ │
│  │ ["cardiology",  │    │   _specialist    │    │                                 │ │
│  │ "neurology"]    │    │                  │    │                                 │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
│                                  │                                                  │
│                                  ▼                                                  │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │  Available      │◀───│   Doctor List    │◀───│     Available Time Slots       │ │
│  │   Doctors       │    │   with Slots     │    │        & Schedules             │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                           📅 APPOINTMENT BOOKING                                   │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   User Selects  │───▶│     Patient      │───▶│       Appointment               │ │
│  │ Doctor & Slot   │    │   Information    │    │      Confirmation               │ │
│  │                 │    │sp_get_patient_   │    │  sp_create_appointment          │ │
│  │                 │    │    details       │    │                                 │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                            💳 PAYMENT PROCESSING                                   │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Razorpay      │───▶│   Secure Payment │───▶│      Booking Confirmation       │ │
│  │   Gateway       │    │    Processing    │    │       & Notification            │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘

                                        ▼

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          ✅ COMPLETE HEALTHCARE JOURNEY                            │
│                                                                                     │
│  🎤 Voice Input → 🤖 AI Analysis → 🏥 Doctor Match → 📅 Booking → 💳 Payment      │
└─────────────────────────────────────────────────────────────────────────────────────┘

🔍 Detailed Component Interaction Flow

1. User Authentication Flow

User Input (Email/Password) → FastAPI Login Endpoint → PostgreSQL sp_login_user → JWT Token/Session

2. Voice Processing Pipeline

Microphone → Web Speech API → Text Conversion → Phrase Array → LangGraph Processing

3. AI Analysis Workflow

Raw Symptoms → Gemini/GPT-4 → Normalized Symptoms → Specialist Mapping → Doctor Recommendations

4. Database Integration Pattern

FastAPI Endpoints → PostgreSQL Stored Procedures → Data Retrieval → JSON Response → Frontend Display

5. Appointment Booking Chain

Doctor Selection → Patient Details Fetch → Slot Validation → Appointment Creation → Payment Processing

🏗️ Technical Architecture Overview

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│                 │    │                 │    │                 │    │                 │
│   FRONTEND      │◀──▶│    BACKEND      │◀──▶│    DATABASE     │◀──▶│   EXTERNAL      │
│                 │    │                 │    │                 │    │   SERVICES      │
│  • React        │    │  • FastAPI      │    │  • PostgreSQL   │    │  • Gemini API   │
│  • Web Speech   │    │  • LangGraph    │    │  • Stored Procs │    │  • OpenAI API   │
│  • JavaScript   │    │  • Python       │    │  • Functions    │    │  • Razorpay     │
│                 │    │                 │    │                 │    │                 │
└─────────────────┘    └─────────────────┘    └─────────────────┘    └─────────────────┘
        │                        │                        │                        │
        └────────────────────────┼────────────────────────┼────────────────────────┘
                                 │                        │
                            HTTP/REST API            SQL Queries
                         (JSON Data Exchange)    (Stored Procedures)

🌟 Overview

This healthcare AI assistant streamlines the patient care journey by providing:

Intelligent Voice Triage: Natural language symptom collection and analysis
AI-Powered Diagnosis: Advanced symptom normalization and specialist recommendation
Smart Doctor Matching: Automated healthcare provider lookup based on specialization
Seamless Booking: Integrated appointment scheduling with payment processing
Conversational Interface: Intuitive voice-enabled user experience

✨ Key Features

🎤 Voice-Enabled Interaction

Real-time speech-to-text conversion using Web Speech API
Natural language processing for symptom collection
Voice-guided patient triage workflow

🤖 AI-Powered Medical Intelligence

Integration with Gemini and GPT-4 for symptom analysis
LangGraph-based symptom normalization
Intelligent specialist mapping and recommendations

🏥 Healthcare Management

Comprehensive doctor database with specialization filtering
PostgreSQL-powered efficient data retrieval
Automated appointment scheduling system

💳 Payment Integration

Secure payment processing via Razorpay
Test mode support for development
Transaction management and tracking

🛠️ Technology Stack

Component	Technology	Purpose
Frontend	React, CSS, JavaScript	User interface and voice interactions
Backend	FastAPI, Python	API services and business logic
AI/ML	Gemini, GPT-4, LangGraph	Natural language processing and AI agents
Database	PostgreSQL	Data persistence and stored procedures
Voice	Web Speech API	Speech recognition and synthesis
Payments	Razorpay	Payment processing and gateway
Deployment	Uvicorn, Vite	Development and production servers

📂 Project Architecture

healthcare-ai-assistant/
├── backend/                    # FastAPI backend services
│   ├── main.py                # Application entry point
│   ├── agents/                # LangGraph AI agents
│   ├── db/                    # Database connections and queries
│   ├── models.py              # Data models and schemas
│   ├── config.py              # Configuration management
│   └── .env                   # Environment variables (excluded from git)
│
├── frontend/                   # React frontend application
│   ├── components/            # Reusable UI components
│   ├── pages/                 # Application pages/views
│   └── .env.local             # Frontend environment variables
│
├── sql/                       # Database schema and functions
│   ├── schema.sql             # Database table definitions
│   └── functions/             # PostgreSQL stored procedures
│       ├── create_login_function.sql
│       ├── create_appointment_function.sql
│       ├── get_patient_details.sql
│       └── get_doctors_by_specialist.sql
│
├── requirements.txt           # Python dependencies
├── package.json              # Node.js dependencies
├── .gitignore                # Git ignore rules
└── README.md                 # Project documentation

🚀 Quick Start Guide

Prerequisites

Python 3.8+
Node.js 16+
PostgreSQL 12+
Git

1. 🗄️ Database Setup

Create Database and User

psql -U postgres

CREATE DATABASE healthcare;
CREATE USER fastapi_user WITH PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE healthcare TO fastapi_user;
\q

Initialize Schema and Functions

psql -U fastapi_user -d healthcare -f sql/schema.sql

psql -U fastapi_user -d healthcare -f sql/functions/create_login_function.sql

psql -U fastapi_user -d healthcare -f sql/functions/create_appointment_function.sql

psql -U fastapi_user -d healthcare -f sql/functions/get_patient_details.sql

psql -U fastapi_user -d healthcare -f sql/functions/get_doctors_by_specialist.sql

2. 🔧 Backend Configuration

Setup Virtual Environment

cd backend

python -m venv venv

Windows:

venv\Scripts\activate

macOS/Linux:

source venv/bin/activate

Install Dependencies

pip install -r requirements.txt

Environment Configuration

Create .env file in the backend/ directory:

# Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_USER=fastapi_user
DB_PASSWORD=your_secure_password
DB_NAME=healthcare

# CORS Configuration
FRONTEND_ORIGIN=http://localhost:5173

# AI API Keys
OPENAI_API_KEY=your_openai_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here

Start Backend Server

uvicorn main:app --reload --host 0.0.0.0 --port 8000

3. 🎨 Frontend Setup

Install Dependencies

npm install

Environment Configuration

Create .env.local file in the root directory:

# API Configuration
VITE_BACKEND_URL=http://localhost:8000
VITE_GEMINI_API_KEY=your_gemini_api_key_here

# Payment Configuration
VITE_RAZORPAY_KEY_ID=your_razorpay_key_id_here

Start Development Server

npm run dev

4. 💳 Payment Setup (Optional)

Create a Razorpay account
Navigate to API Keys section in dashboard
Copy the Key ID and Key Secret
Update the payment configuration in frontend/components/Recommendation.jsx

🔑 API Keys Setup

OpenAI API Key

Visit OpenAI Platform
Create an account and navigate to API Keys
Generate a new secret key
Add to backend .env file

Google Gemini API Key

Go to Google AI Studio
Create a new project or select existing
Generate API key
Add to both backend .env and frontend .env.local

Razorpay Configuration

Sign up at Razorpay Dashboard
Switch to Test Mode for development
Copy API keys from Settings > API Keys
Configure in frontend environment

🏃‍♂️ Running the Application

Start PostgreSQL service
Launch Backend: uvicorn main:app --reload (from backend/ directory)
Launch Frontend: npm run dev (from root directory)
Access Application: Navigate to http://localhost:5173

🧪 Testing

Backend API Testing

curl http://localhost:8000/health

Database Connection Testing

python -c "from backend.db.connection import get_db_connection; print('DB Connected!' if get_db_connection() else 'DB Connection Failed!')"

🚀 Deployment

Backend Deployment

Configure production database credentials
Set up environment variables on hosting platform
Deploy using platforms like Heroku, Railway, or DigitalOcean

Frontend Deployment

Build production bundle: npm run build
Deploy to Vercel, Netlify, or similar platforms
Update CORS settings in backend for production domain

🤝 Contributing

Fork the repository
Create a feature branch: git checkout -b feature/amazing-feature
Commit changes: git commit -m 'Add amazing feature'
Push to branch: git push origin feature/amazing-feature
Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🔧 Troubleshooting

Common Issues

Database Connection Error

Verify PostgreSQL is running
Check database credentials in .env
Ensure database and user exist

Voice Recognition Not Working

Use HTTPS or localhost only
Check browser microphone permissions
Verify Web Speech API support

API Key Errors

Validate API keys are correctly set
Check for trailing spaces or quotes
Verify API key permissions and quotas

🔮 Future Enhancements

Multi-language support
Mobile application development
Advanced AI model integration
Telemedicine video consultation
Electronic health records integration
Real-time chat support
Advanced analytics dashboard

Built with ❤️ for better healthcare accessibility

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

🩺 AI Healthcare Voice Assistant

🔄 System Architecture & User Flow

🔍 Detailed Component Interaction Flow

1. User Authentication Flow

2. Voice Processing Pipeline

3. AI Analysis Workflow

4. Database Integration Pattern

5. Appointment Booking Chain

🏗️ Technical Architecture Overview

🌟 Overview

✨ Key Features

🎤 Voice-Enabled Interaction

🤖 AI-Powered Medical Intelligence

🏥 Healthcare Management

💳 Payment Integration

🛠️ Technology Stack

📂 Project Architecture

🚀 Quick Start Guide

Prerequisites

1. 🗄️ Database Setup

Create Database and User

Initialize Schema and Functions

2. 🔧 Backend Configuration

Setup Virtual Environment

Install Dependencies

Environment Configuration

Start Backend Server

3. 🎨 Frontend Setup

Install Dependencies

Environment Configuration

Start Development Server

4. 💳 Payment Setup (Optional)

🔑 API Keys Setup

OpenAI API Key

Google Gemini API Key

Razorpay Configuration

🏃‍♂️ Running the Application

🧪 Testing

Backend API Testing

Database Connection Testing

🚀 Deployment

Backend Deployment

Frontend Deployment

🤝 Contributing

📄 License

🔧 Troubleshooting

Common Issues

🔮 Future Enhancements