🚀 FastAPI + Rust High-Performance Text Processor

A high-performance text processing API that combines the ease of Python FastAPI with the speed of Rust extensions, demonstrating how to perfectly merge Python's usability with Rust's performance advantages.

🎯 Key Features

• 🔥 Extreme Performance: Core algorithms implemented in Rust with 10-100x performance improvement
• 🐍 Python Simplicity: Built with FastAPI for easy-to-understand and maintainable APIs
• 🦀 Rust Safety: Memory-safe, zero-cost abstractions
• 📊 Real-time Performance Monitoring: Every request includes processing time statistics
• 🎨 Modern Architecture: Follows best practices in project structure

🏗️ Architecture Design

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   HTTP Request  │───▶│   FastAPI       │───▶│   Rust Extension│
│   JSON Data     │    │   Data Validation│    │   High-Perf Compute│
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                ▲                        │
                                │                        ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   JSON Response │◀───│   Python Service│◀───│   Compute Result│
│   Performance   │    │   Format Results│    │   Memory Safety │
└─────────────────┘    └─────────────────┘    └─────────────────┘

🚀 Core Functionality

1. 📊 Word Frequency Counter

• Function: Analyze word occurrence frequency in text
• Algorithm: Regex matching + HashMap counting
• Performance: Process 100k words in < 50ms

2. 📧 Email Extraction

• Function: Extract all email addresses from any text
• Algorithm: Efficient regex pattern matching
• Accuracy: 99.9% standard email format recognition

3. 🧹 Text Cleaning

• Function: Clean and normalize text content
• Features: Parallel processing, multi-threaded optimization
• Applications: Data preprocessing, content filtering

🛠️ Tech Stack

• Backend Framework: FastAPI 0.104+
• Core Languages: Python 3.8+ & Rust 1.70+
• Python-Rust Binding: PyO3
• Build Tool: Maturin
• Parallel Processing: Rayon (Rust)
• Data Validation: Pydantic
• Testing Framework: pytest

📦 Project Structure

fastapi-rust-text-processor/
├── text_processor_rust/        # Rust extension module
│   ├── Cargo.toml              # Rust project configuration
│   ├── pyproject.toml          # Python build configuration
│   └── src/
│       ├── lib.rs              # Rust core implementation
│       └── sentiment.rs        # Sentiment mod
│           ├── tokenizer.rs    # Tokenizes text
│           ├── dictionary.rs   # Manages sentiment dictionary
│           ├── rules.rs        # Defines classification rules
│           └── analyzer.rs     # Combines & analysis
├── app/                        # FastAPI application
│   ├── __init__.py
│   ├── main.py                 # Main application entry
│   ├── models.py               # Pydantic data models
│   ├── services.py             # Business logic services
│   └── pyproject.toml          # Python build configuration
├── tests/                      # Test files
│   ├── conftest.py             # Test configuration and fixtures
│   ├── test_rust_extension.py  # Rust extension unit tests
│   ├── test_services.py        # Business logic tests
│   ├── test_api.py             # API endpoint tests
│   ├── test_performance.py     # Performance comparison tests
│   ├── test_sentiment.py       # Sentiment tests
│   └── test_integration.py     # Integration tests
├── examples/                   # Example code
│   ├── basic_usage.py          # Basic usage example
│   ├── performance_demo.py     # Performance demonstration script 
│   ├── sentiment_demo.py       # Sentiment script 
│   └── batch_processing.py     # Batch processing example             
├── requirements.txt            # Python dependencies
├── requirements-dev.txt        # Development dependencies
├── Makefile                    # Development commands
├── LICENSE                     # License 
├── README.md                   # Project documentation
└── .gitignore                  # Git ignore file

🚀 Quick Start

Prerequisites

• Python 3.8+
• Rust 1.70+
• Git

1. Clone the Repository

git clone https://github.com/Rafa-Gu98/fastapi-rust-text-processor.git
cd fastapi-rust-text-processor

2. Set Up Python Environment

# Create virtual environment
python -m venv venv

# Activate virtual environment
# Windows
venv\Scripts\activate
# Linux/Mac
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

3. Install Rust (if not already installed)

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

4. Build Rust Extension

cd rust_extension
maturin develop --release
cd ..

5. Run the Application

uvicorn app.main:app --reload

6. Access the API

• API Documentation: http://localhost:8000/docs
• Health Check: http://localhost:8000/health

📊 API Endpoints

Word Frequency Count

POST /count-words
Content-Type: application/json

{
    "text": "Hello world! Hello again world."
}

Email Extraction

POST /extract-emails
Content-Type: application/json

{
    "text": "Contact us: admin@example.com or support@company.org"
}

Text Cleaning

POST /clean-text
Content-Type: application/json

{
    "text": "Hello!!! @#$%^&*() World???"
}

🧪 Testing

Prerequisites

First, install development dependencies:

pip install -r requirements-dev.txt

The development dependencies include:

pytest>=7.4.0
pytest-asyncio>=0.21.0
pytest-cov>=4.1.0
pytest-mock>=3.11.0
pytest-benchmark>=4.0.0
httpx>=0.24.0
psutil>=5.9.0
aiohttp>=3.8.0
coverage>=7.2.0

Running Tests

Basic Test Commands

# Run all tests
pytest

# Run specific test file with verbose output
pytest tests/test_performance.py -v

# Run tests with coverage report (HTML format)
pytest --cov=app --cov-report=html

# Run performance benchmark tests with output
pytest tests/test_performance.py::TestPerformanceComparison -v -s

# Run tests in parallel (requires pytest-xdist)
pytest -n auto

Using Makefile (Recommended)

# Install development environment
make dev-install

# Run all tests
make test

# Run performance tests only
make test-performance

# Generate coverage report
make test-coverage

# Clean up build artifacts
make clean

# Start development server
make dev

Test Structure

tests/
├── conftest.py              # Test configuration and fixtures
├── test_rust_extension.py   # Rust extension unit tests
├── test_services.py         # Business logic tests
├── test_api.py             # API endpoint tests
├── test_performance.py      # Performance comparison tests
└── test_integration.py      # Integration tests

Performance Testing

Our performance tests compare Rust implementation against pure Python:

# Run performance comparison
pytest tests/test_performance.py::TestPerformanceComparison -v -s

# Expected output:
# Performance Comparison:
# Rust average: 0.0234s
# Python average: 0.1456s
# Speedup: 6.22x

Coverage Reports

After running pytest --cov=app --cov-report=html, open htmlcov/index.html in your browser to view detailed coverage reports.

Continuous Integration

This project includes comprehensive tests suitable for CI/CD pipelines:

• Unit tests for individual components
• Integration tests for API endpoints
• Performance benchmarks
• Memory usage validation
• Concurrent request handling

Troubleshooting Tests

If tests fail:

1. Rust extension not found: Make sure you've run maturin develop in the rust_extension/ directory
2. Import errors: Ensure all dependencies are installed with pip install -r requirements-dev.txt
3. Performance tests fail: Performance thresholds may vary by system; adjust if necessary
4. Memory tests fail: Close other applications that might affect memory measurements

📈 Performance Comparison

Operation	Pure Python	Python+Rust	Performance Gain
Word Count (100k words)	2.5s	0.05s	50x
Email Extract (1MB text)	0.8s	0.02s	40x
Text Clean (parallel)	1.2s	0.03s	40x

🧪 Development Guide

Adding New Text Processing Functions

1. Implement core algorithm in Rust

#[pyfunction]
fn your_function(text: &str) -> PyResult<YourResult> {
    // High-performance implementation
}

2. Add API endpoint in Python

@app.post("/your-endpoint")
async def your_endpoint(input_data: YourInput):
    return service.your_function(input_data.text)

Performance Optimization Tips

• Use rayon for parallel processing
• Avoid frequent memory allocations
• Leverage Rust's zero-cost abstractions
• Implement intelligent caching mechanisms

🤝 Contributing

1. Fork the project
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📋 Roadmap

• Add sentiment analysis functionality
• Implement text summarization
• Add language detection
• Create Docker deployment
• Add GraphQL support
• Implement caching layer

🔒 Security

• All input is validated using Pydantic models
• Rust extensions provide memory safety
• No eval() or exec() usage
• Input sanitization for all text processing

📊 Benchmarks

System Specifications

• CPU: Intel Core Ultra 9 275HX
• RAM: 32GB DDR5
• OS: Ubuntu 22.04 LTS

Results

Word Count (1M words):
- Pure Python: 25.3s
- Python + Rust: 0.48s
- Speed improvement: 52.7x

Email Extraction (10MB text):
- Pure Python: 8.2s
- Python + Rust: 0.19s
- Speed improvement: 43.2x

Text Cleaning (5MB text):
- Pure Python: 12.1s
- Python + Rust: 0.31s
- Speed improvement: 39.0x

🎓 Learning Resources

• FastAPI Documentation
• Rust Programming Language
• PyO3 User Guide
• Maturin Documentation

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• FastAPI - Modern, fast web framework
• PyO3 - Python-Rust bindings
• Maturin - Build tool for Python extensions
• Rayon - Parallel computing library

📞 Contact

• Project Link: https://github.com/Rafa-Gu98/fastapi-rust-text-processor
• Issue Tracker: https://github.com/Rafa-Gu98/fastapi-rust-text-processor/issues
• Email: rafagr98.dev@gmail.com

🌟 Star History

---

⭐ If this project helps you, please give it a star!