FluentRead is an innovative Android application that revolutionizes the reading experience through hands-free scrolling using eye-tracking technology. Simply look up or down to scroll through content, making it perfect for accessibility and hands-free reading scenarios.
- Eye-Tracking Scrolling: Scroll content by looking up or down using real-time face detection
- Cross-App Compatibility: Works system-wide through Android Accessibility Services
- Real-Time Processing: Continuous camera feed processing with optimized performance
- Privacy-First: All processing happens locally on your device - no data sent to external servers
- Intuitive Onboarding: Step-by-step setup with permission management
- Modern UI: Built with Jetpack Compose and Material Design 3
- Responsive Design: Smooth animations and transitions
- Permission Management: Clear explanation of required permissions
- Language: Kotlin
- UI Framework: Jetpack Compose
- Architecture: MVVM with Clean Architecture principles
- Dependency Injection: Dagger Hilt
- Camera: CameraX for real-time camera processing
- ML/AI: Google ML Kit Face Detection
- Database: Room for local data persistence
- Navigation: Navigation Compose
- Accessibility: Android Accessibility Services
- Face Detection: Uses Google ML Kit to detect facial landmarks and head orientation
- Behavior Analysis: Analyzes head Euler angles to determine if you're looking up or down
- Scroll Control: Converts eye movements into scroll actions via Accessibility Services
- Cross-App Integration: Works with any scrollable content across different apps
- Android 7.0 (API level 24) or higher
- Camera permission
- Accessibility Service permission
-
Clone the repository
git clone https://github.com/[your-username]/FluentRead.git cd FluentRead -
Open in Android Studio
- Import the project
- Sync Gradle files
- Build the project
-
Install on device
./gradlew installDebug
- Launch the app - You'll see the splash screen
- Complete onboarding - Learn about the app's features
- Grant permissions:
- Camera Permission: Required for eye tracking
- Accessibility Service: Required for cross-app scrolling
- Start using - The app will begin tracking your eye movements
app/src/main/java/com/example/fluentread/
├── features/
│ ├── splash/ # Splash screen
│ ├── onboarding/ # Onboarding flow
│ ├── settings/ # App settings
│ └── components/ # Reusable UI components
├── service/
│ ├── camera/ # CameraX integration
│ ├── mlk/ # ML Kit face detection
│ ├── accessibility/ # Accessibility service
│ └── overlay/ # UI overlay components
├── cache/ # Room database
├── di/ # Dependency injection
└── utils/ # Constants and utilities
- Purpose: Required for real-time face detection and eye tracking
- Usage: Captures camera frames to analyze head orientation
- Privacy: All processing happens locally on your device
- Purpose: Enables system-wide scrolling functionality
- Usage: Monitors screen content and performs scroll actions
- Scope: Works across all apps with scrollable content
The following diagram shows how the different services communicate with each other:
graph TB
subgraph "UI Layer"
A[MainActivity] --> B[AppNavigation]
B --> C[SplashScreen]
B --> D[OnboardingScreen]
B --> E[EnablePermissionScreen]
B --> F[SettingScreen]
end
subgraph "Service Layer"
G[AppController] --> H[CameraXService]
G --> I[FaceDetectorService]
G --> J[ScrollAccessibilityService]
H --> K[Camera Frames]
K --> I
I --> L[Face Behavior Analysis]
L --> J
J --> M[System Scroll Actions]
end
subgraph "Data Layer"
N[Room Database]
O[SettingRepository]
P[Permission Manager]
end
subgraph "External Services"
Q[Google ML Kit]
R[Android Accessibility]
end
E --> P
F --> O
O --> N
I --> Q
J --> R
style A fill:#e1f5fe
style G fill:#f3e5f5
style H fill:#fff3e0
style I fill:#e8f5e8
style J fill:#fce4ec
style Q fill:#fff8e1
style R fill:#f1f8e9
The following diagram shows the detailed data flow from camera input to scroll action:
sequenceDiagram
participant User
participant Camera as CameraXService
participant ML as FaceDetectorService
participant Controller as AppController
participant Accessibility as ScrollAccessibilityService
participant System as Android System
User->>Camera: Looks up/down
Camera->>Camera: Capture frame
Camera->>ML: Send ImageProxy
ML->>ML: Detect face landmarks
ML->>ML: Calculate Euler angles
ML->>ML: Determine behavior (UP/DOWN/CENTER)
ML->>Controller: Emit FaceBehavior
Controller->>Accessibility: Call scrollUp()/scrollDown()
Accessibility->>System: Perform scroll action
System->>User: Content scrolls
- Enable the service after granting permissions
- Position yourself in front of the camera
- Look up to scroll up
- Look down to scroll down
- Look straight ahead to stop scrolling
The app includes configurable settings for:
- Sensitivity thresholds for eye movement detection
- Scroll speed and behavior
- Camera settings and optimization
- FaceDetectorService: Handles ML Kit face detection and behavior analysis
- CameraXService: Manages camera lifecycle and frame processing
- ScrollAccessibilityService: Provides system-wide scrolling functionality
- AppController: Orchestrates all services and manages app state
The app follows Clean Architecture principles with:
- Presentation Layer: Jetpack Compose UI with ViewModels
- Domain Layer: Business logic and use cases
- Data Layer: Repository pattern with Room database
- Service Layer: Background services for camera and accessibility
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
- Google ML Kit for face detection capabilities
- Android Accessibility Services for system integration
- CameraX team for camera functionality
- Jetpack Compose for modern UI development
If you encounter any issues or have questions:
- Open an issue on GitHub
- Check the documentation
- Review the troubleshooting guide
FluentRead - Making reading more accessible, one eye movement at a time. 👁️📱