Vokal 🎙️

A production-ready voice bot testing and interaction framework with streaming Speech-to-Text, Text-to-Speech, and AI-powered evaluation

✨ What is Vokal?

Vokal is a comprehensive TypeScript framework for building, testing, and evaluating voice-based applications. It provides a provider-agnostic architecture for Speech-to-Text, Text-to-Speech, and AI-powered evaluation services. Currently supports Google Cloud providers (Speech-to-Text, Text-to-Speech via Neurolink SDK, and Gemini AI), with an extensible design that allows for additional provider integrations.

Perfect for:

• 🤖 Testing voice bots and conversational AI
• 📞 IVR (Interactive Voice Response) system validation
• 🎯 Voice UI/UX testing and evaluation
• 🔊 Speech synthesis and recognition workflows
• 🧪 Automated voice conversation testing

🚀 Key Features

Voice Services

• 🎤 Text-to-Speech (TTS) - High-quality neural speech synthesis via Neurolink SDK
• 🎧 Streaming Speech-to-Text - Real-time audio transcription with voice activity detection
• 🗣️ Voice Interaction Pipeline - Complete TTS → Listen → STT conversation flows
• 🎵 Background Audio Mixing - Realistic test environments (office, cafe, rain, nature, phone, crowd)

Testing & Evaluation

• 🤖 AI-Powered Evaluation - Semantic response validation using Google Gemini
• 📊 Comprehensive Test Suites - JSON-based test configuration with detailed reporting
• 🔄 Automatic Retries - Built-in retry logic with exponential backoff
• 📈 Performance Metrics - Pass rates, confidence scores, and detailed analytics

Developer Experience

• 📘 Full TypeScript Support - Complete type safety with strict mode
• 🛡️ Security First - Input validation, sanitization, and secure credential handling
• 🔧 Easy Configuration - JSON-based configuration with sensible defaults
• 📦 Modular Architecture - Use individual services or the complete framework
• 🖥️ Powerful CLI - Command-line interface for all operations

📋 Table of Contents

• Installation
• Quick Start
• Core Services
• CLI Usage
• Configuration
• Architecture
• Background Sounds
• Contributing

🔧 Installation

Prerequisites

node -v  # Should be 20.x or higher
pnpm -v  # Should be 9.x or higher (npm or yarn also work)

Install Vokal

pnpm add @juspay/vokal

Or clone and build from source:

git clone https://github.com/juspay/vokal.git
cd vokal
pnpm install
pnpm run build

Set Up Credentials

Create a .env file in your project root:

# Option 1: Service Account (Recommended - Full Features)
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

# Option 2: API Key (Limited Features)
GOOGLE_AI_API_KEY=your_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here

💡 Tip: Service account authentication provides access to advanced features like configurable VAD timeouts and enhanced STT capabilities.

🎯 Quick Start

1. Simple Text-to-Speech

import { createVoiceTest } from '@juspay/vokal';

const voiceTest = createVoiceTest();

// Generate and save speech
const audioPath = await voiceTest.generateSpeech({
  text: "Welcome to Vokal! Your voice testing framework.",
  languageCode: 'en-US',
  voiceName: 'en-US-Neural2-F'
});

console.log('Audio saved to:', audioPath);

2. Voice Interaction with Background Audio

import { VoiceInteractionService } from '@juspay/vokal';

const voiceBot = new VoiceInteractionService();

// Run complete voice interaction
const result = await voiceBot.runVoiceInteraction(
  "What is your name?",
  {
    language: 'en-US',
    voice: 'en-US-Neural2-D',
    backgroundSound: 'office',
    backgroundVolume: 0.15
  }
);

console.log('User said:', result.transcript);
console.log('Confidence:', result.confidence);

3. Automated Voice Bot Testing

import { VoiceBotTestService } from '@juspay/vokal';

// Run test suite from configuration
const testService = VoiceBotTestService.create('./test-config.json');
const results = await testService.runTestSuite();

console.log(`✅ Pass Rate: ${results.summary.passRate}%`);
console.log(`📊 Average Score: ${results.summary.averageScore}`);
console.log(`📁 Results: ${results.summary.resultsFile}`);

🎯 Core Services

Service	Description	Use Case
VoiceTestService	Text-to-Speech with background audio via Neurolink	Generate test audio with realistic environments
VoiceInteractionService	Complete TTS + Listen + STT pipeline	Full conversation simulation
VoiceBotTestService	Automated test suite execution	Test multiple scenarios with AI evaluation
AIComparisonService	AI-powered response evaluation	Semantic answer validation using Gemini
AudioMixerService	Background audio mixing	Add realistic noise to test scenarios
AudioRecordingService	Microphone recording via naudiodon	Capture user responses
STTHandlerManager	Provider-agnostic STT management	Unified interface for multiple STT providers

🖥️ CLI Usage

Vokal includes a comprehensive command-line interface:

Generate Speech

# Basic TTS generation
vokal voice generate "Hello, world!" --voice en-US-Neural2-F --lang en-US

# With background audio
vokal voice generate "Welcome" --voice en-US-Neural2-D --lang en-US --bg cafe --bgvol 0.2 --play

# Advanced settings
vokal voice generate "Fast speech" --voice en-US-Neural2-A --rate 1.5 --pitch 5.0 --output speech.mp3

List Available Voices

# List all voices
vokal voices

# Filter by language
vokal voices en-US

# JSON output
vokal voices en-IN --format json

Background Sounds

# List available background sounds
vokal backgrounds

Test Audio Playback

# Test system audio capability
vokal test-audio

# Play an audio file
vokal play ./output.wav

Run Voice Bot Tests

# Create sample configuration
vokal test --save-sample

# Run test suite
vokal test ./config.json

# Run with specific provider and debug mode
vokal test --provider google-ai --debug --verbose

Show Examples

# Display comprehensive usage examples
vokal example

Run vokal --help for complete CLI documentation.

⚙️ Configuration

Test Suite Configuration

Create a JSON file to define your test scenarios:

{
  "metadata": {
    "name": "My Voice Bot Tests",
    "version": "1.0.0",
    "description": "Voice bot test suite"
  },
  "settings": {
    "defaultLanguage": "en-US",
    "defaultVoice": "en-US-Neural2-D",
    "recordingDuration": 10000,
    "passingScore": 0.7,
    "sttProvider": "google-ai",
    "ttsProvider": "google-ai",
    "aiProvider": "google-ai",
    "vadSettings": {
      "silenceThreshold": 0.02,
      "silenceDuration": 2000,
      "speechTimeout": 10000
    }
  },
  "questions": [
    {
      "id": "greeting",
      "question": "Hello! How can I help you?",
      "intent": "User greets and asks for help",
      "expectedElements": ["Greeting", "Request for assistance"],
      "sampleResponse": "Hi, I need help with my account"
    }
  ]
}

See the examples/sample-config.json for a complete example.

🏗️ Architecture

Vokal is built with a provider-agnostic architecture using the Handler pattern for extensibility.

Current Provider Support

Google Cloud (Default)

• TTS: Google Cloud Text-to-Speech via Neurolink SDK
• STT: Google Cloud Speech-to-Text via GoogleAISTTHandler
• AI Evaluation: Google Gemini via AIComparisonService

Project Structure

vokal/
├── src/
│   ├── services/          # Core voice services
│   │   ├── voice-test.ts           # TTS service with Neurolink
│   │   ├── voice-interaction.ts    # Complete pipeline orchestration
│   │   ├── voice-bot-test.ts       # Test suite execution
│   │   ├── ai-comparison.ts        # AI-powered evaluation
│   │   ├── audio-mixer.ts          # Background audio processing
│   │   └── audio-recording.ts      # Microphone capture
│   ├── providers/         # Provider implementations
│   │   ├── google-ai-stt.handler.ts    # Google Cloud STT
│   │   ├── stt-handler-manager.ts      # Provider manager
│   │   └── stt-registry.ts             # Provider registry
│   ├── types/             # TypeScript type definitions
│   ├── utils/             # Utilities (logging, retry, validation, security)
│   ├── constants/         # Audio configuration constants
│   ├── errors/            # Custom error classes
│   └── cli/               # Command-line interface
├── examples/              # Example configurations
│   ├── sample-config.json          # Test suite example
│   ├── basic-example.js            # Basic usage template
│   └── stt-handler-example.ts      # STT provider example
├── assets/                # Background audio files
│   ├── office-ambience.wav
│   ├── cafe-ambience.wav
│   ├── nature-sounds.wav
│   ├── rain-light.wav
│   ├── phone-static.wav
│   └── crowd-distant.wav
├── memory-bank/           # AI assistant context
└── docs/                  # Documentation (coming soon)

Provider Architecture

// Handler pattern for provider abstraction
interface STTHandler {
  startStreaming(config, onResult, onSpeechStart, onSpeechEnd, onError);
  stopStreaming();
}

// Register providers
STTHandlerManager.registerHandler('google-ai', GoogleAISTTHandler);

// Get provider instance
const handler = STTHandlerManager.getHandler('google-ai');

🎵 Background Sounds

Available background sound presets for realistic test environments:

Sound	Description	Recommended Volume	Use Case
office	Office ambience with typing and quiet chatter	0.15	Business applications, productivity bots
cafe	Coffee shop atmosphere with ambient noise	0.20	Customer service, casual conversations
nature	Outdoor setting with birds and gentle wind	0.18	Wellness apps, meditation guides
rain	Gentle rainfall ambience	0.12	Calming applications, sleep aids
phone	Phone line static and connection noise	0.08	IVR testing, call center simulations
crowd	Distant crowd noise and murmurs	0.10	Public space simulations, event apps

All audio files are located in the assets/ directory as WAV files.

🛡️ Security

Vokal follows security best practices:

• ✅ Input validation and sanitization via validation.ts
• ✅ Secure credential handling (no hardcoded secrets)
• ✅ Command injection prevention in secure-exec.ts
• ✅ Safe file path handling with path resolution
• ✅ API key validation
• ✅ Spawn-based command execution (no shell injection)

🧪 Testing

# Build the project
pnpm run build

# Run linting
pnpm run lint

# Format code
pnpm run format

# Type checking
pnpm run typecheck

📦 NPM Scripts

Script	Description
pnpm run build	Build TypeScript to JavaScript (dist/)
pnpm run dev	Build in watch mode
pnpm run clean	Clean build directory
pnpm run lint	Lint code with ESLint
pnpm run format	Format code with Prettier
pnpm run typecheck	Run TypeScript type checking
pnpm run prebuild	Format and lint before build

🤝 Contributing

Contributions are welcome! Please read our Contributing Guide for details.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes using Conventional Commits
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

See CODE_OF_CONDUCT.md for community guidelines.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• @juspay/neurolink - TTS generation and audio synthesis
• Google Cloud Speech-to-Text - Streaming STT with voice activity detection
• Google Gemini - AI-powered semantic evaluation
• naudiodon - Native audio I/O for Node.js

📞 Support

• 📖 Documentation: Coming soon in /docs
• 💬 Issues: GitHub Issues
• 📧 Email: opensource@juspay.in

---

Made with ❤️ by the Breeze Team