Test Plan
Introduction
This test plan covers MVP requirements (36 functional requirements, 25 non-functional requirements).
Test Strategy
Test Levels
Level |
Scope |
Tools |
|---|---|---|
Unit |
Individual functions |
Rust |
Integration |
Component interactions |
Rust integration tests |
System |
End-to-end workflows |
Manual testing |
Features Not Tested
Feature |
Reason |
|---|---|
whisper.cpp internals |
Third-party software |
OS audio APIs |
Platform dependency |
SQLite engine |
Third-party software |
Test Automation Approach
MVP: Manual testing only. Automation deferred until post-MVP stabilization.
Unit tests (automatable):
Component |
What to Test |
Mock Strategy |
|---|---|---|
db.rs |
CRUD operations |
In-memory SQLite ( |
whisper.rs |
Command building, output parsing |
Mock |
audio.rs |
WAV file format validation |
Pre-recorded test files |
Integration tests (partially automatable):
Test |
Automation Notes |
|---|---|
Database + file storage |
Use temp directories |
Settings persistence |
In-memory database |
Path validation |
Unit testable |
System tests (manual only):
Test |
Why Manual |
|---|---|
Tray icon states |
Requires visual verification |
Global hotkeys |
Requires OS-level interaction |
Audio recording |
Requires microphone hardware |
Notifications |
Platform-specific behavior |
Mock whisper.cpp: For CI, create a simple binary that echoes test input:
#!/bin/bash
# mock-whisper.sh - Returns predictable output for testing
echo "This is a test transcription."
Test Cases by Category
Recording Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Hotkey Starts Recording |
high |
||
Hotkey Stops Recording |
high |
||
Tray Icon Recording State |
high |
||
Default Microphone Capture |
high |
||
WAV File Format Validation |
high |
||
Tray Menu Shows Recording Duration |
medium |
||
Recording Duration Limit |
medium |
||
Custom Audio Device |
medium |
Traceable Test Cases (sphinx-needs directives):
Steps:
Expected: Tray icon shows recording state, audio buffer initializes |
Steps:
Expected: WAV file saved, transcription begins |
Steps:
Expected: Icon changes color/appearance to indicate recording state |
Steps:
Expected: Audio captured from correct device |
Steps:
Expected: 16kHz mono WAV format |
Transcription Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Transcription Performance |
medium |
||
Large Transcription Handling |
medium |
||
Whisper.cpp Invocation |
high |
||
Transcription Text Extraction |
high |
||
Transcription Notification |
high |
||
Transcription Progress Icon |
medium |
||
Transcription Error Notification |
high |
Steps:
Expected: whisper.cpp binary executed with correct model path |
Steps:
Expected: Text correctly parsed from whisper.cpp stdout |
Steps:
Expected: Notification shows preview of transcribed text |
Steps:
Expected: Icon indicates transcription in progress |
Steps:
Expected: Error notification displayed with helpful message |
Verify tray menu displays recording duration while active |
Verify recording stops at configured duration limit |
Verify custom audio input device selection works correctly |
History Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Transcription Saved to Database |
high |
||
Tray Menu Shows Recent Items |
high |
||
Menu Items Show Timestamp and Preview |
medium |
||
Click Menu Item Copies to Clipboard |
high |
||
View History Dialog |
medium |
||
Delete Transcription |
medium |
Verify all transcriptions are saved to SQLite database |
Verify tray menu displays recent transcriptions (newest first) |
Verify menu items display timestamp and text preview |
Verify clicking menu item copies full transcription to clipboard |
Verify “View History” opens native dialog with full list |
Verify transcription can be deleted from history dialog |
Output Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Hotkey Copies Last Transcription |
high |
||
Export to Text File |
medium |
Verify global hotkey copies most recent transcription to clipboard |
Verify transcription can be exported as .txt file from history dialog |
Settings Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Settings Dialog Usability |
medium |
||
Settings Dialog Opens |
high |
||
Configure Whisper Path |
high |
||
Configure Model Path |
high |
||
Path Validation |
high |
||
Configure Hotkeys |
high |
||
First-Run Prompt |
medium |
Verify “Settings” menu item opens native settings dialog |
Verify whisper.cpp binary path can be configured with file picker |
Verify model file path can be configured with file picker |
Verify paths are validated to exist before save |
Verify global hotkeys can be configured with conflict detection |
Verify first-run prompt appears when paths not configured |
System Tray Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Tray Icon State Indication |
high |
||
Platform-Specific Tray Support |
high |
||
Tray Icon Shows Status |
high |
||
Left-Click Toggles Recording |
high |
||
Right-Click Shows Menu |
high |
||
App Starts Minimized to Tray |
high |
Verify tray icon shows app status (idle/recording/transcribing) |
Verify left-click on tray icon toggles recording |
Verify right-click shows menu with Recent items, Settings, Quit |
Verify app starts minimized to tray (no main window) |
Security Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Privacy Guarantee Verification |
critical |
||
Optional Database Encryption |
low |
||
Path Traversal Rejected |
critical |
||
SQL Injection Prevention |
critical |
||
No Network Calls (Firewall Test) |
critical |
||
No Network Calls (Packet Capture) |
critical |
||
App Works Offline |
critical |
Verify file paths with |
Verify parameterized queries prevent SQL injection |
Verify no network calls using firewall monitoring |
Verify no network calls using packet capture analysis |
Verify app is 100% functional offline (airplane mode test) |
Deployment Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
Cargo Vendor Directory Present |
high |
||
Build Succeeds Without Internet |
critical |
||
Single-Directory Deployment |
medium |
||
Build on Air-Gapped System |
high |
Verify cargo vendor directory is present for offline build |
Verify build process works without internet after initial setup |
Verify single-directory deployment works (app + whisper.cpp + model) |
Non-Functional Tests
ID |
Title |
Tests |
Priority |
|---|---|---|---|
App Launch Time |
high |
||
Memory Footprint |
high |
||
Build on Air-Gapped System |
high |
||
System Theme Support |
medium |
||
Privacy Guarantee Verification |
critical |
||
Offline Functionality Test |
critical |
||
Transcription Performance |
medium |
||
Audio File Integrity Check |
high |
||
Database Transaction Safety |
high |
||
Whisper.cpp Crash Handling |
high |
||
Hotkey Registration Recovery |
medium |
||
Notification Clarity |
high |
||
Tray Icon State Indication |
high |
||
Settings Dialog Usability |
medium |
||
First-Time Setup Experience |
medium |
||
Test Coverage Verification |
high |
||
API Documentation Completeness |
high |
||
Clippy Compliance Check |
high |
||
Code Formatting Check |
high |
||
Platform-Specific Tray Support |
high |
||
Audio Device Compatibility |
high |
||
Database Growth Performance |
medium |
||
Long Audio Recording Support |
medium |
||
Large Transcription Handling |
medium |
||
Optional Database Encryption |
low |
Verify app launches in < 2 seconds |
Verify memory usage < 100 MB (excluding whisper.cpp) |
Verify build succeeds on air-gapped system with no internet |
Verify app follows system theme (dark/light) |
Verify no network calls under any circumstance (monitor with network sniffer). Note Overlaps with TC-SEC-003 (firewall test) and TC-SEC-004 (packet capture). Consider consolidating into a single network isolation test suite. |
Verify 100% functionality with network disconnected. Note Overlaps with TC-SEC-005 (airplane mode test). Consider consolidating into a single offline verification procedure. |
Verify transcription completes within 2x recording duration |
Verify WAV file integrity verified before transcription |
Verify database operations use transactions preventing data loss |
Verify whisper.cpp crashes handled gracefully without audio loss |
Verify detection and recovery from hotkey registration failures |
Verify notifications include clear status and transcription preview |
Verify tray icon clearly indicates state (idle/recording/transcribing) |
Verify settings dialog provides clear labels and validation feedback |
Verify first-time users guided to configure whisper.cpp path |
Verify codebase achieves ≥80% test coverage via cargo tarpaulin |
Verify all public APIs have rustdoc documentation |
Verify cargo clippy passes with zero warnings |
Verify code formatted with rustfmt (cargo fmt –check) |
Verify native tray icons on macOS, Windows, Linux with GNOME AppIndicator |
Verify compatibility with standard audio input devices across platforms |
Verify performance with 10,000+ transcriptions without degradation |
Verify support for audio recordings up to 120 minutes |
Verify handling of transcriptions with 50,000+ characters |
Verify optional SQLite encryption for sensitive transcriptions (if implemented) |
Test Procedures
Network Isolation Test (TC-SEC-003)
Preconditions:
App installed
Firewall configured to block all app connections
Steps:
Enable firewall blocking for app
Launch app
Record audio (30 seconds)
Stop and transcribe
View history
Check firewall logs
Pass Criteria: Zero blocked connection attempts in logs.
Offline Operation Test (TC-SEC-005)
Preconditions:
App installed with configured whisper.cpp
Network disconnected (airplane mode)
Steps:
Disconnect network
Launch app
Record → transcribe → copy
Verify all features work
Pass Criteria: All operations complete successfully.
Air-Gap Build Test (TC-DEP-002)
Preconditions:
Fresh clone of repository with vendored dependencies
Network disconnected (airplane mode or air-gapped VM)
Steps:
Clone repository to air-gapped system
Run
cargo build --release --offlineVerify binary is produced
Pass Criteria: Build completes successfully with no network access.
Pass/Fail Criteria
All Critical tests must pass before release
All High priority tests must pass before release
Medium priority tests: 90% pass rate acceptable