291 lines
13 KiB
Markdown
291 lines
13 KiB
Markdown
# keydr - Terminal Typing Tutor Architecture Plan
|
|
|
|
## Context
|
|
|
|
**Problem**: No terminal-based typing tutor exists that combines keybr.com's adaptive learning algorithm (gradual letter unlocking, per-key confidence tracking, phonetic pseudo-word generation) with code syntax training. Existing tools either lack adaptive learning entirely (ttyper, smassh, typr) or have incomplete implementations (gokeybr intentionally ignores error stats, ivan-volnov/keybr is focused on Anki integration).
|
|
|
|
**Goal**: Build a full-featured Rust TUI typing tutor that clones keybr.com's core algorithm, extends it to code syntax training, and provides a polished statistics dashboard - all in the terminal.
|
|
|
|
---
|
|
|
|
## Research Summary
|
|
|
|
### keybr.com Algorithm (from reading source: `packages/keybr-lesson/lib/guided.ts`, `keybr-phonetic-model/lib/phoneticmodel.ts`, `keybr-result/lib/keystats.ts`)
|
|
|
|
**Letter Unlocking**: Letters sorted by frequency. Starts with minimum 6. New letter unlocked only when ALL included keys have `confidence >= 1.0`. Weakest key (lowest confidence) gets "focused" - drills bias heavily toward it.
|
|
|
|
**Confidence Model**: `confidence = target_time_ms / filtered_time_to_type`, where `target_time_ms = 60000 / target_speed_cpm` (default target: 175 CPM ~ 35 WPM). `filtered_time_to_type` is an exponential moving average (alpha=0.1) of raw per-key typing times.
|
|
|
|
**Phonetic Word Generation**: Markov chain transition table maps character bigrams to next-character probability distributions. Chain is walked with a `Filter` that restricts to unlocked characters only. Focused letter gets prefix biasing - the generator searches for chain states containing the focused letter and starts from there. Words are 3-10 chars; space probability boosted by `1.3^word_length` to keep words short.
|
|
|
|
**Scoring**: `score = (speed_cpm * complexity) / (errors + 1) * (length / 50)`
|
|
|
|
**Learning Rate**: Polynomial regression (degree 1-3 based on sample count) on last 30 per-key time samples, with R^2 threshold of 0.5 for meaningful predictions.
|
|
|
|
### Key Insights from Prior Art
|
|
|
|
- **gokeybr**: Trigram-based scoring with `frequency * effort(speed)` is a good complementary approach. Its Bellman-Ford shortest-path for drill generation is clever but complex.
|
|
- **ttyper**: Clean Rust/Ratatui architecture to reference. Uses `crossterm` events, `State::Test | State::Results` enum, `Config` from TOML. Dependencies: `ratatui ^0.25`, `crossterm ^0.27`, `clap`, `serde`, `toml`, `rand`, `rust-embed`.
|
|
- **keybr-code**: Uses PEG grammars to generate code snippets for 12+ languages. Each grammar produces realistic syntax patterns.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
### Technology Stack
|
|
- **TUI**: Ratatui + Crossterm (the standard Rust TUI stack, battle-tested by ttyper and many others)
|
|
- **CLI**: Clap (derive)
|
|
- **Serialization**: Serde + serde_json + toml
|
|
- **HTTP**: Reqwest (blocking, for GitHub API)
|
|
- **Persistence**: JSON files via `dirs` crate (XDG paths)
|
|
- **Embedded Assets**: rust-embed
|
|
- **Error Handling**: anyhow + thiserror
|
|
- **Time**: chrono
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
src/
|
|
main.rs # CLI parsing, terminal init, main event loop
|
|
app.rs # App state machine (TEA pattern), message dispatch
|
|
event.rs # Crossterm event polling thread -> AppMessage channel
|
|
config.rs # Config loading (~/.config/keydr/config.toml)
|
|
|
|
engine/
|
|
mod.rs
|
|
letter_unlock.rs # Letter ordering, unlock logic, focus selection
|
|
key_stats.rs # Per-key EMA, confidence, best-time tracking
|
|
scoring.rs # Lesson score formula, gamification (levels, streaks)
|
|
learning_rate.rs # Polynomial regression for speed prediction
|
|
filter.rs # Active character set filter
|
|
|
|
generator/
|
|
mod.rs # TextGenerator trait
|
|
phonetic.rs # Markov chain pseudo-word generator
|
|
transition_table.rs # Binary transition table (de)serialization
|
|
code_syntax.rs # PEG grammar interpreter for code snippets
|
|
passage.rs # Book passage loading
|
|
github_code.rs # GitHub API code fetching + caching
|
|
|
|
session/
|
|
mod.rs
|
|
lesson.rs # LessonState: target text, cursor, timing
|
|
input.rs # Keystroke processing, match/mismatch, backspace
|
|
result.rs # LessonResult computation from raw events
|
|
|
|
store/
|
|
mod.rs # StorageBackend trait
|
|
json_store.rs # JSON file persistence with atomic writes
|
|
schema.rs # Serializable data models
|
|
|
|
ui/
|
|
mod.rs # Root render dispatcher
|
|
theme.rs # Theme TOML parsing, color resolution
|
|
layout.rs # Responsive screen layout (ratatui Rect splitting)
|
|
components/
|
|
mod.rs
|
|
typing_area.rs # Main typing widget (correct/incorrect/pending coloring)
|
|
stats_sidebar.rs # Live WPM, accuracy, key confidence bars
|
|
keyboard_diagram.rs # Visual keyboard with finger colors + focus highlight
|
|
progress_bar.rs # Letter unlock progress
|
|
chart.rs # WPM-over-time line charts (ratatui Chart widget)
|
|
menu.rs # Mode selection menu
|
|
dashboard.rs # Post-lesson results view
|
|
stats_dashboard.rs # Historical statistics with graphs
|
|
|
|
keyboard/
|
|
mod.rs
|
|
layout.rs # KeyboardLayout, key positions, finger assignments
|
|
finger.rs # Finger enum, hand assignment
|
|
|
|
assets/
|
|
models/en.bin # Pre-built English phonetic transition table
|
|
themes/*.toml # Built-in themes (catppuccin, dracula, gruvbox, nord, etc.)
|
|
grammars/*.toml # Code syntax grammars (rust, python, js, go, etc.)
|
|
layouts/*.toml # Keyboard layouts (qwerty, dvorak, colemak)
|
|
```
|
|
|
|
### Core Data Flow
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ Event Loop │
|
|
└──────┬──────┘
|
|
│ AppMessage
|
|
▼
|
|
┌──────────┐ ┌─────────────────┐ ┌───────────┐
|
|
│Generator │────▶│ App State │────▶│ UI Layer │
|
|
│(phonetic,│ │ (TEA pattern) │ │ (ratatui) │
|
|
│ code, │ │ │ │ │
|
|
│ passage) │ │ ┌─────────────┐ │ └───────────┘
|
|
└──────────┘ │ │ Engine │ │
|
|
│ │ (key_stats, │ │ ┌───────────┐
|
|
│ │ unlock, │ │────▶│ Store │
|
|
│ │ scoring) │ │ │ (JSON) │
|
|
│ └─────────────┘ │ └───────────┘
|
|
└─────────────────┘
|
|
```
|
|
|
|
### App State Machine
|
|
|
|
```
|
|
Start → Menu
|
|
Menu → Lesson (on mode select)
|
|
Menu → StatsDashboard (on 's')
|
|
Menu → Settings (on 'c')
|
|
Lesson → LessonResult (on completion or ESC)
|
|
LessonResult → Lesson (on 'r' retry)
|
|
LessonResult → Menu (on 'q'/ESC)
|
|
LessonResult → StatsDashboard (on 's')
|
|
StatsDashboard → Menu (on ESC)
|
|
Settings → Menu (on ESC, saves config)
|
|
Any → Quit (on Ctrl+C)
|
|
```
|
|
|
|
### The Adaptive Algorithm
|
|
|
|
**Step 1 - Letter Order**: English frequency order: `e t a o i n s h r d l c u m w f g y p b v k j x q z`
|
|
|
|
**Step 2 - Unlock Logic** (after each lesson):
|
|
```
|
|
min_letters = 6
|
|
for each letter in frequency_order:
|
|
if included.len() < min_letters:
|
|
include(letter)
|
|
elif all included keys have confidence >= 1.0:
|
|
include(letter)
|
|
else:
|
|
break
|
|
```
|
|
|
|
**Step 3 - Focus Selection**:
|
|
```
|
|
focused = included_keys
|
|
.filter(|k| k.confidence < 1.0)
|
|
.min_by(|a, b| a.confidence.cmp(&b.confidence))
|
|
```
|
|
|
|
**Step 4 - Stats Update** (per key, after each lesson):
|
|
```
|
|
alpha = 0.1
|
|
stat.filtered_time = alpha * new_time + (1 - alpha) * stat.filtered_time
|
|
stat.best_time = min(stat.best_time, stat.filtered_time)
|
|
stat.confidence = (60000.0 / target_speed_cpm) / stat.filtered_time
|
|
```
|
|
|
|
**Step 5 - Text Generation Biasing**:
|
|
- Only allow characters in the unlocked set (Filter)
|
|
- When a focused letter exists, find Markov chain prefixes containing it and start words from those prefixes
|
|
- This naturally creates words heavy in the weak letter
|
|
|
|
### Code Syntax Extension
|
|
|
|
After all 26 prose letters are unlocked, the system transitions to code syntax training:
|
|
- Introduces code-relevant characters: `{ } [ ] ( ) < > ; : . , = + - * / & | ! ? _ " ' # @ \ ~ ^ %`
|
|
- Uses PEG grammars per language to generate realistic code snippets
|
|
- Gradual character unlocking continues for syntax characters
|
|
- Users select their target programming languages in config
|
|
|
|
### Theme System
|
|
|
|
Themes are TOML files with semantic color names:
|
|
```toml
|
|
[colors]
|
|
bg = "#1e1e2e"
|
|
text_correct = "#a6e3a1"
|
|
text_incorrect = "#f38ba8"
|
|
text_pending = "#585b70"
|
|
text_cursor_bg = "#f5e0dc"
|
|
focused_key = "#f9e2af"
|
|
# ... etc
|
|
```
|
|
|
|
Resolution order: CLI flag → config → user themes dir → bundled → default fallback.
|
|
|
|
Built-in themes: Catppuccin Mocha, Catppuccin Latte, Dracula, Gruvbox Dark, Nord, Tokyo Night, Solarized Dark, One Dark, plus an ANSI-safe default.
|
|
|
|
### Persistence
|
|
|
|
JSON files in `~/.local/share/keydr/`:
|
|
- `key_stats.json` - Per-key EMA, confidence, sample history
|
|
- `lesson_history.json` - Last 500 lesson results
|
|
- `profile.json` - Unlock state, settings, gamification data
|
|
|
|
Atomic writes (temp file → fsync → rename) to prevent corruption. Schema version field for forward-compatible migrations.
|
|
|
|
---
|
|
|
|
## Implementation Phases
|
|
|
|
### Phase 1: Foundation (Core Loop + Basic Typing)
|
|
Create the terminal init/restore with crossterm, event polling thread, TEA-based App state machine, basic typing against a hardcoded word list with correct/incorrect coloring.
|
|
|
|
**Key files**: `main.rs`, `app.rs`, `event.rs`, `session/lesson.rs`, `session/input.rs`, `ui/components/typing_area.rs`, `ui/layout.rs`
|
|
|
|
### Phase 2: Adaptive Engine + Statistics
|
|
Implement per-key stats (EMA, confidence), letter unlocking, focus selection, scoring, live stats sidebar, and progress bar.
|
|
|
|
**Key files**: `engine/key_stats.rs`, `engine/letter_unlock.rs`, `engine/scoring.rs`, `engine/filter.rs`, `session/result.rs`, `ui/components/stats_sidebar.rs`, `ui/components/progress_bar.rs`
|
|
|
|
### Phase 3: Phonetic Text Generation
|
|
Build the English transition table (offline tool or build script), implement the Markov chain walker with filter and focus biasing, integrate with the lesson system.
|
|
|
|
**Key files**: `generator/transition_table.rs`, `generator/phonetic.rs`, `generator/mod.rs`, a `build.rs` or `tools/` script for table generation
|
|
|
|
### Phase 4: Persistence + Theming
|
|
JSON storage backend, atomic writes, config loading, theme parsing, built-in theme files, apply themes throughout all UI components.
|
|
|
|
**Key files**: `store/json_store.rs`, `store/schema.rs`, `config.rs`, `ui/theme.rs`, `assets/themes/*.toml`
|
|
|
|
### Phase 5: Results + Dashboard
|
|
Post-lesson results screen, historical stats dashboard with charts (ratatui Chart widget), learning rate prediction.
|
|
|
|
**Key files**: `ui/components/dashboard.rs`, `ui/components/stats_dashboard.rs`, `ui/components/chart.rs`, `engine/learning_rate.rs`
|
|
|
|
### Phase 6: Code Practice + Passages
|
|
PEG grammar interpreter for code syntax generation, book passage mode, GitHub code fetching + caching.
|
|
|
|
**Key files**: `generator/code_syntax.rs`, `generator/passage.rs`, `generator/github_code.rs`, `assets/grammars/*.toml`
|
|
|
|
### Phase 7: Keyboard Diagram + Layouts
|
|
Visual keyboard widget with finger color coding, multiple layout support (QWERTY, Dvorak, Colemak).
|
|
|
|
**Key files**: `keyboard/layout.rs`, `keyboard/finger.rs`, `ui/components/keyboard_diagram.rs`, `assets/layouts/*.toml`
|
|
|
|
### Phase 8: Polish + Gamification
|
|
Level system, streaks, badges, CLI completeness, error handling, performance, testing, documentation.
|
|
|
|
---
|
|
|
|
## Verification
|
|
|
|
After each phase, verify by:
|
|
1. `cargo build` compiles without errors
|
|
2. `cargo test` passes all unit tests
|
|
3. Manual testing: launch `cargo run`, exercise the new features, verify UI rendering
|
|
4. For Phase 2+: verify letter unlocking by typing accurately and watching new letters appear
|
|
5. For Phase 3+: verify generated words only contain unlocked letters and bias toward the focused key
|
|
6. For Phase 4+: verify stats persist across app restarts
|
|
7. For Phase 5+: verify charts render correctly with historical data
|
|
|
|
---
|
|
|
|
## Dependencies (Cargo.toml)
|
|
|
|
```toml
|
|
[dependencies]
|
|
ratatui = "0.30"
|
|
crossterm = "0.28"
|
|
clap = { version = "4.5", features = ["derive"] }
|
|
serde = { version = "1.0", features = ["derive"] }
|
|
serde_json = "1.0"
|
|
toml = "0.8"
|
|
rand = { version = "0.8", features = ["small_rng"] }
|
|
reqwest = { version = "0.12", features = ["json", "blocking"] }
|
|
dirs = "6.0"
|
|
rust-embed = "8.5"
|
|
chrono = { version = "0.4", features = ["serde"] }
|
|
anyhow = "1.0"
|
|
thiserror = "2.0"
|
|
```
|