# Remove Drill History Cap

## Context

The app currently truncates persisted `drill_history` to the most recent 500 drills in both:

- `finish_drill()`
- `finish_partial_drill()`

This is implemented in `src/app.rs` by removing the oldest history entry once `drill_history.len() > 500`.

The original motivation for revisiting this came from the sticky-mastery work, where replay correctness becomes more important. But this history-cap issue is broader than mastery and should be handled as a separate task.

## Current Findings

### 1. The cap affects persisted data, not just UI

`lesson_history.json` only retains the most recent 500 entries because the in-memory history is pruned before save.

### 2. Full-history replay already exists in some paths

The app already walks all retained `drill_history` in several places:

- startup: `App::new()` calls `rebuild_ngram_stats()`
- import: imported `drill_history` is followed by `rebuild_ngram_stats()`
- delete-history-entry: `rebuild_from_history()` replays remaining history, then calls `rebuild_ngram_stats()`

So removing the cap is not just a data-retention change. It also changes the amount of work done by:

- startup
- import
- delete-history rebuilds

### 3. Saves rewrite the full history file

After each completed or partial drill, `save_data()` writes the entire `drill_history` back to `lesson_history.json`.

That means uncapping history will likely increase:

- per-drill save latency
- JSON serialization cost
- disk usage

## Questions To Resolve Later

1. Should history become fully uncapped immediately, or should there be a configurable retention policy?
2. Is JSON blob storage still acceptable for very large histories, or should history move to a more append-friendly format?
3. Should startup continue rebuilding n-gram state from full history on every launch, or should the app persist derived caches?
4. Is delete-history-entry expected to remain a full rebuild operation, or should that workflow be redesigned for large histories?
5. Do we want the exported data format to always include full history, even if local persistence later changes format?

## Likely Scope

At minimum, this future plan will need to cover:

### Data retention behavior

- remove the 500-entry truncation logic in `src/app.rs`
- verify `save_drill_history()` persists full history
- update stale comments/docs that still describe history as capped

### Performance analysis

- startup replay cost from `rebuild_ngram_stats()`
- import cost
- delete-history rebuild cost
- save latency from rewriting the full file every drill

### Possible optimization directions

- keep full history but persist derived n-gram caches
- keep full history but move storage away from one monolithic JSON blob
- keep full history but make rebuilds incremental where possible

## Non-Goals For This Stub

- choosing the final storage redesign now
- changing history persistence in the mastery plan
- making performance guarantees before measuring realistic history sizes

## Files Likely Involved

- `src/app.rs`
- `src/store/json_store.rs`
- `src/store/schema.rs`
- `docs/plans/2026-02-09-initial-plan.md`
- `docs/plans/2026-02-22-n-gram-error-tracking-adaptive-drill-selection.md`

## Acceptance Criteria For The Future Task

To be defined in the full plan after deeper investigation, but likely to include:

1. users retain full drill history
2. startup performance remains acceptable
3. per-drill save latency remains acceptable
4. replay-derived features remain correct with large histories
