Bisect provider-rejected batches in triage and deep assessment
DeepSeek's content filter rejects a whole request (400 "Content Exists Risk") when one article trips it, which cost the other articles in the batch their assessment and retried them every run. New curate/batch.rs runs both stages through a bisecting runner: a rejected batch is split until the offending article is isolated, that article is retried once on the editor provider when it is a different one, and a still-rejected article is recorded as a provider_rejected assessment row so it is not retried for assessment_reuse_days. Cache reuse accepts rows from either configured model. Each stage logs reused/requested/rejected counts, the curation: line shows rejections when non-zero, explain prints them, and the llm_assess span reports the deep-set size. Implemented by a Claude agent from an orchestrator brief; verified fmt/clippy(-W dead_code)/test green (354 lib tests). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A1rCLQeKBgnBo3oTgHuTMe
This commit is contained in:
@@ -416,8 +416,8 @@ pub fn decode_blob(bytes: &[u8], dimension: usize) -> Result<Vec<f32>, Embedding
|
||||
});
|
||||
}
|
||||
let mut vector = Vec::with_capacity(dimension);
|
||||
for chunk in bytes.chunks_exact(4) {
|
||||
let value = f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]);
|
||||
for chunk in bytes.as_chunks::<4>().0 {
|
||||
let value = f32::from_le_bytes(*chunk);
|
||||
if !value.is_finite() {
|
||||
return Err(EmbeddingError::NonFinite);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user