~/dimslaev main โ€บ ls writing/
about
software engineer @ infomaniak
focus
llms ยท agents ยท rag ยท data pipelines
stack
typescript ยท node.js ยท python
tools
claude code ยท opencode ยท tmux
now
building agentique.ch, a platform for ai news
links
github ยท linkedin
writing/
~/dimslaev main โ€บ cat writing/distilling-a-filter.md
cd ..

distilling a filter so the model runs less

2026-07-10 ยท note

per-item llm calls stop making sense the moment ingestion volume is real. every run was paying for a model to look at things that were obviously junk.

so i distilled the keep/drop decision. the llm had already produced thousands of labelled examples just by doing its job; a small static classifier trained on those agrees with it closely enough to be the high-recall first pass.

the expensive model now only sees what survives the cheap filter. same quality bar, a fraction of the calls. the agreement rate between the two is also a number i can watch โ€” free evaluation signal.