Stop Scoring. Start Ranking. If you’ve asked an LLM to rate a document on a scale of 1 to 10, you might have noticed something strange. The scores cluster together, most responses land between 3 and 7, and the sorted list you end up with is nearly useless. LLMs
Most developers adding tool access to their agents just pick one and move on. They add a few MCP servers, notice things feel slow or weird, and assume it's the model. It's not the model. The choice between using CLIs or MCPs has a direct, measurable
Here is a situation that comes up more than it should. A company has an AI workflow running in production. Maybe it’s extracting information from construction defect reports, or answering client questions from a structured knowledge base. The output isn’t quite right yet, so they bring in a
Last week I asked Gemini CLI for a Product Requirements Document (PRD). Then got back a generic spec. It was confident, nicely formatted... yet utterly USELESS. In fact, I'd fixed this problem before. A few days ago, I'd already created a skill defining how to write
The weird thing about the internet is that it’s only possible because of one weird maths fact. That is; some operations are expensive to do, but cheap to check. Factoring a huge number into two primes is hard, but multiplying those primes back together and verifying you got the
In an internal experiment, outputs using logit bias were judged "less AI-like" than the baseline 74.5% of the time over 353 blind A/B evaluations. Logit bias is a vocabulary knob. You push specific tokens down (ban) or up (boost) during generation. It works on tokens, not
A PASS/FAIL eval isn’t actionable unless it tells you WHY the judge evaluated it as such. Ensure every binary LLM-as-judge evaluation returns REASONING and EVIDENCE alongside the DECISION. RED Framework = (Reasoning + Evidence + Decision) This allows you to identify failures in seconds and improve your prompts and models faster
I help builders ship production AI features that stay reliable, safe, and maintainable. Read about the practices behind non-deterministic systems, including evals, RAG, routing, and fine-tuning.
