Here is a situation that comes up more than it should. A company has an AI workflow running in production. Maybe it’s extracting information from construction defect reports, or answering client questions from a structured knowledge base. The output isn’t quite right yet, so they bring in a
Last week I asked Gemini CLI for a Product Requirements Document (PRD). Then got back a generic spec. It was confident, nicely formatted... yet utterly USELESS. In fact, I'd fixed this problem before. A few days ago, I'd already created a skill defining how to write
The weird thing about the internet is that it’s only possible because of one weird maths fact. That is; some operations are expensive to do, but cheap to check. Factoring a huge number into two primes is hard, but multiplying those primes back together and verifying you got the
In an internal experiment, outputs using logit bias were judged "less AI-like" than the baseline 74.5% of the time over 353 blind A/B evaluations. Logit bias is a vocabulary knob. You push specific tokens down (ban) or up (boost) during generation. It works on tokens, not
A PASS/FAIL eval isn’t actionable unless it tells you WHY the judge evaluated it as such. Ensure every binary LLM-as-judge evaluation returns REASONING and EVIDENCE alongside the DECISION. RED Framework = (Reasoning + Evidence + Decision) This allows you to identify failures in seconds and improve your prompts and models faster