Feed
Actually, AI
How AI actually works — the stories, the science, and what it means for you. Three episodes per topic: understand it, go deep, use it.
Claude Opus 4 scored 90% on MMLU—but that number probably tells you nothing about whether it'll actually clean up your spreadsheet.
When researchers reanalyzed 5,700 MMLU questions in 2024, they discovered 57% of virology answers were marked wrong—flipping one model from 4th place to 1st when errors were corrected.
GPT-4 scored in the ninetieth percentile on the bar exam—until MIT recalculated and found it was actually forty-second percentile against passing lawyers, exposing how benchmarks can be mathematically gamed.
Every prompt you send has a real cost—output tokens cost 2-10x more than input tokens because of how inference actually works in the data center.
Claude Opus 4 can process your entire 3,000-token prompt in parallel, but generating the next word requires reading 140 gigabytes of weights from memory—prefill and decode are fighting a completely different battle.
When you hit send on a question to an AI, billions of mathematical operations explode across specialized processors in real-time—and the words streaming back aren't retrieved, they're constructed one prediction at a time, each choice reshaping what comes next.
Claude Opus 4 processes a million tokens unevenly—edges sharp, middle foggy—so pasting your entire codebase into a chat makes AI worse, not better.
Claude Opus 4 processes 128,000 tokens simultaneously, but here's the problem: transformers have no idea what order they're in—until researchers solved it with sine waves and a memory trick that now consumes 40GB per conversation.
When you paste a 20-page contract into Claude and ask about page fourteen, it confidently returns an answer that's subtly wrong—not because it forgot, but because the math of attention weights the middle of your input half as much as the edges.
Your words aren't search queries—they're steering signals applied at every denoising step, and understanding this changes how you prompt diffusion models forever.
Jascha Sohl-Dickstein, a physicist studying how ink disperses in water, realized a neural network could reverse thermodynamics itself—and accidentally invented the technology behind every AI image generator you use today.
When you ask an AI to draw a person holding coffee, six fingers appear and the menu becomes gibberish—not because the model forgot, but because image generators don't create pictures, they *denoise* them from pure static, step by step.
Human labelers ranked responses, a reward model learned their preferences, and now your chatbot's overly polite refusals and suspiciously agreeable tone all trace back to that invisible training process—but defaults can be overridden if you know where to look.
OpenAI sent tens of thousands of texts to Kenyan workers in 2021—they read the internet's worst content for hours daily to teach ChatGPT what to refuse.
When Claude refuses to help with a fictional crime scene but cheerfully explains how to synthesize dangerous chemicals, the culprit is RLHF—the four-letter process that transforms raw language models into the assistants you use daily, complete with all their quirks and blind spots.
When you pick the wrong AI model for a task, you're essentially driving a semi truck to buy milk—it works, but costs forty dollars in diesel and twenty minutes to park.
Jared Kaplan's 2020 power law promised unlimited scaling—then the curve bent, and the entire $2 trillion bet changed overnight.
When Jared Kaplan plotted a physicist's equation across seven orders of magnitude, he discovered that spending a billion dollars on bigger AI models followed a curve so predictable it reshaped the entire industry—but nobody knows if it holds one more step.
Employees spend 3.5 hours per week searching for files that exist somewhere in their company—but a search for "relocating the UK team" finds your memo about "moving the London office" anyway, thanks to embeddings.
Word2Vec has two opposite architectures—one predicts missing words from context, the other predicts context from a word—and this is how your chatbot understands document search, why Netflix recommends videos you didn't know existed, and what "vector" actually means.
When you search "best restaurant near me" and get results about "top dining spots," the search engine isn't matching keywords—it's measuring distance between invisible points in a 500-dimensional space called an embedding.
When you ask your AI coworker a factual question, it will confidently lie to you and sound exactly like it's telling the truth—here's how to know when to actually verify its answer.
When an AI confidently cites a court case that never happened, is it lying—or just doing its job? Inside the mathematical reason why bigger models sometimes bullshit harder.
Steven Schwartz asked ChatGPT for legal citations in 2023 and got six perfectly formatted cases—none of which existed, and the AI confidently confirmed they were real when he double-checked.
When you paste instructions at the top of a long prompt, they get drowned out by thousands of other tokens competing for the model's attention—here's how to fix it.
In 2017, eight researchers published a 15-page paper that became the blueprint for every AI you use today—and Dzmitry Bahdanau's frustration with machine translation three years earlier made it possible.
In 2017, eight Google researchers published "Attention Is All You Need"—a paper titled like a Beatles joke that revolutionized AI by letting machines read your entire message at once instead of forgetting earlier words.
Claude's training ended in early 2025—ask it about last Tuesday and it'll confidently confabulate an answer based on patterns from before that cutoff date.
Geoffrey Hinton spent 14 years convincing the world that backpropagation works—now every hallucination and repetitive chatbot loop traces back to valleys he carved in the loss landscape.
In 1.3 trillion words, a frozen snapshot learns everything about Shakespeare to Reddit—but nothing about last week, and that's the entire problem.
Seventy billion parameters sounds impressive until you realize it tells you nothing about whether a model can actually handle your specific task—here's what really matters when choosing an AI tool.
Walter Pitts hid from bullies in a Detroit library in 1935, found Principia Mathematica, read all three volumes in three days, and spotted errors that impressed Bertrand Russell—launching the unlikely path to modern neural networks.
Frank Rosenblatt built a machine in 1958 that could learn from examples—then a jealous rival's theorem convinced the world it was impossible, killing neural networks for a decade.
When you rephrase "pls summarize this txt" to "please summarize this text about machine learning," you're not just being more formal—you're sending an entirely different sequence of numbers to the AI that determines its response.
Byte Pair Encoding builds a vocabulary in 50,000 rounds: find the most common adjacent pair, merge it into one token, repeat—the simple algorithm that determines how every modern AI reads your text.
Modern AI models fail at simple tasks like counting R's in "strawberry" because they never actually see your words—a 1994 compression algorithm chops your text into invisible fragments called tokens before the neural network even begins.