How Watson Actually Works

Kids ask me this constantly, so let's actually answer it.

Watson is a real, physical robot. Right now that means a screen with an animated face, mounted on a stand, no full robot body yet. Inside, there's a small computer, not connected to the internet at all. Whatever Watson knows either came loaded in from the start, or I taught it to him myself.

Here's the part that surprises people: Watson doesn't "know" things the way a person does. He runs on something called a large language model. Picture reading an enormous number of books and articles, so many that you start noticing patterns in how words and ideas usually fit together. That's roughly what a language model does, except at a scale no person could actually manage. It's not reading with understanding the way you do. It's predicting, based on patterns, what's likely to come next.

That's actually useful for a lot of things. It's also exactly why Watson sometimes gets things wrong. If I haven't taught him something specific, he'll occasionally still take a guess, based on patterns that are close but not quite right. Picture putting together a puzzle using only pieces you've seen before, and the puzzle in front of you has a shape you've never run into. You might jam a close-but-wrong piece in there because it's the best fit you've got. That's basically what happens when Watson makes a mistake.

My job, then, is simple: catch it, correct it, and add the right information for next time. Not that different from teaching any student.

What's actually inside him

Watson runs on a small single-board computer, small enough to fit in one hand, with three main pieces doing the real work.

A CPU, the general-purpose processor, handles the everyday stuff: running his operating system, managing his voice output, keeping track of what's happening in a lesson.

A dedicated NPU, or neural processing unit, is the part that actually powers his "thinking." A CPU can technically run a language model, but painfully slowly. An NPU is a chip built specifically to handle the kind of math language models need, huge numbers of small calculations happening at once, so Watson can respond in real time instead of everyone sitting around waiting on him.

Then there's RAM, the memory the system uses while it's actively working. Watson runs on 8GB of it, enough to hold his language model and keep it running smoothly without constantly digging back to storage for information.

There's also a text-to-speech system, the part that turns his written responses into an actual voice. He doesn't have a recording of a person saying every possible sentence he might need, that would be impossible. Instead, the system breaks a response down into individual sounds and rebuilds them into speech on the fly, all running locally, same as everything else.

None of it is connected to the internet, and none of it is a repurposed laptop or an old desktop. It's small, specialized hardware, built for exactly one job: running an offline AI system reliably in a classroom.

Why he doesn't have a body yet

People tend to assume the body comes first and the thinking gets added in afterward. I did it the other way around, on purpose. Getting the actual thinking right, the part that understands language, holds a real conversation, and reacts to what's happening in a lesson, is the genuinely hard problem. Motors, wheels, arms, that's real engineering too, but it's a far more solved problem at this point than language understanding is. Get the brain working reliably first, and everything added to the body later is just movement bolted onto something that already works, instead of debugging two brand new systems at the same time.

Two more things worth knowing about him, since kids ask about these constantly too:

  • He tracks his own exact age, down to the day, and mentions it whether or not anyone asked

  • He has a genuine weakness for quoting Sherlock Holmes and Star Trek, whether or not it's relevant to what's happening

Want to actually meet him? Read more on his page here.

Next
Next

Why We Started Teaching Critical Thinking at Four