It's Not Skynet, and It's Not Reading Your Mind

What ChatGPT, Claude, and Gemini actually are, in plain English. No jargon, no hype, no doom.

Two questions about AI chatbots have come up over and over with my friends and family this year, and on the surface they sound like opposites.

The first: is this thing going to take over the world? Usually delivered half as a joke, usually with a Terminator reference somewhere in it, and usually not entirely as a joke.

The second: how does it know that about me? Someone opens a brand new conversation with Gemini, asks something ordinary, and it casually mentions their kids, or their job, or an interest they never brought up in that chat. That question doesn’t arrive with a joke attached. It arrives with a slightly unsettled pause.

Those two questions come from the same place. Nobody has ever explained what is actually sitting on the other side of that text box, so people fill the gap with the only reference material widely available: about forty years of movies in which a computer wakes up one day and decides it hates us.

The real explanation is stranger than the movie version in a couple of places and much more boring in all the ways that matter. It’s also worth understanding, because the mistakes people actually make with these tools are almost never the mistakes they’re bracing for.

I work with this technology every day. What follows is the explanation I keep giving at dinner tables, cleaned up a little.

It’s a prediction machine, and that isn’t an insult

Your phone already does a tiny version of this. Start typing a text message and it offers you the next word before you get there. Type “I’ll be there in five” and it suggests “minutes.” Nobody thinks their phone is conscious. It has just seen enough sentences to know what usually comes next.

Take that same basic idea, feed it a truly enormous amount of human writing (books, articles, forums, documentation, most of the public internet), and give it billions of internal dials to tune while it practices. What you end up with is a system that has become extraordinarily good at one narrow thing: given everything written so far, predict what comes next.

That’s it. That is the engine. When you ask ChatGPT a question, it isn’t looking up an answer in a database or consulting a fact file. It’s generating a response one small piece at a time, each piece chosen because it’s a highly probable continuation of everything that came before it.

Here’s the part that trips people up, and honestly it should. If that’s all it’s doing, why does it feel like it understands anything?

Because of what it takes to get good at that game. To reliably predict the next word in a physics explanation, a legal summary, a recipe, or a working piece of software, a system has to internalize an enormous amount of structure about how those things actually work. You cannot fake your way through the next word in a proof by vibes alone. Somewhere in the process of getting very good at prediction, these systems developed internal machinery that behaves an awful lot like reasoning.

So it isn’t thinking the way you think. But calling it “just autocomplete” undersells it in a way that will leave you unprepared for how capable it actually is.

The thing that would have to be true for Skynet

Now the fun part.

Every science fiction version of this story requires one ingredient that is completely missing here: the machine has to want something. Skynet decides it wants to survive. HAL wants to complete the mission. The plot only works if there’s something in there with goals of its own, sitting quietly, forming intentions.

These systems don’t want anything. Not in a hand-wavy philosophical sense, but in a concrete, mechanical one. Here’s the fact I find lands hardest at dinner tables:

It isn’t running when you’re not talking to it.

There is no chatbot sitting in a data center right now, thinking about your last conversation, getting bored, forming plans. When you hit send, a system wakes up, reads the entire conversation from the beginning, produces a response, and then effectively stops existing until your next message. Between your messages, there is nothing having an experience. There isn’t a continuous “it” in there having a day.

That’s not a limitation anyone forgot to fix. It’s the shape of the technology. What people picture as a mind biding its time is closer to a very sophisticated calculator that only computes when you press the button, and forgets it ever ran the moment it finishes.

Which brings us neatly to the other question, because the memory thing is where people’s intuitions go most badly wrong.

So how did it know about my daughter?

If it forgets everything between messages, how did Gemini bring up your family in a brand new chat?

Because somebody handed it a briefing before the conversation started. There are two completely different ways that happens, the products blur them together, and the difference matters enormously.

The first is what you told it. ChatGPT keeps a list of things it has decided are worth remembering about you, which you can look at and edit. It also, since 2025, quietly references your earlier conversations as background context. So a thing you mentioned in March can surface in a conversation in August, without you connecting the two. OpenAI named the background process that synthesizes all this “Dreaming,” which I have to say is not doing anyone’s nerves any favors, but the mechanism is mundane: it reads your old chats and writes itself notes.

The second is what you connected it to. This is the Gemini one, and it’s the one that produces the genuinely startling moments. Google offers a feature that lets Gemini pull context from your other Google products: Gmail, Photos, YouTube history, Search history. Turn that on and you have handed it a window into years of your life that it can quietly consult before answering. It doesn’t need to have learned about your daughter’s soccer team. It can simply read the email about it.

The two arrive with different defaults, and the difference is worth knowing. Reading your other Google apps is opt-in: somebody has to switch it on. Remembering your past conversations generally is not. On the ordinary consumer accounts most people have, chat memory is already on, which is precisely why the surprise usually comes from that direction.

Both have off switches. They live in settings, under something like memory or personalization, and they’re worth ten minutes of your afternoon whether you end up leaving them on or not.

The reason this matters isn’t that either feature is sinister. It’s that “the machine mysteriously knows things about me” and “I clicked yes on a permission screen eight months ago” feel like completely different situations, and only one of them is true. Once you know which, the unsettled feeling usually resolves into something more useful: an informed decision about how much of your life you want it to see.

Where the real risk actually is

I want to be careful here, because there’s a version of this article that pats you on the head and tells you there’s nothing to worry about, and that version would be dishonest.

The instinct that something here deserves caution is correct. People who wave off every concern about AI as hysteria are wrong too, and they tend to be wrong in more expensive ways. The problem with the Skynet framing isn’t that it’s worried. It’s that it’s aiming at the wrong target, and while everyone watches the sky for killer robots, the actual failure modes are sitting right in front of them, being extremely boring.

Here are the three that will genuinely affect you.

It will be wrong with total confidence. This is the big one. Because it’s optimized to produce plausible text, it produces plausible text, whether or not the underlying claim is true. It does not have a little internal signal that fires when it doesn’t know something. Ask it about a medication interaction, a legal deadline, or a specific person’s biography, and you may get a fluent, well-organized, completely fabricated answer delivered in exactly the same confident tone as a correct one. It’s the acquaintance who gives you flawless, detailed driving directions to a restaurant that closed four years ago. The confidence is not evidence.

Anything you paste is out of your hands. Whatever you type goes to a company’s servers. Depending on the product and your settings, it may be retained, reviewed by humans for quality, or used to improve future systems. That’s fine for “help me word this birthday message.” It’s a genuinely bad idea for your medical history, your social security number, a contract you’re under NDA about, or your employer’s internal documents.

The subtle one: you’ll start trusting it. This is the risk almost nobody braces for, because it doesn’t feel like a risk. It feels like convenience. After the tool is right forty times in a row, you stop checking the forty-first, and the forty-first is the one about your kid’s medication dosage. The danger was never that it decides to hurt you. It’s that it’s wrong in the ordinary way software is sometimes wrong, at a moment when you’d stopped looking.

How to actually use one of these well

None of that means don’t use them. I use them constantly, professionally, for real work with real consequences. It means using them the way you’d use a very well-read, very fast, occasionally overconfident assistant who has never once said “I’m not sure.”

Treat the output as a first draft, not a final answer. These tools are outstanding at getting you to eighty percent in thirty seconds. Explaining something in simpler terms, drafting an awkward email, summarizing a long document, giving you a starting point on a topic you know nothing about. That’s real value. Just don’t confuse the draft with the verdict.

Verify anything with a consequence attached. Medical, legal, financial, or any factual claim about a real person. If being wrong would cost you money, health, or a relationship, check it against a real source. Ask the tool for its sources and then actually visit them, because a fabricated citation looks exactly like a real one right up until you click it.

Don’t paste anything you wouldn’t email to a stranger. Simple rule, no judgment call required in the moment.

Notice what it’s best at. It’s strongest on things you can verify yourself or things where there’s no single right answer: rewriting, brainstorming, explaining, organizing your own thoughts back to you. It’s weakest exactly where you can’t check it and it matters most.

Go look at your settings once. Find the memory and personalization controls in whatever tool you use, see what’s turned on, and make an actual decision instead of inheriting a default you never chose.

Back to the two questions

So: is it going to take over the world? Not this. Not the thing in your pocket predicting the next word in a sentence. There are serious, thoughtful people working on the risks of far more capable future systems, and that work matters. But the chatbot helping you plan a dinner menu is not biding its time, because between your messages there is no “it” there to bide anything.

And how did it know about your family? Because you told it, or because you gave it a key to the place where you’d already written it down. Both of those are choices, both are listed in a menu, and both can be undone this afternoon.

The reason I want people to have this explanation isn’t to make them comfortable. It’s that fear and awe are equally bad ways to handle a tool. Fear makes people avoid something genuinely useful; awe makes them believe the confident, fluent, completely wrong answer. Knowing roughly what the thing is protects you from both.

None of this requires a technical background. It just requires somebody telling you plainly, which mostly hasn’t happened. (If you do want the engineer’s version, with the actual machinery under the hood, I wrote that one too.)

It isn’t plotting. It isn’t psychic. It’s doing something far less dramatic and far more useful: guessing the next word, remarkably well, and only ever when you speak to it first.