Who--or what--are the Numerati?

They're members of a global elite, and are busy analyzing our every move. They're rummaging through mountains of data, looking for patterns of our behavior so that they can predict what we might want to buy, who we're likely to vote for, what job we'd do better than our colleagues. Some are even matching us with potential lovers. The Numerati are masters of symbolic realm. They're great at math and computer science. The Googleplex is crawling with Numerati. So is IBM.

When I started this book, I thought that the Numerati were different from the rest of us, that they were “numbers people.” As I watched them studying our shopping and voting patterns, I saw this wasn’t true. They analyze data much the way all of us do every day. Let’s say a friend asks you for a $100 loan. You immediately begin working through data and probabilities. How much does she earn? Does she lie more than most people? What are the chances she’ll move? What will she do if you say no? You give different weight to each of these variables. That’s exactly what the Numerati do. In a sense, we’re all Numerati. But they deal with millions of us at the same time and they use big machines.

Data collection is old hat. Why are things different now?

Imagine that a detective or a biographer wanted to piece together a year of your life. Say 1991. For this, he might have to climb up into your attic and dig through boxes of letters, big folders of snap shots, telephone bills, all sorts of paper. He might have to interview friends, neighbors and co-workers. Our histories existed largely on paper and in foggy memories. That has changed. Our photos and correspondence, and practically everything we do at the office now travels as digital data. With this shift, we now deliver our details in a single standard made of ones and zeros. Who can make sense of all that data, turning it into new insights about us, new services, new industries? Only the Numerati.

This can be frustrating for those of us who studied humanities. There used to be a pretty clear divide. The math types stuck to engineering and science and architecture--and they left the study of humans to us. Those of us who dropped Calc 101 could still rise high in psychology, journalism, law and marketing. But now the Numerati are storming into the humanities.

Not that we’re standing still. We’ll increasingly be using their tools and methods on our own behalf. This might mean scouting out the most reliable heart surgeons in our area, drawing up resumes that pop to the top of search results, or installing software to crunch our jogging data and put us on the optimal regime for the Boston Marathon.

Is this a business book?

No. It’s about us. It's about how the Numerati are deciphering our behavior, and how this will change our world.

Don't get me wrong. This is a crucial trend in business. The Numerati are remaking entire industries, starting with advertising and media. This book began as a BusinessWeek cover story, "Math Will Rock Your World." Lots of the Numerati we get to know in this book are working for businesses, including Yahoo, Accenture and Chemistry.com. But the focus is not on the business, and how they'll make money. It's on how the Numerati are taking control of our lives, at work, at the mall, at the doctor's office.

Do the Numerati make mistakes?

All the time. They rely on statistics and probability, and sometimes they get it wrong. Just one piece of data can topple the most brilliant mathematical analysis. That happened to me; it kept Chemistry.com, when we tested the system, from lining me up with my wife. One danger is that people will accept the analyses of the Numerati, because they’re delivered with the certainty of science. I hope that people who read this book will be in a better position to understand what goes into these conclusions—and to refute them if necessary.

Should we be scared of the Numerati?

Let's say vigilant. They have unprecedented power to uncover our secrets. And their predictions, produced by algorithms, will have a lot to say about whether we get a job, how much we spend for health insurance, even if we'll get swept up as a terrorism suspect.

Here's what we have to keep in mind. The Numerati are not always right. They work with statistics, often delivering stunning results. A grocer, for example, will be thrilled if 60% of targeted shoppers go for a promotion on filet mignon. It won't matter if a few of those getting coupons are vegans or devout Hindus. Shift the focus from shopping to something like brain cancer or homeland security. Then the errors--what the Numerati call "false positives"--start to become a very big deal.

So, we don't want them to misread us. At the same time, we don't want them to know and predict us too well. That would feel a bit like Big Brother, which is a danger.

What was the most frightening thing you discovered while writing this book?

The possibility of a police state. The Numerati are crunching our consumer and demographic data to predict what kind of voters we are, and whether we're likely to buy Hummers. Fair enough. But let's say a government committed to law and order goes through the consumer records of convicted pedophiles and builds a statistical profile of a child molester. Are civil libertarians going to fight this? Whose side are they on, they'll be asked, children's or sex criminals? That's a hard battle to fight.

Then let's say the authorities match this profile against that of every teacher in the country. It wouldn't be hard to do. It turns out the guy teaching eighth-grade English at the local school has a 43% chance of being a pedophile. What then? Should they fire him? Tell the parents? Are they legally liable if they do nothing and he later commits a crime? This opens a big can of worms and potentially undermines our legal presumption of innocence.

The most inspiring?

That would be David Heckerman. He's a researcher at Microsoft, and also an M.D. He was doing research on filtering e-mail for spam. It was a moving target, because the spammers kept tweaking their e-mails to get past filters. It was almost as if that spam mutated. It occurred to him one day that maybe his spam-hunting algorithm could anticipate the mutations of HIV, the virus that leads to AIDs. That's one thing that's really cool about the Numerati. They can jump from one discipline to the next, often riding the same algorithms. And that's what Heckerman did. He moved his Microsoft team into HIV research. While they haven't developed an effective vaccine yet, they're making progress.

How are the Numerati affecting this election?

Microtargeting is the rage this election year. It's based on the statistical analysis of every conceivable piece of our data, from our subscription to Wired or HBO to the number of school-age children we have living at home. The big campaign push, and most of the money, is still in mass market, things like TV ads. But if either candidate can use microtargeting to zero in on a few thousand voters in crucial districts, it could spell the difference in the election.

Which chapter was the most fun to write?

I loved reporting the medicine chapter. It took me out to Oregon, where Intel researchers have wired the homes of elderly people with countless sensors. They have cameras, motion detectors, microphones that pick up changes in the voice, even sensors under the floor to detect shifts in balance. Some of it sounds outrageous, but I think many of us are going to be living our golden years under this type of surveillance, all of our patterns analyzed statistically. Sometimes this can lead to misunderstandings. The Intel people had this one woman's bed wired to monitor her weight. One night it seemed she gained seven or eight pounds. Was she taking on liquids? Congestive heart failure? It turned out her little dog had jumped on the bed. Another false positive.

How did your experience as a journalist prepare you to write this book?

Well, it certainly didn't teach me any math. In fact, I wrote the book for people like me--those of us who tend to close any book that has formulas or Greek letters running across the pages...

I'd say my experience as foreign correspondent helped out. I spent years in places like Mexico, France and Venezuela, and I viewed my job as anthropological--bridging cultural and linguistic divides. I approached the Numerati as a foreign culture, one that had its own lingo and world view. One of the keys of this kind of reporting, I should add, is a readiness to ask dumb questions. I'm quite good at that.

What are the biggest misconceptions about the Numerati?

Did you ever hear the joke about the extrovert at the math party? He was the one looking at someone else's shoes.

We have this idea that there are word people and numbers people, and that the numbers people lack social skills. That's not true. In fact, we're all both word and numbers people. And when it came to communication, most of the Numerati I met were highly articulate in English, even when it was their second or third language.

Was there a spark moment that inspired you to write this book?

I was at IBM Research, talking to Samer Takriti, a Syrian-born mathematician. He was telling me about his project to build mathematical models of 50,000 of his colleagues. He explained that he and his team had all of this data now, and that they could use it to create these simulations of workers. One day, they would be able to predict how productive each one would be, which ones would work best together, which ones were worth training.

That afternoon I drove home from IBM's Watson lab, about 40 miles north of New York. I remember replaying Takriti's words as I crossed the Tappan Zee Bridge and thinking: If they can model us as workers, then others can model us as patients, shoppers, voters... This is the mathematical modeling of humanity! That was when the book popped into focus.

Were you and your wife surprised by the results of your online matchmaking experiment?

I was surprised that my wife put up with it. She was deeply reluctant to sign up for Chemistry.com and not happy at all that I did it so enthusiastically. Initially, we were a both perplexed that the system didn't match us. Sure, we were both Explorer/Negotiators, and we were told that we might be too similar. They have this idea that opposites attract. But still, they were trying to set her up with some guy who lived on the far side of JFK Airport, which is about an hour and a half away from our house. And I wasn't even showing up in her results, even though I lived in the same town (and the same house). Eventually, we learned about that small error on my part that was keeping us apart. We both felt better in the end.

Which area of our lives are the Numerati transforming most quickly?

They're racing ahead in shopping, marketing, advertising and media. Look at Google. It's revolutionizing entire industries (including my own) by applying mathematics and computer science--the tools of the Numerati--on the world of information. It's numbers colonizing words.

When they get to the final page of The Numerati, what do you hope readers will be thinking and feeling?

I hope they start seeing (or imagining) the Numerati at work everywhere they look, whether it's at school, at work, at the hospital or the grocery store. That’s what has happened to me. I’m acutely aware of the data that I’m sending out into the world, whether it’s driving through an EZ Pass on the Garden State Parkway or checking the baseball scores on my PC at work. Hmm, I wonder. What conclusions will they draw from that? If readers of the book start thinking this way, perhaps they'll analyze the patterns of their own lives and the data they produce. For many of us, it's a new way to learn about ourselves. And like it or not, it's the way the rest of the world will get to know us.