Home - posts tagged as News
Why Nate Silver is never wrong
November 8, 2012News
|Election day was a big one for the Numerati. Rayid Ghani, who I profiled in the "Shopper" chapter, was President Obama's chief data scientist. He moved from modeling people's preferences in food to their politics, and apparently pulled it off brilliantly.
Then there's Nate Silver. I interviewed him onstage at South by Southwest in 2009. He represented the rise of statistical methods in politics. Now everyone knows about it, and him. Actually, he was a huge star at that 2009 event. There were a couple thousand people in the auditorium, and hundreds of others in spill-over rooms watching it on TV. But then he was a more of a star for politicos and geeks. Now he's mainstream.
As I mention in the headline, he's never wrong. And this is something that the gut-level pundits can't understand. He's never wrong, because he doesn't call elections, he calculates probabilities. He wrote on the Election Day eve that Romney had a 10% chance of winning. In other words, if he ran his statistical model through 100 scenarios, Romney would come out on top in 10 of them.
Silver made his first big splash in baseball analysis, where this type of thinking is older than the spitball. Romney, in essense, was coming to the plate with the chances of a miserable hitter, someone like Sandy Koufax. The Dodger lefty was one of the greatest pitchers ever, but a sorry hitter. His lifetime batting average was .097. An easy out. But still, 10% of the time his grounders found a way through the infield, his bloops fell in. Twice in his career, once in 1962 and again a year later, he actually hit a home run. So Romney on Tuesday was essentially Koufax at bat. Likely to lose, but not certain to. If Romney had won, Nate Silver would not have been wrong.
People are going to have to get used to this type of statistical thinking, because we're going to be besieged by probabilities. In medicine, for example, we're going to hear that a certain condition has a 10% or 14% chance of killing us within 10 years or 15 years. It's not mortal, but it could be. Before, we didn't have the numbers to play the odds. Now we will.
The rise of the Numerati threatens pundits and ruminators in the worlds of politics and sports. They make money by sitting around and speculating about events that could go either way. Some, like Joe Scarborough, feel threatened by the statisticians. He should, because statisticians like Silver make it clear just how empty his talk really is.
What Dylan can teach Jonah Lehrer (and the rest of us)
|So it turns out that Jonah Lehrer fabricated some Bob Dylan lines for his best-seller, Imagine: How Creativity Works. I feel sad for the whole team at Houghton Mifflin, my publisher and his. They worked madly on his book and helped turn it into a best-seller, only to see it self destruct. (You can't even get the link for it on Amazon anymore.) I also feel sorry for Jonah Lehrer. He must be pinching himself, hoping it's just a nightmare.
Just a few minutes ago, I was humming Like a Rolling Stone, the song at the center of the Bob Dylan section in the book, and it occurred to me that the lyrics could almost have been written for Lehrer. It's directed toward a young, rich and privileged person (or generation) who "once upon a time dressed so fine, threw the bums a dime," etc. It goes on:
"People'd call, say, "Beware doll, you're bound to fall"
You thought they were all kiddin' you..."
Of course, there's a fall. It seems disastrous. But falls can lead to growth, and to liberation. Becoming unmoored--a rolling stone--can be exhilarating. In the beginning of the song, Dylan sings with contempt, which is what Lehrer is on the receiving end of today. But by the end, the singer is in cheerleader mode. The fall is an opportunity.
...you got nothing, you got nothing to lose
You're invisible now, you got no secrets to conceal.
September 12, 2011News
|Just a reminder that the Watson Jeopardy matches, originally broadcast in February, will be on TV again today through Wednesday in North American markets. For those of you who read the book following the broadcast, this is a chance to see the machine that you've come to know! Here's a podcast I did with Bloomberg BusinessWeek's Rachael King in anicipation of Watson's return.
Watson: a patent goldmine?
|Steve Lohr writes in the NYT about the booming market for tech patents. Clearly, the smart phone is turning out to be a war zone for intellectual property. This would be expected, since the smart phone holds the keys to the biggest consumer tech market in the world. Five years from now, perhaps two billion people will have smart phones, and many of them will be replacing their handsets every year or two.
But it's not just size. The smart phone is a stitching together of many existing technologies. And each time a researcher figures out how to run, say, the movie camera and the ebook reader off the same segment of the battery, or how to make sure that a swipe across the screen answers the phone without interfering with the email you're writing, there's a patent to file. The smart phone is the epitome of technology consolidation--and every connection in it requires innovation, and a patent.
As I read the article, I was thinking, as I often do, about IBM's Watson. Like a smart phone, Watson congregates lots of disparate technologies. It has natural language recognition, information retrieval, multiple layers of analytics, even advanced betting strategies. All of it is stitched together so that it can develop confidence in answers within three seconds.
As I write in Final Jeopardy, it's no sure bet that Watson and its kin will be answering loads of our questions in five or ten years. But if the Watson family isn't, other machines will be. Question-answering will be a mammoth market. (Many of the services, of course, will be accessed through smart phones.) And even if Watson doesn't prevail, I'm betting that the intellectual property in Watson--the patents--will pay off royally.
"Corporations are People": an op-ed
|Here's a piece I wrote for the L.A. Times after reading Mitt Romney's comment (below).
"Corporations are people, my friend." Mitt Romney, Aug. 11, campaigning in Iowa
I've been thinking about people
I know. Apple comes to mind. It's
terrific at electronics and has a great eye for design. It's far richer
than everyone in my neighborhood. Another person
I've known for years is General Motors
. It almost died awhile back. Enron, which passed away in 2001, wasn't an especially nice person
, or an honest one. But people
will be people
You might find this hard to believe, but these people —
with CEOs and corporate headquarters — need money more than we do. For
them, money is like oxygen or food. If they come up short, they get
eaten or die. So you really can't blame them for laying us off, slashing
our medical benefits or moving our jobs offshore. They do it to
survive. On the other hand, you shouldn't take them too seriously when
they talk about, say, clean air or early childhood nutrition. If
anything threatens to cost them money, they tend to line up against it,
because every dollar they lose leads these people
one step closer to extinction.
Now, there was a time that the government was considered a person, or at
least "the people." It looked out for the population's welfare and
education. It built roads and chipped in for retirement. This turned out
to be an enormous and expensive job, and before long we began hearing
that government was not the solution but the problem. Its taxes and
regulations got in the way. If it didn't back off, people
like GE and Exxon Mobil
would take their jobs elsewhere. Government was not the people, it was a thing — a bad thing, at least for certain people.
Fortunately, government is run by politics, and politics run on the very stuff that these people
built to amass: money. So they can weigh in, advertising their agenda
and investing in politicians who see things their way. Their argument is
simple: If they aren't free to make lots of money, they'll go somewhere
else where they can, leaving the rest of us in bad shape. One good way
for them to make money is to shrink this thing — the government — that
has its hand in their pockets and always seems to be writing irksome and
expensive new rules.
Money is so important to these people
, sad to say, that it can
cloud their judgment. A decade ago, a number of them chased the illusion
of ever-expanding wealth and ended up making a boatload of bad bets.
Two of these folks, Bear Stearns and Merrill Lynch, were eaten. A third,
Lehman Bros., died. Others faced grave danger and threatened to carry
the economy down with them. So the people
held their noses and
asked the despised government — that thing — to save them. The
government borrowed trillions of dollars to do just that and avert
economic collapse. Yet that rescue appeared to make the government even
much more dangerous. It now was deeper in debt, which meant it might
have to raise taxes on the very people it rescued.
Fortunately, these concerned people
got relief from the courts. Early last year, the Supreme Court ruled that everyone — all kinds of people —
benefit from the 1st Amendment right to free speech, which included
giving unlimited and secret campaign contributions. Given this
opportunity, some people
had more money to give to politicians
than the rest of us. And naturally, the first Congress elected following
this ruling tended to its patrons' interests. Who wouldn't? The
politicians promptly carried the battle to their donors' enemy — the
government — and came within a whisker of starving it of funding.
In the end, it appears, the government of the people, by the people and
for the people, might not perish from the Earth. But it will at least be
brought to heel — by and for a different kind of people
kind with shareholders and CEOs. And now — wouldn't you know it? — as we
emerge from the bitter battle over the government debt, some folks,
like Exxon Mobil, Johnson & Johnson and Microsoft
, find themselves with a higher credit rating than the battered United States government. You might call it the people's
Stephen Baker, a former senior writer for Business Week, is the author
most recently of "Final Jeopardy: Man vs. Machine and the Quest to Know
Nokia and the iPhone
|I covered Nokia in its glory years, from 1998 to 2002, when it
towered above the rest of the mobile industry. With its focus on
consumers, mastery of intuitive interface and enormous advantage in
scale, it thrashed Motorola, Ericsson, and a host of other pretenders
(Alcatel, Sony, Philips, Siemens, etc.) quarter after quarter. The company appeared poised to dominate the next phase
of Internet communications--the mobile revolution. I went so far as to
propose a book on the company. (I dropped the idea when BusinessWeek
moved me back from Europe to New York, following the dot-com bust.)
Nokia's future competitors, as phones turned into computers, were likely to come
from the software and consumer electronics industries. Nokia understood
this, and they beefed up research in Silicon Valley. But whenever I
raised computer-makers like Dell as future competitors, the Nokia folks
always had the same response: They had spent decades building expertise
in radio technologies, and it was hard stuff. It would be no easy thing
for a computer or software company to slap "telephone chips" into their
machines, as my writing implied. Radio was hard.
| Nokia's original Webphone, the Communicator, in 1996
Well, I'm sure they were right. It was no easy thing. But companies like
Apple and Google figured it out and, along with Canada's RIM,
maker of the Blackberry, they have taken over the smart-phone market.
Nokia's stock is swooning and its bonds are rated only one notch above
I was reminded of this while reading this morning that Nokia and Apple
have reached an agreement
on disputed intellectual property. Apple will
now pay Nokia royalties. I have no ideas what the patents cover, but it
must be cold comfort to Nokia. Instead of exercising its hard-won
mastery over radio technology to win control over mobile, the world's
most important communications market, it simply gets a slice of
royalties from the marquee company it lost to.
A couple more points from those Nokia years. When I was writing, it
seemed that Europe, with its single GSM standard, was well positioned to
lead the mobile Internet. Asia, with some of the most advanced mobile
markets in the world, appeared a much more powerful rival than North
America, which was viewed as a mobile backwater. But software
innovation, along with design, transformed the market. (Producing great
software can make up for lots of handicaps, which is one reason to be
optimistic about continued innovation leadership in the U.S.)
One more point about Apple. As I mentioned above, when I was looking at
computer-industry competitors for Nokia, I had my eyes on Microsoft,
Dell and Palm. I saw Apple as a potential acquisition. It seemed to me
that Nokia, whose market cap climbed to a quarter of a trillion dollars
during the boom, might gobble up the computer maker, or at least form an
alliance with it, to benefit from Apple's design and software prowess. I
don't know if Nokia execs ever considered such an idea. But if they
read BusinessWeek, they were at least familiar with it.
One more point, this one extending into the realm of wild hypotheticals.
If Nokia had taken over Apple, would Nokia have produced the iPhone? My
guess is that the iPhone would never have happened. Nokia would surely
have insisted that any smart phone work on top of its Symbian operating
system and fit into the family of Nokia phones. That would have been the
kiss of death. Still, Nokia would no doubt be better off today if it
had spent billions from its boomtime stash to buy--and kill--Apple.
Tuesday night massacre hurt Watson's ratings
February 17, 2011News
|Wall Street Journal's Speakeasy reports that interest in Jeopardy's Man-vs-Machine match dipped a bit for the final of the three-evening event. The Nielsen ratings climbed following Monday's match, in which Brad Rutter ended in a deadlock with the IBM computer. Ratings climbed from Monday's 8.5% of U.S. homes to 9.5% on Tuesday. But Watson thrashed the humans on Tuesday, only letting them answer a combined five clues. The audience then dipped to 9.1% for the final show.
Apparently, people thought the computer had it in the bag--I point I debate in the last chapter of my book. Fact is, if one of the two humans had won a quick $10,000 or so in the final game, and then doubled and quadrupled it with Daily Doubles, they could have been neck-and-neck with Watson. The trouble? It's hard to land on Daily Doubles when your bionic foe is controlling the board, hitting the buzzer about 1/100 of a second after the light goes on.
Interesting to note that the attendance figures represent only a 30% boost from the same week last year. I would have expected multiples of the normal Jeopardy viewership. But then, I happen to think this was the most fascinating thing on TV since... well maybe game five of the 2008 World Series.
By the way, Speakeasy is the same blog that published an article about our ebook strategy yesterday.
The Jeopardy match: Finale
February 16, 2011News
|As IBM's Watson headed into its final match against Ken Jennings and Brad Rutter,
it seemed to many that the game was locked up. Walking out of the
session following Watson's dominant game-one performance, IBM Chairman
suggested to colleague that maybe the team "should have dialled it down a
little." Maybe he was joking, but Palmisano was sitting with Sony
executives, who could not have been happy about Watson's day-two
shellacking, which threatened to drain the Man vs Machine match of its
But the IBM team led by David Ferrucci
knew all too well that Watson was still vulnerable. To win, though, one
of the human players would have to get lucky and bet audaciously. (This
was the formula Greg Lindsay used in his three wins against Watson in sparring matches a year ago.)
key was Daily Doubles. To win, either Jennings or Rutter had to earn a
quick $10,000 or so, and then double it in a Daily Double, and then
double it again in another one. Only by this type of doubling could one
of them get to $40,000 or $50,000, and erase the lead Watson had
established in game one (about $30,000 ahead of Jennings, $25,000 ahead
So the game would feature a relentless hunt for the
three Daily Doubles on the board, one in the Jeopardy round, and two in
Double Jeopardy. If Watson landed on them, the machine would bet small.
It was not about to risk its lead. The key for Watson, just like a
basketball team playing against the clock, was to deny its opponents
these chances to catch up.
UPDATE: Now I can say, as most of you already know, that Watson won. Here's Arik Hesseldahl's account in AllThingsD, along with his podcast with me.
I'm heading to an IBM event tonight.
I'll watch the game with at their NYC headquarters, and afterwards the
company will distribute copies of the book. It goes on sale tomorrow in
books stores, and the update with the last chapter should be arriving
By the way, here are some recent articles. In an op-ed in the Boston Globe, I argue that Watson will be our underling, not--as Ken Jennings might joke--our overlord. In the LA Times,
I write about what will happen to franchises like Jeopardy and, equally
important, our minds, as the Watsons of the world progress.
Ferrucci on Watson's Jeopardy performance: Day One
February 14, 2011News
|I went over to IBM Research this morning and watched Watson's performance on Jeopardy with David Ferrucci, the chief scientist on the project and the lead protagonist, along with Watson, in Final Jeopardy.
First things first. IBM's nightmare scenario can now be discarded. Watson can compete with the best human players. There was fear going in that one of the humans--and specifically Brad Rutter--might dominate the buzz. This didn't happen. If anything, Watson was the fastest of the three on the buzzer.
Here's one clue that tripped up Watson:
"It was the anatomical oddity of US gymnist George Eyser who won a gold medal on the parallel bars in 1904."
Ken Jennings won the buzz and guessed that Eyser was missing an arm. This was wrong, and Watson got the rebound. It answered, "What is leg?" Initially, as the game was taped, host Alex Trebek accepted the answer. But a judge stopped the game. After a lengthy parley, it was agreed that Watson missed the important point. Eyser's oddity was not his leg, but that he was missing one. They retaped the segment and Watson lost $1,000.
Ferrucci explained today that an "oddity" is a hard thing for a computer to puzzle out. What is it exactly? Well, the word is subjective from the get-go. We might find it an oddity if someone walked around my town in New Jersey carrying groceries on his head. In much of the world, this is normal.
So Watson, lacking its own point of view, would have had to find documents indicating that not only that Eyser was missing a leg, but that this was odd.
It came close. It concluded that leg was what distinguished Eyser. But it left it at that. Ferrucci said that Watson has been programmed to respond succinctly. Sometimes an extra detail can screw up an otherwise correct answer. If the host was not satisfied with leg, he could have asked for more details, and perhaps Watson would have added that it was missing. (This sometimes happens. For example, if Watson's response is The St Louis Cardinals, it can expand it to "The St. Louis Cardinals Baseball Club. But I haven't seen it refocus an answer, as it would have had to here. In any case, it never got the chance, because Trebek initially accepted "leg." This also prevented Rutter from a chance for an easy rebound.)
Watson's second notable mistake was an embarrassing one. Jennings missed the decade that gave birth to the Oreos, guessing the 1920s. The deaf and blind Watson, oblivious to its opponents, simply repeated Jennings' incorrect response.
Ferrucci was especially happy with Watson's final response, which pulled the machine into a tie with Rutter (at $5,000).
The clue, under the category Literary Characters APB: "Wanted for general evilness, last seen at the Tower of Barad-Dur. It's a giant eye, folks, kinda hard to miss." On this one, Watson first used "Tower of Barad-Dur" to lead it to J.R.R. Tolkien's Lord of the Rings. Then it had to find an "eye" associated with "evilness."
"If you manually craft a database of all eyes," Ferrucci said, referring to a traditional question-answering approach, "it's not clear that you'd include Sauron," a character in the book. But Watson managed to locate Sauron, and overcame the doubts it had that a character could also be an eye. It had 74% confidence in Sauron--and responded correctly.
One more point from the game. Ferrucci and his team were irked that "APB" in the Literary Characters category was spelled out for the contestants. Jennings and Rutter could glean from this that the characters would be villains. Watson could not benefit from this intelligence. That may have led it to lost on a Harry Potter clue to Rutter, who identified the book's uber villain, Valdemort. Watson was befuddled on the clue. It had 37% confidence in Harry Potter, only 20% in Valdemort.
Still, a good opening performance for the computer. In Double Jeopardy, on Tuesday, much more money will be at stake.
PBS Nova to air documentary on IBM's Watson
December 5, 2010News
|PBS's Nova will air a one-hour documentary on the Watson, IBM's Jeopardy computer (and the subject of my upcoming book), on Feb. 9.
NOVA “Smartest Machine on Earth”
Wednesday, February 9, 2011, 10:00-11:00 p.m. ET
investigates the world of artificial intelligence and profiles the
computer that could be the “Smartest Machine on Earth.” Known as
“Watson,” this IBM supercomputer is so advanced it’s pursuing the
first-of-its-kind challenge competing against “Jeopardy!” champions to
prove its uncanny ability to mimic the human thought process. This
program is the culmination of a special NOVA three-hour primetime block
on February 9 dubbed as “The Smartest Night on TV”: NOVA scienceNOW’s
“How Smart Are Animals?” at 8:00 pm ET, followed by NOVA’s “Making Stuff
Smarter” at 9:00 pm ET, concluding with NOVA’s “Smartest Machine on
Earth” at 10:00 pm ET.
The documentary is being produced by Michael Bicks and Sharon Kay. I've been seeing a lot of them while researching my book, and will certainly be watching that Wednesday in February.
RT @marthagabriel: "It is possible to store the mind with a million facts and still be entirely uneducated."
-- Alec Bourne #quote #goodmor…
follow me on twitter
Kirkus - Kirkus Reviews
Andrew Dunn - Bloomberg News
Culture Mob - Dan Sampson
Shelfari (Amazon) - Tom Nissley
read more reviews
Why Nate Silver is never wrong
- November 8, 2012
The psychology behind bankers' hatred for Obama
- September 10, 2012
"Corporations are People": an op-ed
- August 16, 2011
Wall Street Journal excerpt: Final Jeopardy
- February 4, 2011
Why IBM's Watson is Smarter than Google
- January 9, 2011
- October 3, 2010
The coming privacy boom
- August 17, 2010
The appeal of virtual
- May 18, 2010
My next book: IBM's Jeopardy mission
- March 22, 2010
- November 12, 2009
BusinessWeek cannot afford to stay within McGraw-Hill
- August 6, 2009
How to remake BusinessWeek?
- July 16, 2009