Esther Dyson on data visualization - Stephen Baker

Home - Viewing one post

Esther Dyson on data visualization posted on September 2, 2009

Datamining

I witnessed some conflict yesterday on one of my favorite blogs, Flowing Data. The blogger, Nathan Yao, who is pursuing his Phd in statistics and data visualization, charged a Boston consultant with ripping off his work. (UPDATE: the post has been removed, so maybe the issue was resolved.)

A tussle over data charts might seem like an arcane dispute, but it's right at the heart of the information economy. We're increasingly confronting massive data sets. Whether this information describes the patterns of our own bodies or the minute-to-minute chemical composition of the Hudson River, it will be way too big and unwieldy for most of us to draw conclusions from it.

I had an email exchange with Esther Dyson about this yesterday. She and I were at the big data roundtable last month at the Aspen Institute, and all agreed that visualization tools were crucial. (IBM's Fernanda Viegas, a co-creator of Many Eyes, had a great presentation on the subject. If I could get my hands on her slides, I'd post them.)

Here's what Esther had to say:

Long ago, we predicted that people would keep recipes or manage their wine cellars on their computers, but three things were lacking:

First, the computers weren't where the data was (which we have now solved with smart phones and with automatic data generation from many actions - such as buying the wine or (with other kinds of data) booking a flight - or from wearable monitors or other devices.

Second, there was no software to make sense of all the data, or to present it visually.

And finally, there weren't other people to compare your own data with - competing, collaborating or simply seeing yourself in context.

Now, almost everything will be knowable. The real trick will be filtering, not finding or searching. Right now, the media, advertisers and investors are still fascinated with search, but what we need now is software to extract order and meaning from the chaos of proliferating data. We don't need a searchlight illuminating a single web page in a dark space, but a flood of light showing us all the data in neatly labeled charts and diagrams and other visualizations.