Big Data Is Less About Size, And More About Freedom
We are in a Renaissance for computer science, engineering, and learning from data right now. The scale of data and computations is an important issue, but the data age is less about the raw size of your data, and more about the cool stuff you can do with it. Now that there is so much data, it is time to unlock its value. Really neat things are happening already—like the way the people of the world can educate themselves on all manner of issues and topics, or the way data and computing serves as leverage in other scientific and technical endeavors. There will be lots of amazing stuff on the web, but innovation will come in other domains as well.
The recent big data trend is about the democratization of large data more than its growth. In articles like the Economist’s recent piece on the data deluge, we hear about big data everywhere. We hear about what big data and the cloud mean for the enterprise, but they have had big data for a long time. eBay manages petabytes in its Teradata and Greenplum data warehouses. Sophisticated startups extracting value from big data is also nothing new—it has been happening at least since the days of Yahoo! and Google, and they have done it without the data warehousing folks.
Now focused early stage startups can get up and running faster than ever. Less technical analysts at companies like Facebook and Twitter can access massive amounts of data easily. Even individuals can undertake cool projects with big data, such as Pete Skomoroch of Data Wrangling did with trending topics for Wikipedia.