Expectations surrounding the future of big data range from the just huge to absolutely enormous – a reflection perhaps of both its real inherent potential and all the massive hype. Certainly though there is no dispute that companies can reap big benefits from exploring patterns found in the data they already generate and collect. Further, depending on the algorithms used, machine learning can even serve as ...

What’s standing between your staff and big data analysis? That was the existential question posed of DJ Patil and Jeff Hammerbacher at the GigaOM Structure:Data event today in New York. The two had different takes on how easy it was to give people the power to use data, with Hammerbacher, who is the co-founder of Cloudera, saying that it’s pretty simple today. He did say that today many aspects of the input ...

Much has been done to bring big data closer to the people who need it. The advent of public cloud infrastructure has decimated the cost of collecting, maintaining and processing vast amounts of data. The next frontier is making that data reproducible, said Matt Wood, principal data scientist for Amazon Web Services, at GigaOM’s Structure:Data 2013 event Wednesday. In short, it’s great to get a result from y ...

We talk a lot about big data, but only analyze 1 percent of what’s available. In order to take advantage of the other 99 percent, we need to reconsider how we do big data. Many would be shocked to know that researchers analyze and gather insights from only 1 percent of the world’s data. That 1 percent of analyzed data has been the only driver of innovation and insights in what we now know as “big data.” The ...

The cloud computing world has been anxious to hear former NASA CTO Chris Kemp’s plans since he left his post in the spring to pursue a startup venture, and on Wednesday morning, he introduced his new company, Nebula. It’s selling an OpenStack-based appliance for managing scale-out cloud deployments, and is backed by premier talent and investors. Nebula isn’t disclosing how much it has raised, but Co-Founder ...

Another study out this week has found that if companies adopt cloud computing, they can reduce the energy consumption of their IT and save money on energy bills. The report, created by research firm Verdantix and sponsored by AT&T, estimates that cloud computing could enable companies to save $12.3 billion off their energy bills. That translates into carbon emission savings of 85.7 million metric tons per y ...

In the past year, big data has emerged as one of the most closely watched trends in IT. Organizations today are generating more data in a single day than that the entire Internet was generated as recently as 2000. The explosion of “big data”–much of it in complex and unstructured formats–has presented companies with a tremendous opportunity to leverage their data for better business insights through analyti ...

The ability to analyze enormous amounts of data and use that to better target an ad, make a new drug or any number of other lofty goals is talked about constantly, but what about harnessing Hadoop for the rest of us? Something as simple as figuring out when the best time to schedule a meeting and get everyone to attend or let a regional manager select what inventory to order can have huge boosts for busines ...

BigDataCloud would be live blogging the GigaOM Structure 2011 event. Dj Das would be onsite & live – blogging with his own perspective of the action on the ground! This event is now sold out and you can watch the live video stream here About Structure 2011 Making Sense of the Real Cloud After years of questions about what cloud computing is and how it will affect IT, we’re finally starting to get answers. W ...

LexisNexis is releasing a set of open source data-processing tools that it says outperforms Hadoop and even handles workloads that Hadoop presently cannot. The technology (and new business line) is called HPCC Systems, and was created 10 years ago within the LexisNexis Risk Solutions division that analyzes huge amounts of data for its customers in intelligence, financial services and other high-profile indu ...