Big Data is clearly a disruptive technology, but using it successfully is as much art as it is science. The key is integrating Big Data with traditional BI to create a data ecosystem that allows you to generate new insights while executing on what you already know. CIO — Large enterprises, especially financial services firms, are adopting Big Data at a more rapid pace than expected, according to a recently ...

Hadoop Platform as a Service in the Cloud by Sriram Krishnan and Eva Tse, Data Science & Engineering Hadoop has become the de facto standard for managing and processing hundreds of terabytes to petabytes of data. At Netflix, our Hadoop-based data warehouse is petabyte-scale, and growing rapidly. However, with the big data explosion in recent times, even this is not very novel anymore. Our architecture, ...

Organizations as diverse as Northrop Grumman (NOC), Harvard University, Zynga, and the New York Stock Exchange (NYX) have filled job websites with requests for talented puppeteers and master chefs. A quick dig into the job listings reveals that these positions have nothing to do with office entertainment or gourmet meals. Instead, the companies want people who have mastered Puppet or Chef, competing softwar ...

Today Microsoft Research announced the availability of a free technology preview of Project Daytona MapReduce Runtime for Windows Azure. Using a set of tools for working with big data based on Google's MapReduce paper, it provides an alternative to Apache Hadoop. Daytona was created by the eXtreme Computing Group at Microsoft Research. It's designed to help scientists take advantage of Azure for working wit ...

CouchConf is the only conference dedicated to all things Couch. This one-day event is for any developer who wants to take a deeper dive into Couchbase technology, learn where it’s headed and build really cool stuff. Join CouchConf for a day in San Francisco to find out what’s new with Couchbase and learn more about harnessing this powerful technology for your web and mobile applications. BigDataCloud member ...

My last post was on location data query and indexing using MongoDB. Location data query and index support is an unique and powerful feature of MongoDB. Continuing along the same thread, I will dig into Map Reduce framework built right into MongoDB. Some NOSQL database systems provide built in map reduce framework. When the query engine is not enough for complex aggregate queries or other complex computation ...

My last post was on location data query and indexing using MongoDB. Location data query and index support is an unique and powerful feature of MongoDB. Continuing along the same thread, I will dig into Map Reduce framework built right into MongoDB. In case of MongoDB. the developer provides the map and reduce functions in javascript and then  executes the MongoDB mapreduce command. Although map reduce is a ...

Innovation around the management of large data sets is coming from the cloud, such as through MapReduce and Hadoop InfoWorld's own Pete Babb provided some good coverage around the "analytics cloud" recently debuted by IBM, called Blue Insight. You can think of Blue Insight as a system that gathers data from those who use it and externalizes the data to those who need it, doing so on a cloud -- a private clo ...

Discover how MapReduce and cloud computing are ideal for dealing with lots of data At times, you need to be able to access more physical and virtual resources to achieve complex compute-intensive results, but setting up a grid system within an organization can face resource, logistical, and technical hurdles; even some political ones. Cloud computing comes to the rescue in this case. It also combines perfec ...

Storage, MapReduce and Query are ushering in data-driven products and service "Big data" is data that becomes large enough that it cannot be processed using conventional methods. Creators of web search engines were among the first to confront this problem. Today, social networks, mobile phones, sensors and science contribute to petabytes of data created daily. To meet the challenge of processing such large ...