The potential of “big data,” the massive explosion of sources of information from sensors, smart devices, and all other devices connected to the Internet, is probably under-appreciated in terms of its eventual business impact. However, to take maximum advantage of big data, IT is going to have to press the re-start button on its architecture for acquiring and understanding information. IT will need to const ...

New to the Open Source Conference this year is OSCON Data, for developers pioneering the evolving architectures and tools to manage data. See how Hadoop is used to optimize scalability and reliability at Yahoo. Find out how Facebook utilizes HBase to manage real-time messaging. Why Netflix moved from relational DBs to NoSQL cloud systems for personalized movie choosing. The in-depth sessions at OSCON Data, ...

Big data is getting bigger, and Hadoop is becoming the tool that will allow companies to make full use of their growing amounts of data, says Peter Fenton of Benchmark Capita. Fenton is a partner at the venture capital firm which owns the majority stake of HortonWorks, a startup that separated from Yahoo’s Hadoop dev team, and is offering services around the open-source big data analytics platform. In an in ...

Is the day of reckoning for big data upon us? To many observers, the growth in data is nothing short of incomprehensible. Data is streaming into, out of, and through enterprises from a dizzying array of sources-transactions, remote devices, partner sites, websites, and nonstop user-generated content. Not only are the data stores resulting from this information driving databases to scale into the terabyte an ...

When your relational database takes longer to process your data than to collect it, it's time to call in big data technology, said panelists at Interop. Not everyone is sure whether they have big data or not, or whether they need a NoSQL system to handle it. One way to find out, said one adopter of a NoSQL approach, is to ask yourself whether it is taking you longer to process your data than it did to colle ...

There is a lot of buzz around Big Data and the NOSQL movement these days and rightly so. The issues with data have essentially been two-fold: find cost effective ways to store ever increasing amounts of data and information, and find ways to mine this information to extract meaningful Business Intelligence. This problem has been compounded by the emergence of web 2.0 technologies whose legion of loyal fans ...

My last post was on location data query and indexing using MongoDB. Location data query and index support is an unique and powerful feature of MongoDB. Continuing along the same thread, I will dig into Map Reduce framework built right into MongoDB. Some NOSQL database systems provide built in map reduce framework. When the query engine is not enough for complex aggregate queries or other complex computation ...

Scale breaks everything. Scale even breaks your assumptions about how best to store and query data. Scale does not care about your personal engineering preferences, or about SQL vs. NoSQL. The demands of rapid growth and ever-higher expectations for availability, performance, and cost efficiency force you to re-evaluate and re-imagine what you need, what is possible, and how to best achieve your business go ...

Today’s data volumes are growing incrementally The amount of data a business collects has grown exponentially over the past decade and there is no end in sight. This amount of data, although more data is good, presents significant data management issues for DBAs and data analysts. Firstly, how does a business contend with daily VLD (Very Large Data sets) volumes timely?  In my particular case, we were tryin ...