New to the Open Source Conference this year is OSCON Data, for developers pioneering the evolving architectures and tools to manage data. See how Hadoop is used to optimize scalability and reliability at Yahoo. Find out how Facebook utilizes HBase to manage real-time messaging. Why Netflix moved from relational DBs to NoSQL cloud systems for personalized movie choosing. The in-depth sessions at OSCON Data, ...

Big data is getting bigger, and Hadoop is becoming the tool that will allow companies to make full use of their growing amounts of data, says Peter Fenton of Benchmark Capita. Fenton is a partner at the venture capital firm which owns the majority stake of HortonWorks, a startup that separated from Yahoo’s Hadoop dev team, and is offering services around the open-source big data analytics platform. In an in ...

Hadoop, the data analytics-for-huge-data-sets invention of Apache Chairman Doug Cutting that found its original home at Yahoo, made some big news this week at the fifth annual Hadoop Summit in Santa Clara, Calif. First, it was revealed that Hadoop officially—but not "spiritually"—will break away from Yahoo and be shepherded by a new VC-funded company called Hortonworks, named after the Dr. Seuss elephant ch ...

Informatica is joining the growing ranks of vendors moving to support Hadoop, the open-source framework for large-scale or "big data" processing, the company announced Monday. The 9.1 version of Informatica's platform features a connector to the Hadoop file system (HDFS), allowing customers to move data in and out of Hadoop clusters. While the Hadoop project has its roots in Web companies, having been led b ...

At la 2006, today, Google, Microsoft, and Yahoo collectively announced that they will be partnering to create schema.org, a resource for site owners and developers to learn about structured data and gain insight into how to improve their sites’ search results. The site adds more than 100 new forms of website markup for content ranging from movies to places in an effort to standardize, and thus improve, how ...

Most data processing tasks with Hadoop require multiple Hadoop jobs with dependencies between them. The dependency arises out of the need for one job to use the output for another job. The dependency between Hadoop jobs can be expressed as a directed acyclic graph (DAG), where each node represents a Hadoop job. DAG is useful for modelling relationship between entities that have partial ordering. The depende ...

The Cloud means different things to different people. Everyone has their own views of the Cloud and myriad usage patterns have evolved, even while new and innovative options are emerging everyday. The standard definition of Cloud computing, as quoted in Wikipedia: So in essence, the Cloud is an environment where all aspects of computing are availed of as a service and consumers, be they individuals or busin ...

Innovation around the management of large data sets is coming from the cloud, such as through MapReduce and Hadoop InfoWorld's own Pete Babb provided some good coverage around the "analytics cloud" recently debuted by IBM, called Blue Insight. You can think of Blue Insight as a system that gathers data from those who use it and externalizes the data to those who need it, doing so on a cloud -- a private clo ...

Storage, MapReduce and Query are ushering in data-driven products and service "Big data" is data that becomes large enough that it cannot be processed using conventional methods. Creators of web search engines were among the first to confront this problem. Today, social networks, mobile phones, sensors and science contribute to petabytes of data created daily. To meet the challenge of processing such large ...

We are in a Renaissance for computer science, engineering, and learning from data right now. The scale of data and computations is an important issue, but the data age is less about the raw size of your data, and more about the cool stuff you can do with it. Now that there is so much data, it is time to unlock its value. Really neat things are happening already—like the way the people of the world can educa ...