Hadoop technologies have started to mature and many of us are now tasked at implementing Hadoop MapReduce based solutions at our respective organizations. From our own practical experiences, we can tell that the going is not going to be as smooth as you might have expected. That’s what peaked our interest in this webinar by Platform Computing: Top Issues IT faces with Hadoop MapReduce This webinar claims th ...

I spend lot of time tracking and monitoring Hadoop jobs running across multiple clusters in my current project. Typically I navigate around multiple Job tracker web admin consoles. Although the job tracker web console gives some basic system level statuses and metrics for Hadoop daemons, it leaves a lot to be desired. What’s missing is a monitoring platform at the application level. In my Hadoop job I may h ...

Users share billions of pieces of content daily on Facebook, and it’s the data infrastructure team's job to analyze that data so we can present it to those users and their friends in the quickest and most relevant manner. This requires a lot of infrastructure and supporting data, so much so that we need to move that data periodically to ever larger data centers. Just last month, the data infrastructure team ...

Services is going to play a huge role in the ultimate success (or failure) of wide-spread adoption of Hadoop and related Big Data technologies by “mainstream” enterprises. The question is: Will Big Data services be delivered by commercial Hadoop vendors like Cloudera and Hortonworks as part of their value-add to the open source framework, or will a separate market of third-party services providers and consu ...

A decade ago, the open-source LAMP (Linux, Apache, MySQL, PHP/Python) stack began to transform web startup economics. As new open-source webservers, databases, and web-friendly programming languages liberated developers from proprietary software and big iron hardware, startup costs plummeted. This lowered the barrier to entry, changed the startup funding game, and led to the emergence of the current Angel/S ...

The ability to analyze enormous amounts of data and use that to better target an ad, make a new drug or any number of other lofty goals is talked about constantly, but what about harnessing Hadoop for the rest of us? Something as simple as figuring out when the best time to schedule a meeting and get everyone to attend or let a regional manager select what inventory to order can have huge boosts for busines ...

New to the Open Source Conference this year is OSCON Data, for developers pioneering the evolving architectures and tools to manage data. See how Hadoop is used to optimize scalability and reliability at Yahoo. Find out how Facebook utilizes HBase to manage real-time messaging. Why Netflix moved from relational DBs to NoSQL cloud systems for personalized movie choosing. The in-depth sessions at OSCON Data, ...

Last week was definitely a busy one in the MapReduce world! At the annual Hadoop Summit, Yahoo! officially announced the spinoff of HortonWorks (possibly the worst kept secret in the Hadoop community), and Cloudera and MapR both announced new distributions. With even more fragmentation coming to the Hadoop community, what better time to wrap up this series on the state of MapReduce. In my previous two posts ...

Hadoop, the data analytics-for-huge-data-sets invention of Apache Chairman Doug Cutting that found its original home at Yahoo, made some big news this week at the fifth annual Hadoop Summit in Santa Clara, Calif. First, it was revealed that Hadoop officially—but not "spiritually"—will break away from Yahoo and be shepherded by a new VC-funded company called Hortonworks, named after the Dr. Seuss elephant ch ...

LexisNexis is releasing a set of open source data-processing tools that it says outperforms Hadoop and even handles workloads that Hadoop presently cannot. The technology (and new business line) is called HPCC Systems, and was created 10 years ago within the LexisNexis Risk Solutions division that analyzes huge amounts of data for its customers in intelligence, financial services and other high-profile indu ...