Cloudera on Tuesday announced the general release of its Impala query engine for Hadoop after six months of beta testing by more than 40 customers. It's the first so-called SQL-on-Hadoop product to reach general release. But with a bevy of such systems on the way -- including options from IBM (Big SQL), Hortonworks (Stinger), MapR (Drill), Pivotal (HAWQ) and Teradata (SQL-H) -- the question is whether Impal ...

It has been a busy time for announcements coinciding with this week’s Strata conference. There’s no corner of the technology world that has not embraced Apache Hadoop as the new platform for big data. Apache Hadoop began as a telegram from the future from Google, turned into real software by Doug Cutting while on a freelance assignment. While Hadoop’s origins are surprising, its ongoing popularity is not – ...

Introduction to Hadoop 2 Core Hadoop 2 consists of the distributed filesystem HDFS and the compute framework YARN. HDFS is a distributed filesystem that can be used to store anywhere from a few gigabytes to many petabytes of data. It is distributed in the sense that it utilizes a number of slave servers, ranging from 3 to a few thousand, to store and serve files from. YARN is the compute framework for Hadoo ...

The Big Data Infra for Enterprises – Making Hadoop Secure for Enterprises session was probably my 4th or 5th attendance to BigDataCloud meetup hosted by DJ and Jeeta Das. Time is luxury and if you login to meetup.com, several meetups with names that generate significant interest crop up. I am sure all agree with me. However, attending BigDataCloud meetup is time well invested. Few days ago I had spoken to J ...

Hadoop technologies have started to mature and many of us are now tasked at implementing Hadoop MapReduce based solutions at our respective organizations. From our own practical experiences, we can tell that the going is not going to be as smooth as you might have expected. That’s what peaked our interest in this webinar by Platform Computing: Top Issues IT faces with Hadoop MapReduce This webinar claims th ...

I spend lot of time tracking and monitoring Hadoop jobs running across multiple clusters in my current project. Typically I navigate around multiple Job tracker web admin consoles. Although the job tracker web console gives some basic system level statuses and metrics for Hadoop daemons, it leaves a lot to be desired. What’s missing is a monitoring platform at the application level. In my Hadoop job I may h ...

Users share billions of pieces of content daily on Facebook, and it’s the data infrastructure team's job to analyze that data so we can present it to those users and their friends in the quickest and most relevant manner. This requires a lot of infrastructure and supporting data, so much so that we need to move that data periodically to ever larger data centers. Just last month, the data infrastructure team ...

Services is going to play a huge role in the ultimate success (or failure) of wide-spread adoption of Hadoop and related Big Data technologies by “mainstream” enterprises. The question is: Will Big Data services be delivered by commercial Hadoop vendors like Cloudera and Hortonworks as part of their value-add to the open source framework, or will a separate market of third-party services providers and consu ...

A decade ago, the open-source LAMP (Linux, Apache, MySQL, PHP/Python) stack began to transform web startup economics. As new open-source webservers, databases, and web-friendly programming languages liberated developers from proprietary software and big iron hardware, startup costs plummeted. This lowered the barrier to entry, changed the startup funding game, and led to the emergence of the current Angel/S ...