Big Data Cloud August 11th Meetup – Hadoop powered Engines; 100+ Attendees; Corporate Sponsorships & Giveaways

BigDataCloud’s theme of “Hadoop Powered Predictions & Recommendations Engines” attracted over 100 people to the meetup last night, sponsored by LexisNexis & ThirdEyeCloud.

The attendees thronged the LexisNexis’s booth about its newly debuted HPCC Systems & got an understanding of how its Hadoop alternative can actually solve “Big Data” challenges in enterprises. The attendees also had a chance to “meet & greet the gurus” taking the “Byte-sized Hadoop classes” at Third Eye Cloud’s booth.

The attendees networked & mingled with each other as they munched away. Many of our first time attendees expressed their reason for attending was to “grow knowledge” and get a “better understanding of Big Data” and ultimately discover if Hadoop is a viable solution for their own projects. It was especially encouraging to see a wide range of industries present, with representatives from companies like Adobe, Cisco, Eucalyptus Systems, Google, RockYou and many others, all eager to learn more about Hadoop and hear from our two guest speakers, Pranab Ghosh (Motorola) and Mitul Tiwari (LinkedIn).

Pranab Ghosh, kicked off the evening by presenting his recommendations for collaborative filtering using Hadoop. He went into great detail about the importance of finding correlations between similar products and calculating rating correlations using the Pearson Correlation coefficient formula. He further outlined how to execute MapReduce jobs in tandem that ultimately become new items recommended to a visitor. [Mitul Tiwari from LinkedIn, brought the audience alive with a presentation on LinkedIn’s ‘People You May Know’ product which analyzes daily billions of edges and terabytes of data. The system is built using a large scale distributed compute infrastructure. Kafka publish-subscribe messaging system is used to get the data in Hadoop file system. Hadoop MapReduce is used as the basic building block to analyze billions of potential options, and predict recommendation. Over a hundred MapReduce tasks are combined together in a work-flow using Azkaban, a Hadoop work-flow management tool. The output of Hadoop jobs is finally stored in Voldemort key-value store to serve the data at run-time for efficiency. [Third Eye Consulting Services for making this meetup such a huge success!