Hadoop, the clear king of big-data analytics, is focused on batch processing. This model is sufficient for many cases (such as indexing the web), but other use models exist in which real-time information from highly dynamic sources is required. Solving this problem resulted in the introduction of Storm from Nathan Marz (now with Twitter by way of BackType). Storm operates not on static data but on streaming ...

Companies are finding that big data doesn't necessarily translate into easy success. Experts who specialize in computer-driven analysis of large streams of information say too many companies throw themselves into big-data projects, only to fall into common traps and end up with nothing to show for their efforts. Some 44% of information-technology professionals surveyed by business-software firm Infochimps I ...

Haul out those data scientist demand projections for a refresh — just as several tech consultants had predicted a year ago, new technology looks ready to simplify a lot of the analytics so a business analyst, maybe even a mere business person, can handle them. Estimates of the data scientist shortage had ranged from 150,000 to 1.5 million, depending on the geography and the definition of data scientist or d ...

Intel today announced that it is joining the Hadoop ecosystem in a permanent way that will accelerate adoption of Hadoop by the broader enterprise market. While the news that Intel is going to offer an distribution that has been extended with features for security and improved performance is interesting enough, Intel’s arrival illustrates some broader trends at play in the world of Hadoop, namely that: Hado ...

Network World — OK, lots of interesting stuff for you this week. First up, LinkedIn has open sourced a system called Databus, a real-time database change capture system that provides a "timeline-consistent stream of change capture events ... grouped in transactions, in source commit order." This code has been in use inside LinkedIn since 2011 where they use it to "propagate profile, connection, company upda ...

Infosys launches BigDataEdge: New Platform Develops Enterprise Insights up to Eight Times Faster, Fosters Near Real-Time Decisions and Actions New York, February 20, 2013: Infosys today announced the launch of BigDataEdge to radically simplify the complex task of analyzing Big Data to discover relevant information. By empowering business users to rapidly develop insights from vast amounts of structured and ...

Intel, a company famous for its computer processing hardware, has jumped into the Hadoop software business, asserting that it wants to spur growth of big data analytics deployments in large data centers Data Informed reported. Intel announced plans on Feb. 26 to make the third version of its distribution of Hadoop and its accompanying Intel Manager for Apache Hadoop software available globally in the second ...

EMC on Monday rolled out its own distribution of Hadoop in a move that integrates the open source big data software directly with its Greenplum intellectual property. The aim: Take on Cloudera. The distribution, called Pivotal HD, is notable because it puts EMC in competition with Cloudera, which has a bevy of partners and is often seen as the Red Hat of big data. Many data warehousing players have built Ha ...