You Are Here: Home » Big Data

What is a blockchain? GeekWire’s guide to this game-changing technology and its vast potential – GeekWire

Most technologies touted as “game-changing” are just somewhat modified versions of something already in use. Blockchains are different, and while it will take some time to work out the kinks, they could one day make as much of an impact on the world as the database, smartphone or web browser. If you’ve heard of bitcoin — and unless the state of the world has forced you to stop following the news, you probab ...

Read more

A Technical Overview of Azure Databricks – The Databricks Blog

Today at Microsoft Connect(); we introduced Azure Databricks, an exciting new service in preview that brings together the best of the Apache Spark analytics platform and Azure cloud. As a close partnership between Databricks and Microsoft, Azure Databricks brings unique benefits not present in other cloud platforms. This blog post introduces the technology and new capabilities available for data scientists, ...

Read more

Apache Kafka and the four challenges of production machine learning systems – O’Reilly Media

Machine learning has become mainstream, and suddenly businesses everywhere are looking to build systems that use it to optimize aspects of their product, processes or customer experience. The cartoon version of machine learning sounds quite easy: you feed in training data made up of examples of good and bad outcomes, and the computer automatically learns from these and spits out a model that can make simila ...

Read more

Getting Started Analyzing Twitter Data in Apache Kafka through KSQL

KSQL is the open source streaming SQL engine for Apache Kafka. It lets you do sophisticated stream processing on Kafka topics, easily, using a simple and interactive SQL interface. In this short article we’ll see how easy it is to get up and running with a sandbox for exploring it, using everyone’s favourite demo streaming data source: Twitter. We’ll go from ingesting the raw stream of tweets, through to fi ...

Read more

Go Python, Go! Stream Processing for Python · Wallaroo Labs

We’ve been hard at work for 18 months on a new processing engine called Wallaroo for deploying and operating big data, fast data, and machine learning applications. We designed Wallaroo to make the infrastructure virtually disappear, so you get rapid deployment and easy-to-operate applications. It provides a simple model for building fast applications that scale automatically across any number of workers. W ...

Read more

How Big Data Could Help Prevent Fatal Heart Attacks

To celebrate World Heart Day, which was on Sept. 29, today I will join the American Heart Association Conference, AHA Health Tech & Innovation Forum to discuss how emerging technologies can transform and create healthy communities. According to the American Heart Association, approximately 2,200 Americans die of cardiovascular disease each day, an average of one death every 40 seconds. In fact, cardiova ...

Read more

Open Sourcing Vespa, Yahoo’s Big Data Processing and Serving Engine

Ever since we open sourced Hadoop in 2006, Yahoo – and now, Oath – has been committed to opening up its big data infrastructure to the larger developer community. Today, we are taking another major step in this direction by making Vespa, Yahoo's big data processing and serving engine, available as open source on GitHub. Vespa architecture overview Building applications increasingly means dealing with huge a ...

Read more

Publishing with Apache Kafka at The New York Times | Confluent

At The New York Times we have a number of different systems that are used for producing content. We have several Content Management Systems, and we use third-party data and wire stories. Furthermore, given 161 years of journalism and 21 years of publishing content online, we have huge archives of content that still need to be available online, that need to be searchable, and that generally need to be availa ...

Read more

Unite Real-Time and Batch Analytics Using the Big Data Lambda Architecture, Without Servers! | AWS Big Data Blog

The Big Data Lambda Architecture seeks to provide data engineers and architects with a scalable, fault-tolerant data processing architecture and framework using loosely coupled, distributed systems. At a high level, the Lambda Architecture is designed to handle both real-time and historically aggregated batched data in an integrated fashion. It separates the duties of real-time and batch processing so purpo ...

Read more

Introducing KSQL: Open Source Streaming SQL for Apache Kafka

What does it even mean to query streaming data, and how does this compare to a SQL database? Well, it’s actually quite different to a SQL database. Most databases are used for doing on-demand lookups and modifications to stored data. KSQL doesn’t do lookups (yet), what it does do is continuous transformations— that is, stream processing. For example, imagine that I have a stream of clicks from users and a t ...

Read more

2015 © Big Data Cloud Inc. All Rights Reserved.

Hadoop and the Hadoop elephant logo, Sprark are trademarks of the Apache Software Foundation.

Scroll to top