You Are Here: Home » Real Time

Getting Started Analyzing Twitter Data in Apache Kafka through KSQL

KSQL is the open source streaming SQL engine for Apache Kafka. It lets you do sophisticated stream processing on Kafka topics, easily, using a simple and interactive SQL interface. In this short article we’ll see how easy it is to get up and running with a sandbox for exploring it, using everyone’s favourite demo streaming data source: Twitter. We’ll go from ingesting the raw stream of tweets, through to fi ...

Read more

Unite Real-Time and Batch Analytics Using the Big Data Lambda Architecture, Without Servers! | AWS Big Data Blog

The Big Data Lambda Architecture seeks to provide data engineers and architects with a scalable, fault-tolerant data processing architecture and framework using loosely coupled, distributed systems. At a high level, the Lambda Architecture is designed to handle both real-time and historically aggregated batched data in an integrated fashion. It separates the duties of real-time and batch processing so purpo ...

Read more

Introducing KSQL: Open Source Streaming SQL for Apache Kafka

What does it even mean to query streaming data, and how does this compare to a SQL database? Well, it’s actually quite different to a SQL database. Most databases are used for doing on-demand lookups and modifications to stored data. KSQL doesn’t do lookups (yet), what it does do is continuous transformations— that is, stream processing. For example, imagine that I have a stream of clicks from users and a t ...

Read more

When every drop counts: Schneider Electric transforms agriculture with the Internet of Things for sustainable farming – Transform

In the grassy Canterbury Plains of New Zealand, Craig Blackburn raises cattle and sheep in a line of work with a long tradition, in which he keeps a close eye on crops, land, weather and water. But Blackburn blends modern technology with his agricultural roots to manage the 990-acre Blackhills farm, a complex, bustling operation with 2,100 cattle and 800 sheep. The farm runs on irrigated water from the scen ...

Read more

Running Streaming Jobs Once a Day For 10x Cost Savings – The Databricks Blog

This is the sixth post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. Traditionally, when people think about streaming, terms such as “real-time,” “24/7,” or “always on” come to mind. You may have cases where data only arrives at fixed intervals. That is, data appears every hour or once a day. For these use cases, it is still beneficial to perform incrementa ...

Read more

Really Big Data At Walmart: Real-Time Insights From Their 40+ Petabyte Data Cloud

Walmart – the world’s biggest retailer with over 20,000 stores in 28 countries, is in the process of building the world’ biggest private cloud, to process 2.5 petabytes of data every hour. To make sense of all of this information, and put it to work solving problems, the company has created what it calls its Data Café – a state-of-the-art analytics hub located within its Bentonville, Arkansas headquarters. ...

Read more

BMW’s vision for connected cars includes Cortana in your dash | Windows Central

BMW is at CES showing off a range of tech it envisions as part of a connected car experience. While the entire vision is a neat look into where BMW expects the near-future of connected, automated cars to go, one particular inclusion will stick out to Microsoft fans: Cortana. AS shown off as part of CES 2017 demo, BMW sees Cortana living in the car's dash to offer the same voice-activated assistance with whi ...

Read more

Microsoft launches a new cloud platform for connected cars | TechCrunch

Microsoft isn’t building its own connected car — but it is launching a new Azure-based cloud platform for car manufacturers that want to use the cloud to power their own connected-car services. The new Microsoft Connected Vehicle Platform will go live as a public preview later this year. “This is not an in-car operating system or a ‘finished product’,” Microsoft’s EVP for business development Peggy Johnson ...

Read more

Writing SQL on Streaming Data with Amazon Kinesis Analytics – Part 2 – AWS Big Data Blog

Amazon Kinesis Analytics allows you to easily write SQL ­­­on streaming data, providing a powerful way to build a stream processing application in minutes. The service allows you to connect to streaming data sources, process the data with sub-second latencies, and continuously emit results to downstream destinations for use in real-time alerts, dashboards, or further analysis. This post introduces you to th ...

Read more

Analyze Realtime Data from Amazon Kinesis Streams Using Zeppelin and Spark Streaming – AWS Big Data Blog

There is streaming data everywhere. This includes clickstream data, data from sensors, data emitted from billions of IoT devices, and more. Not suprisingly, data scientists want to analyze and explore these data streams in real time. This post shows you how you can use Spark Streaming to process data coming fromAmazon Kinesis streams, build some graphs using Zeppelin, and then store the Zeppelin notebook in ...

Read more

2015 © Big Data Cloud Inc. All Rights Reserved.

Hadoop and the Hadoop elephant logo, Sprark are trademarks of the Apache Software Foundation.

Scroll to top