You Are Here: Home » Real Time

Introducing KSQL: Open Source Streaming SQL for Apache Kafka

What does it even mean to query streaming data, and how does this compare to a SQL database? Well, it’s actually quite different to a SQL database. Most databases are used for doing on-demand lookups and modifications to stored data. KSQL doesn’t do lookups (yet), what it does do is continuous transformations— that is, stream processing. For example, imagine that I have a stream of clicks from users and a t ...

Read more

When every drop counts: Schneider Electric transforms agriculture with the Internet of Things for sustainable farming – Transform

In the grassy Canterbury Plains of New Zealand, Craig Blackburn raises cattle and sheep in a line of work with a long tradition, in which he keeps a close eye on crops, land, weather and water. But Blackburn blends modern technology with his agricultural roots to manage the 990-acre Blackhills farm, a complex, bustling operation with 2,100 cattle and 800 sheep. The farm runs on irrigated water from the scen ...

Read more

Running Streaming Jobs Once a Day For 10x Cost Savings – The Databricks Blog

This is the sixth post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. Traditionally, when people think about streaming, terms such as “real-time,” “24/7,” or “always on” come to mind. You may have cases where data only arrives at fixed intervals. That is, data appears every hour or once a day. For these use cases, it is still beneficial to perform incrementa ...

Read more

Really Big Data At Walmart: Real-Time Insights From Their 40+ Petabyte Data Cloud

Walmart – the world’s biggest retailer with over 20,000 stores in 28 countries, is in the process of building the world’ biggest private cloud, to process 2.5 petabytes of data every hour. To make sense of all of this information, and put it to work solving problems, the company has created what it calls its Data Café – a state-of-the-art analytics hub located within its Bentonville, Arkansas headquarters. ...

Read more

BMW’s vision for connected cars includes Cortana in your dash | Windows Central

BMW is at CES showing off a range of tech it envisions as part of a connected car experience. While the entire vision is a neat look into where BMW expects the near-future of connected, automated cars to go, one particular inclusion will stick out to Microsoft fans: Cortana. AS shown off as part of CES 2017 demo, BMW sees Cortana living in the car's dash to offer the same voice-activated assistance with whi ...

Read more

Microsoft launches a new cloud platform for connected cars | TechCrunch

Microsoft isn’t building its own connected car — but it is launching a new Azure-based cloud platform for car manufacturers that want to use the cloud to power their own connected-car services. The new Microsoft Connected Vehicle Platform will go live as a public preview later this year. “This is not an in-car operating system or a ‘finished product’,” Microsoft’s EVP for business development Peggy Johnson ...

Read more

Writing SQL on Streaming Data with Amazon Kinesis Analytics – Part 2 – AWS Big Data Blog

Amazon Kinesis Analytics allows you to easily write SQL ­­­on streaming data, providing a powerful way to build a stream processing application in minutes. The service allows you to connect to streaming data sources, process the data with sub-second latencies, and continuously emit results to downstream destinations for use in real-time alerts, dashboards, or further analysis. This post introduces you to th ...

Read more

Analyze Realtime Data from Amazon Kinesis Streams Using Zeppelin and Spark Streaming – AWS Big Data Blog

There is streaming data everywhere. This includes clickstream data, data from sensors, data emitted from billions of IoT devices, and more. Not suprisingly, data scientists want to analyze and explore these data streams in real time. This post shows you how you can use Spark Streaming to process data coming fromAmazon Kinesis streams, build some graphs using Zeppelin, and then store the Zeppelin notebook in ...

Read more

Distributed, Real-time Joins and Aggregations on User Activity Events using Kafka Streams

In previous blog posts we introduced Kafka Streams and demonstrated an end-to-end Hello World streaming application that analyzes Wikipedia real-time updates through a combination of Kafka Streams and Kafka Connect. In this blog post we want to continue the introduction series on Kafka Streams by implementing a very common and very important use case in stream processing: to enrich an incoming stream of eve ...

Read more

How-to: Detect and Report Web-Traffic Anomalies in Near Real-Time – Cloudera Engineering Blog

This framework based on Apache Flume, Apache Spark Streaming, and Apache Impala (incubating) can detect and report on abnormal bad HTTP requests within seconds.    Website performance and availability are mission-critical for companies of all types and sizes, not just those with a revenue stream directly tied to the web. Web pages can become unavailable for many reasons, including overburdened backing data ...

Read more

2015 © Big Data Cloud Inc. All Rights Reserved.

Hadoop and the Hadoop elephant logo, Sprark are trademarks of the Apache Software Foundation.

Scroll to top