Norikra: Stream processing by SQL

Today, we are using many batch processing middleware/framework such as Hive, Facebook Presto and others. On the other hand, many companies and service providers are pushing their stream processing middleware/services, like Storm, Kafka, AWS Kinesis or Google Dataflow.

Stream processing make our business extremely faster, but have some problems in its designs. For example, event ordering, fault tolerance and replays of calculations. We can and should do both of batch processing and stream processing by these reasons. That is called as "Lambda architecture".

In this talk, I'll introduce a SQL based stream processing middleware "Norikra", and I'll also describe a "Lambda architecture" application with Norikra and Hive/Presto, especially about its advantage and how to use each middleware for each purpose, and what problems we can solve by SQL based lambda architecture platform.