X
X
xxx2019-04-17 13:14:54
Python
xxx, 2019-04-17 13:14:54

kinesis+spark stream architecture?

There is a stream on AWS. I am using the KCL library for python. Consumer and producer work, I run them on the local machine, the data is run through the stream, everything is fine. Question - I want to integrate spark streaming here, the documentation says that it is used in conjunction with kinesis in the following way: the data comes to the kinesis consumer and from there the data goes to spark streaming. Do I understand correctly that you need to deploy an EMR Cluster with Spark and already throw the consumer, which is kinesis, into the master? If so, where would the spark streaming code be? Please explain the architecture.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question