Spark Streaming Pros and Cons

Its ability to handle large volumes of data makes it horizontally scalable and its fault tolerance through data replication and and also its support for batch streaming make data process faster Review collected by and hosted on G2.com.

Micro batching latency reduces the latency and also its resource-intensive consuming large amount of resources Review collected by and hosted on G2.com.

What I like best about Spark Streaming is its ability to handle real-time data processing efficiently while maintaining high throughput. It enables seamless integration with the Apache Spark ecosystem, providing access to a wide range of libraries and tools. The programming model is easy to work with, and its fault tolerance mechanisms ensure reliable data processing even in the face of failures. Additionally, Spark Streaming's scalability and integration with various data sources make it a versatile choice for handling streaming data. Review collected by and hosted on G2.com.

No built-in support for event time processing. Review collected by and hosted on G2.com.

With the help of spark streaming huge amounts of data can be transferred with literally zero latency. Scripts are easy to configure and execute using spark clusters. Most important, failures can be found and and resolved with the spark UI logs Review collected by and hosted on G2.com.

There is a lot to learn about Spark Streaming and bulks of documentation can sometimes be a bit overwhelming to go through. Data visualisation can be more enhanced rather than having the basic interface. It can be costly at times if clusters are not properly optimised. Review collected by and hosted on G2.com.

Spark streaming is very simple and easy to implement as we need to only configure parameters to the existing package. Review collected by and hosted on G2.com.

Sometimes when drivers are not available we loose the connection easily and we have make another run by clearing states to get a proper run Review collected by and hosted on G2.com.

Spark streaming is one the best thing to stream data easily, after the kafka

If you want to steam small or medium amount of data you can easily and safely use the spark streaming Review collected by and hosted on G2.com.

Kafka is more better than spark streaming because spark streaming do not work properly with larger amount of data where as Kafka streaming handle the data very well. Review collected by and hosted on G2.com.

Real-time Analytics and an open-source solution. Review collected by and hosted on G2.com.

Complex to setup initially and not that much relevant for small applications. Review collected by and hosted on G2.com.

Spark streaming is one of the key components which helps the real time streaming of data and also gives lots of enhancement that helps procesisng larger datasets Review collected by and hosted on G2.com.

There is no dislike I feel In general but the compatibility does matter some or the other time on different platforms. But still its the best streaming and processing Review collected by and hosted on G2.com.

I was able to build complex data pipeline using apache spark Review collected by and hosted on G2.com.

Spark is generally not suitable when the dataset is relatively small. Review collected by and hosted on G2.com.

It's evolution in Big Data World. Very trendy and evolving. Also people are using for real time processing as well as batch processing which saves cost too. Thankful Review collected by and hosted on G2.com.

It's difficult to understand and learn. Not much resources available. Also, people must have a hard core big data background with map reduce and java understanding to further understand spark streaming Review collected by and hosted on G2.com.

Spark is a very powerful framework and we run spark streaming jobs for multiple requirements such as gathering data from flume, kafka ,sqoop , hdfs and pushing it into other nodes.

One of the daily used spark streaming job is for copying our data from production to DR using spark streaming job. What we do here is we copy the fsimages from production and dr cluster, and then run a spark streaming job to flatten the image and calculate diff, post which the data is then pushed to a database and data is copied from production to dr using the diff of namespace image. We have copied almost 800+ TB data using this streaming job. Review collected by and hosted on G2.com.

Spark streaming jobs are resources intensive as well as complex so you need engineers who know well how to tune the job else one spark streaming job could consume resources enough to bring a multi-node cluster down. Review collected by and hosted on G2.com.