Spark Reviews & Product Details

I have used spark for data processing purpose the thing that I like the most is the speed , it process huge amount of data because of in memory computation which is very better a compare to Hadoop map reduce Review collected by and hosted on G2.com.

The thing that I don't like about spark is that infrastructure cost that is very high when it comes to run the data on a cluster environment Review collected by and hosted on G2.com.

The speed of spark

The integration feature of spark with custom softwares, with other tools.

The ease of use and user adaptability. Review collected by and hosted on G2.com.

No File system to manage.

Limited functions/ algos. Review collected by and hosted on G2.com.

I love the speed of data processing. Also the Immense Ecosystem of integration with APIs and the huge amount of optimization in memory we can achieve through it.

It is very easy to use and implement with versatile data processing approaches we can achieve through it and also the customer support with community help is great! Review collected by and hosted on G2.com.

It needs a steep learning curve as non-tech persons or beginners may find it very complex.

When the data complexity is huge, it is hard to debug and find bugs in the processed big data.

It is not suitable for small data processing. Review collected by and hosted on G2.com.

Integration with powerful scripting languages (Python, Scala and Java).

Consume available Apache datastore files for developing ML models and quickly deploy to production.

Integration with Knime provided no-code development of ETL pipelines, and merging with Apache datastores allowed us to quickly educate the traditional talent (SQL and Excel-based) to build robust data insights.

Knime platform integration with Spark did not require any additional computing power as it performed all the processing on the Spark infrastructure. Review collected by and hosted on G2.com.

Out of the box, Spark has fewer algorithms for ML models, but we can extend with other programming languages which involves additional effort while implementing with Knime. Review collected by and hosted on G2.com.

1. I really like the concept of RDD in spark as they are immutable.

2. Spark provided lots of system udfs(out of the box) to play with.

3. We can easily debug a spark issue by checking lineage on spark UI Review collected by and hosted on G2.com.

1. Sometimes, out-of-memory errors on spark become very frequent, and a SIGKILL command is invoked without any proper stack trace by spark. This way it becomes very difficult to debug a spark job Review collected by and hosted on G2.com.

Spark makes processing very large data sets possible and also handles these data sets in a fairly quick manner.

Spark seems to be rapidly advancing software.

Spark is one of the trending software in the recent times.

It is a great computing engine for solving complex logics. Review collected by and hosted on G2.com.

Spark seems to be little slow on wide data.

Spark lacks in supporting its users a bit.

Sparks needs some advance ability to understand and structure the modeling of big data. Review collected by and hosted on G2.com.

-It incorporates with powerful scripting languages.

-Easy debugging via UI

-Data Processing is significantly faster than the conventional Hadoop Big Data System because of its in-memory calculations and several other optimizations. Review collected by and hosted on G2.com.

There may be Out of Memory mistakes as a result of In Memory calculations.

Query Execution time is somewhat high however it is expected, but can be optimized upto certain levels. Review collected by and hosted on G2.com.

Spark's in-memory computations makes it suparfast over traditional Map-Reduce jobs.

Also spark has capacity to not just read from hdfs but also from any storage. Spark streaming is best for streaming applications.

Dataframes are also the best part of spark. Review collected by and hosted on G2.com.

There is nothing as such to dislike about spark. Review collected by and hosted on G2.com.

In memory computation and storage levels

GRAPHX and sparkmlib to execute ML jobs in distributed env

Support for multiple languages Review collected by and hosted on G2.com.

Performance issues for NON scala udfs

Not able to self optimise skewness Review collected by and hosted on G2.com.

Everything was very perfect. It was a great experience overall. All the things were properly in place. Review collected by and hosted on G2.com.

Nothing as such. Everything was perfect. Review collected by and hosted on G2.com.