Databricks Data Intelligence Platform Reviews & Product Details

As a data engineer who has been working with Databricks for the past two years, I can honestly say the platform has completely transformed the way we approach data engineering projects. Before Databricks, me and my team often faced challenges with managing large datasets and ensuring smooth collaboration between data engineers and data scientists. There were times when workflows felt disjointed, and troubleshooting issues across different tools consumed a lot of our time.

Databricks has changed all of that. The collaborative notebooks feature, in particular, has been a game-changer. I can now work seamlessly with data scientists in real-time, troubleshooting issues and iterating on solutions much faster. For example, during a recent project, we were able to refine a machine learning model within days, thanks to the ability to easily share notebooks and quickly run experiments together. This level of collaboration used to take weeks with previous tools.

The auto-scaling feature has been a lifesaver. I vividly remember struggling with performance issues when processing large datasets on our old infrastructure. Now, Databricks automatically adjusts resources based on workload, so we never have to worry about managing compute power. This has drastically cut down on processing times. For instance, a data transformation job that used to take hours now finishes in a fraction of the time, allowing us to deliver projects faster.

Delta Lake has also been invaluable. Before we started using it, data consistency and quality were constant concerns, especially when dealing with large and varied data sources. Now, with Delta Lake, we can trust that our data is not only high quality but also easily accessible and queryable. One particular example was when we had to rebuild a complex dataset pipeline. Delta Lake allowed us to work with incremental data updates, making the process much more efficient and reliable.

In short, Databricks has greatly reduced development time and improved the overall quality of our deliveries. It’s helped me streamline complex workflows, improve collaboration across teams, and most importantly, deliver data-driven solutions faster and with greater confidence. Review collected by and hosted on G2.com.

Cost Optimisation - While I appreciate the granular billing information provided, predicting costs for large projects or shared environments can still feel opaque. Many teams struggle to control runaway costs from idle clusters or suboptimal configurations. Introducing smarter autoscaling and recommendations tailored to our workloads would be invaluable. For instance, alerts for "idle clusters" or "cost hotspots" in our environment could proactively save budgets and improve efficiency.

Simplified Governance and Security - Managing access at fine-grained levels can be cumbersome. For example, controlling who can view versus who can execute a notebook or job often requires workarounds. Audit logs are excellent, but making sense of them for actionable insights sometimes feels like solving a puzzle. Enhanced attribute-based access control (ABAC) and more intuitive UI-based controls for permission management would greatly streamline operations.

User Experience - The collaborative notebook interface is one of Databricks' standout features, yet there are areas where it could be smoother. Collaboration is sometimes hindered when two users edit the same notebook. Version control feels basic compared to Git-based systems. Debugging within notebooks, especially for non-Python workloads, could use significant improvement. Adding inline commenting, conflict resolution tools, and robust debugging features would take the platform to the next level. A workspace-level activity feed to show what’s happening in shared projects would also be immensely helpful.

Workflow Automation - Include AI-driven insights for optimizing workflows (e.g., spotting bottlenecks or inefficiencies). Enable easier integration with external workflow automation tools. Review collected by and hosted on G2.com.

Databricks data intelligence is really fast and can handle large amounts of data easily, you can run complex Sql quries on huge datasets in seconds without needing to worry about managing serves or infrastructure, everthing is taken care of for you, like maintance and backups, so you don't have to think about that either. It also works smoothly with other parts of the databricks, making it easier to build software workflows and data piprlines for analyzing and managing your data. Review collected by and hosted on G2.com.

Databricks can be hard for beginners, if they don't know Sql. it can also get expensive and complicated for many users. Review collected by and hosted on G2.com.

Databricks Intelliegence Platform provide a common platform for ETL,Reporting and AI. It's help user to monitor all the data lineage with the help of Unity Catalog and that can be used to create audit report at account level also databricks Genie help user to direct query the tables with simple english words. Review collected by and hosted on G2.com.

Overall it's good for Data Engineer ML Engineer and Analysts but need to work on workflow part that can be more robust and feature reach Review collected by and hosted on G2.com.

I really like Databricks Genie, It helps me to identify the error and give suggestions to resolve it.

Also If I ask to imrove the current code to faster performance Genie's suggestion are helpful. It helps to implement the ETL logic in effiecient way. Review collected by and hosted on G2.com.

Most of the features which I use are helpful but some sql functionalities are not supported such as Update table using join. Review collected by and hosted on G2.com.

- It has an excellent connection with the MLFlow system which guarantees that our clients have access to creation, management, monitoring and progress in Machine Learning.

- It offers professional processes to manage the clients infrastructure and manage all the clusters, all this can be done from the cloud and saves time in collecting data from the clusters.

- We can link several data sources perfectly and simultaneously, this helps collect all the data of our clients in a safe and automated manner, without going through complex data registration process, we can collect a large volume of data easily. Review collected by and hosted on G2.com.

Databricks never gave us any type of negative experience, at all times it was able to offer management, data storage and collection of large volumes of data. With Databricks, our MSP-type functions have improved and have never had any failures collecting all the data of our clients who access IT services. Review collected by and hosted on G2.com.

The user interface is very easy to use and simple of databricks and navigation is also simple which helps you to do the things quickly Review collected by and hosted on G2.com.

Customer support need to improve and they need to recruit people in their team from Asia to overcome the regional language issue Review collected by and hosted on G2.com.

I like the way Databricks does data management, all in one place. What it does is it unites the data engineers and data scientists on the same platform to collaborate and solve problems quickly. Scaling became effortless thanks to the integration with tools like AWS, as well as keeping up with the progress in the notes continues to keep us all on the same page in the notes. It’s helped remove communication issues and it’s helped take care of things faster. Review collected by and hosted on G2.com.

Databricks has one downside and that is the learning curve, especially for people that want to get started with a more complex configuration. We spent some time troubleshooting the setup, and it’s not the easiest one to begin with. The pricing model is also a little unclear, so it isn’t as easy to predict cost as your usage gets bigger. At times that has led to some unforeseen expenses that we might have cut if we had better cost visibility. Review collected by and hosted on G2.com.

Databricks has the great ability to handle streaming data and integrate with Kafka. This is an essential feature for our organisation as we used Databricks to enhance our real time fraud detection system in the financial service sector. This has improved security and reduced fraud activities. The real time processing capabilities were also a crucial feature for our use case. Databricks also support multiple languages development, which is a key benefit for our organisation as we have both Python and Scala developers. Review collected by and hosted on G2.com.

During a critical phase of the project, we faced few challenges while optimising our Spark jobs. The user interface for cluster management could be improved, as we occasionally face delays when scaling clusters to handle large workloads. Review collected by and hosted on G2.com.

My team recently used Databricks to implement a machine learning model for fraud detection. We used the Delta Lake for data preprocessing and insured real time updates from our database. One of the most helpful features in Databricks is the Delta Lake functionality, which ensures data consistency. The platform supports both Python and SQL, which fills the cap between Data engineers and Analysts. This makes it easy for teams to collaborate. Customer support is another highlight as they respond quickly and provide clear guidance. Review collected by and hosted on G2.com.

While integrating Databricks with our existing Azure Data Lake, we faced issues syncing access permissions for multiple datasets. Additionally, their pricing models makes it better suited for large organisations, but for smaller teams scaling up can be expensive. Review collected by and hosted on G2.com.

One thing that strikes me about Databricks is the fact that the platform offers robust methodologies for working with big data while maintaining tremendously high efficiency. I also like how it can be set up to work with multiple jobs at once and is highly beneficial for working on different kinds of datasets in parallel. The integrated collaboration tools enable multiple authors to edit a given document simultaneously hence enhancing the flow of information in the team. Review collected by and hosted on G2.com.

The issue that Databricks could address is the insufficient tools for identifying and addressing problems. While the platform is useful, it’s sometimes not clear where errors lie, meaning that faults might not be as easy to identify. It can result in further time consumed in error analysis in large processes arrangements since these classifications are vague. At times it seems as though there is great strain to searc original solutions thereby slowing the process. Review collected by and hosted on G2.com.