WhyLabs Pros and Cons

The core functionality is explicitly important when you are having AI/ML solutions running on production. Instead of spending time on manual checks (that can be prone to error) you can set up monitors to inform you if something is going wrong. It is flexible enough to build your cusotm metric and monitors which allows for versatile usage.

What is highly important to me is the contact with WhyLabs team. Their support is superb which results in fact that even if something is difficult to implement, I know I have people who will help me get the best out of the platform. Review collected by and hosted on G2.com.

Building custom monitors and metrics is a chellanging experience. I struggle sometimes with code issues so better FAQ or documentation could be helpfull to avoid them. Review collected by and hosted on G2.com.

One of the standout features of WhyLabs is its robust data observability capabilities. It provides continuous monitoring of data pipelines and ML models, allowing teams to quickly identify issues like data drift, model degradation, and training-serving skew. The platform's privacy-preserving integration ensures that data can be analyzed without moving or duplicating it, which is critical for maintaining security and privacy in sensitive industries like healthcare and finance​ Review collected by and hosted on G2.com.

One potential drawback of WhyLabs is its relatively limited user reviews and feedback due to its newness in the market, making it harder for potential users to gauge its real-world performance at scale. This lack of detailed reviews can raise concerns about its maturity and support infrastructure​.Additionally, since it’s a newer platform, some advanced features might still be in development, and there could be steep learning curves for teams unfamiliar with observability tools in machine learning​. Review collected by and hosted on G2.com.

* The customer support is very helpful and proactive

* Tool allows for easy ingestion of big number of features and setting up initial monitoring on them

* We can use it to monitor both: input quality and model performance

* The alerts can be raised to specific group of users via specific channels (email/slack), which is helpful Review collected by and hosted on G2.com.

* It can be challenging to setup the monitoring in the correct way when it comes to sensitivty - it requires a lot of trial and error

* Some actions are not possible via UI and require specific API calls

* Documentation can be hard to navigate Review collected by and hosted on G2.com.

Great to collaborate with; very responsive; really appreciate their OHs to help out with issues that pop up; many out-of-the-box solutions for different kinds of ML models which really helped us out given the wide variety of ML models we run at the company. Review collected by and hosted on G2.com.

Nothing major to mention! We got everything resolved and the team was very helpful. Review collected by and hosted on G2.com.

The foremost aspect that stood out for us was the ease of integration. It was quite straightforward, almost as same as the sample code to put LangKit into our LLM pipeline and upload it to WhyLabs. The getting-started docs were clear and concise. Even for companies with limited resources like ours, WhyLabs AI Observatory proves to be an accessible and easily adaptable solution. Review collected by and hosted on G2.com.

The data visualization tools are highly capable, although there is some learning curve to master it. However, their customer success and support team are responsive, helpful, and have consistently gone above and beyond to ensure that we utilize the platform to its full potential. Review collected by and hosted on G2.com.

WhyLabs is the second observability platform I have ever used, and I can say the core features I like about the platform is that it is easy to set up and implement the features, the checks and metrics were already pre-loaded so I did not need to do much in configuring the application, and monitoring was not difficult to get started with. It also integrates well with the serving and data libraries we used for the production tutorial setup. Review collected by and hosted on G2.com.

Nothing so far, I only experienced a stability issues once (sometime in 2022), but support was able to help me quickly fix it. Review collected by and hosted on G2.com.

I've used WhyLabs for a few weeks and I was extremely pleased with it!

I will evaluate some dimensions of the tool that summarize my experience with it.

Easy Data Ingestion:

The ingestion API is straightforward to use and supports multiple connectors such as BigQuery, Databricks, and Spark, making data importation easy. Whylabs' use of Data Profiling ensures fast and secure data processing, eliminating the need to upload entire datasets, and making all the process very secure, since your data doesn't leave your servers.

Reliable Data Features:

Whylabs delivers all standard feature metrics accurately. Tracking data and model drift is very straightforward using Monitors.

Also, the platform supports custom metrics creation during or after ingestion.

Grouping by variables (segments) works well but must be defined during ingestion. Then you can analyze dataset features and track model performance per segment.

Flexible Monitors:

The monitoring system in Whylabs is highly adaptable and user-friendly, covering multiple variables with ease.

Monitors are easy to set up via the UI or JSON import, with summarized notifications for each monitor, keeping users informed without overwhelming them.

Additionally, monitors are JSON serializable, which is very helpful since you can track them with version control.

User-Friendly Usability:

Whylabs have a clean and intuitive UI, simplifying navigation for users.

While some advanced features may require programming knowledge, most tasks can be accomplished within the UI.

Thanks to data profiling, Whylabs delivers speedy performance without compromising on accuracy.

Solid Documentation:

The documentation provided by Whylabs is comprehensive and easy to understand, enabling users to make the most of the platform.

Pricing:

It's simply cheaper than its competition while having top notch features.

Customer Support:

They are always very helpful, answering all our questions and having several calls showcasing us different uses cases directly on the platform.

Overall, Whylabs offers a straightforward, efficient and affordable solution for monitoring Machine Learning models, with easy data ingestion, reliable feature analysis, and flexible monitoring options. Review collected by and hosted on G2.com.

There are a few cons:

- Dashboards are in beta, and while functional, they lack polish in terms of user interface. They are working actively on this, so probably a few months after this review this may be already fixed.

- Defining groupings by variables must be done at ingestion time, limiting flexibility for post-ingestion analysis.

That being said, they are very open to feedback and they may change or add features based on your needs.

In our case, dashboards were important and they are working on them. Review collected by and hosted on G2.com.

My experience with the WhyLabs Platform was enlightening, offering intuitive insights into AI model behavior and enhancing my understanding of AI observability. The best part of about the interface is its user friendly interface and comprehensive suite of analytics which provide valuable insights into AI model behavior and which would also enable a dashboard for data monitoring, model features. It also facilitates real time monitoring, ultimately enhancing efficiency of AI development processes. It also gives you the leverage to customise the model features as per the project requirement. Review collected by and hosted on G2.com.

Few advanced customisation options for analytics is not available as of now, but as informed it would be available soon as an update in the platform. So that solves the downside down the line. Review collected by and hosted on G2.com.

WhyLabs is so useful for debugging, maintaining, and developing production ML systems. It's portable, highly configurable, and provides unified visibility across a broad range of systems/services in a production ML ecosystem. It dramatically improves ML devops and is incredibly easy to get started. It integrates very easily with all the systems I've used, implementation is a breeze, and there's very low friction to get started. I've caught tons of issues using it. Review collected by and hosted on G2.com.

I don't really have any complaints about it, it does its thing incredibly well for my use cases. Review collected by and hosted on G2.com.

The UI is very user friendly and the support team is extremely responsive and helpful. Review collected by and hosted on G2.com.

Some features such as deleting a profile or changing the data type of a feature can only be done through the API. Review collected by and hosted on G2.com.