Weights & Biases Pros and Cons

Visualization of data from different perspective Review collected by and hosted on G2.com.

Nothing really, maybe it can be good to customize the hardware monitoring interval Review collected by and hosted on G2.com.

- Ability to view in browser

- Ability to share with collaborators

- Ability to add figures to experiment reports Review collected by and hosted on G2.com.

- Annoying bug with visualising segmentation labels - red speckles appear in the label masks which can make it hard to evaluate. Review collected by and hosted on G2.com.

It is easy, I can live preview the results, all the plots are done automatically and smartly. It is a great gain of time. Review collected by and hosted on G2.com.

The user interface is slow but it is acceptable. Retrieving runs data from wandb using the wandb.Api() takes forever (e.g., 30h for around 30 000 runs of hyperparameter in several environments). I would like to be able to download all data from a set of runs selected from filters in a single api call. Since it represents less than 100 mb of data, it should be feasible in a few minutes maximum, right? The documentation is not great. Review collected by and hosted on G2.com.

That it is super easy to log all metrics, loss curves and all different kind of data and to get that data visualized in an interpretable manner. I really like how it is integrated into other frameworks like eg pytorch lightning.

I use W&B almost daily, at least I haven't started a single training run without using W&B ever since I subscribed. Review collected by and hosted on G2.com.

1) I would like to customize the plots and visualizations of the metrics shown even more. That would be nice to be able to do by using a script or something that can be used across multiple projects --> eg I would like to determine (from my python script) the color of all lines in a plot, and I don't know how to do that.

It would just be usefull for communication to team leaders, bosses etc., if they knew that eg. accuracy would also be plotted with a red, dotted line and that recall would always be a thick, blue line or something like that.

2) I have a hard time figuring out how to navigate all artifacts and how to use those. In eg the integration with ultralytics W&B will create an artifact for each epoch, which quickly fills up my storage. However, that is just a minor thing as I have just created a stand alone script to delete artifacts that aren't tagged with "best" etc. Review collected by and hosted on G2.com.

I use it for every project, it became a key tool for me to keep an overview about experiments no matter if or when using LLMs Review collected by and hosted on G2.com.

Working in teams is limited in the free version Review collected by and hosted on G2.com.

The API is self explanatory and the UI is smart Review collected by and hosted on G2.com.

Caching management is not clear, dis not find a way to clean old logs Review collected by and hosted on G2.com.

Much easier than tensorboard. Way easier to get started and then a lot more functionality once you're more experienced.

Easy monitoring of gpu use.

Very easy to compare runs.

Easy to upload tables and images.

Also easy to compare just the runs you want and to save working experiments in a nice format (reports) Review collected by and hosted on G2.com.

The only downsides which I hope will be fixed at some point is you don't have an easy way of deleting just one run.

Would be nice if you could restart a run from the step you left it at as well.

But in the day to day use they're pretty minor and the positives outweigh the downsides. Review collected by and hosted on G2.com.

This tool is perfect for logging information from programs, especially during training. It allows you to see how your model is training from anywhere in the world. I like how you can just dump some raw data onto the platform, and then you can make your graphs and manipulate the data separately from your training loop. Review collected by and hosted on G2.com.

Some features are missing but I am sure they would come if I did a feature request. Review collected by and hosted on G2.com.

Easy to plug in and use. Works well with PyTorch and lightning and easy to compare models when away from local network. Review collected by and hosted on G2.com.

Can slow down training speed and there are some minor bugs sometimes when using on a less well supported device/framework. Mobile UI also slow. Review collected by and hosted on G2.com.

Easy to use, already incorported into major libraries, but still powerful. Review collected by and hosted on G2.com.

The only thing I wish was different is pricing per "tracked hour". For my workflow, this number seems very inflated - I have a few powerful GPUs, and run multiple experiments at a time on each one. This results in "tracked hours" of many multiples of realtime, for each GPU, which doesn't seem right. This is OK for me now as an academic, on the personal plan with unlimited tracked hours, but discourages me from using this for commercial projects in the future, where cost would quickly become prohibitive. Review collected by and hosted on G2.com.