First of all, I need to say that I like neptune.ml a lot. I tend to generate a lot of points (even 1 mln) per metric and no other tool is able to handle them so conveniently and informatively as neptune GUI (e.g., it takes ages to load then on tensorboard). Having said that there is a couple of things I am missing (in the order of importance):
- No possibility to filter the experiments basing on parameters (e.g. dataset name or learning rate).
- No information on how much disk space individual experiments use (could be new a column). If I have it I could decide to remove them selectively. (It took me just 23 experiments to reach 10GB and I wonder why… I have also a suspicion that it is still counting the deleted experiments since when I deleted them from trash my usage statistics did not change).
- No possibility to remove just artifacts from the experiments (to save storage space).
- In many places of the UI the strings are truncated (with “…”) (e.g. in the names of artifacts, plot series)
- When comparing the experiments, I see only the cryptic experiment ids (like XYZ-19). I would prefer to see more information (like the names).
- No way to make an individual plot full-screen.
- Also, when comparing experiments, it is hard to identify the experiment under the mouse pointer (you can do it only by comparing colors, which is suboptimal). TB handles that very nicely.
- Setting a manual scale of OY axis for a given plot would be sometimes helpful (it takes ages to get from 0-1e20 OY range to 0-1e4 range in case of metrics where some divergence happened). Another solution would be to support zoom on axes using mouse scroll (Bokeh does that right).
- I hope to see at some point an ability to group experiments by some property (e.g. name) and then compare groups of experiments (e.g. averages + confidence interval).
I hope that this list could help you develop the service, which I already like.