Now when I run grid search over some parameters, every time Neptune starts new experiment inside grid search, it runs it with current version of code. This way I am not able to work on my code after running a grid search, because when I change something in the code, following experiments will be running with different code than previous ones, and the results will be useless.
Usually, when I run grid search, it takes a lot of time to compute, so during this time I would be pleased to be able to work on my code. It would be nice, if every experiment in grid search was running with the same version of code (and of course different parameter set) as it was when I started grid search.
What do you thing? In my opinion that could help users avoid a lot of confusion.