I’ve written a basic batch uploader using the Neptune API to log our experiment data after our learning jobs have run. Now I’d like to push this data into Neptune while the experiments are running.
neptune.log_statistic() understand what experiment the data is supposed to be logged to ? If the neptune.experiment_create() is handled in a parent script and I kick off a bunch of filewatcher subprocesses in the background to log new lines in a logfile to Neptune as they are written, will this work ?
I’m using the Neptune API with Python, but our main application is written in Rust and it doesn’t seem like there’s a Rust binding for the Neptune API at the moment. My current plan then, is to write a “tail -f” equivalent in Python and push new log lines to Neptune as they are written.
Thanks for any help or thoughts.