neptune 1.0 upgrade guide#

Info

This guide concerns the Neptune Python client library (formerly neptune-client, now neptune ).

This page lists all the major changes to the Neptune client library with the 1.0 release. Here you'll find all the information you need to upgrade your library and update your code.

Renamed packages#

As we formally promote the "new" Neptune API to the main one, the library and package names have changed as follows:

Before	After	Comment
`neptune` (package)	`neptune.legacy`	The legacy API is moved to a different package (and continues to be deprecated).
`neptune.new` (package)	`neptune`	The `neptune` package is now the main API (`neptune.new` works, but this use is deprecated).
`neptune-client` (library)	`neptune`	The Python library name has changed (`neptune-client` still works, but we recommend uninstalling it and instead installing `neptune`).

Installing

No

pip install neptune-client

Yes

pip install neptune

Upgrading from neptune-client

To smoothly upgrade to the 1.0 version of the client library, first uninstall the neptune-client library and then install neptune.

pip uninstall neptune-client

pip install neptune

Importing

No

import neptune.new as neptune

Yes

import neptune

Removals of previously deprecated parts#

The following were already deprecated previously and have been removed with the 1.0 release:

Deprecated functions#

Before	After	Comment
`get_run_url()`	`get_url()`	Replaced with universal method for fetching the URL of any Neptune object type.
`neptune.get_project()`	`neptune.init_project()`	Using the `mode="read-only"` parameter makes `init_project()` behave just like `get_project()`.
`neptune.init()`	`neptune.init_run()`	Now that we have model and project objects to work with as well, the function names are clearer when they express the object that's being initialized.
`neptune.get_last_run()`	–	This was a mainly a helper function for transitioning from the legacy API to the new one.

Replacement for log()

Instead of log(), you can use append() and extend() to log series of values or files.

However, we won't deprecate log() for the time being.

Deprecated parameters and constants#

Before	After	Comment
(in `init` functions) `run`, `model`, `version`	`with_id`	When initializing an existing Neptune object, use the universal parameter `with_id` to pass the object's Neptune ID.
(for existing project name) `name`	`project`	Whenever you pass a project name as the argument, the argument name is `project`. The only exception is when you actually create the project, in which case the `name` parameter is used to pass the name of the new project.
(for existing workspace name) `name`	`workspace`	Whenever you pass a workspace name as the argument, the argument name is `workspace`.
`neptune.ANONYMOUS`	`neptune.ANONYMOUS_API_TOKEN`	–
`neptune.NEPTUNE_RUNS_DIRECTORY`	`neptune.NEPTUNE_DATA_DIRECTORY`	–

Deprecated CLI arguments#

The following Neptune Command Line Interface argument is removed:

Argument	Command	Description
`--run`	`neptune sync`	Used to specify which runs or other Neptune objects should be synchronized.

No

neptune sync --run jackie/sandbox/SAN-42

Yes

neptune sync --object jackie/sandbox/SAN-42

Changes in behavior#

Some functions require keyword arguments#

It has been possible to use many arguments positionally ‐ that is, passing them to Neptune functions without using the names of the arguments.

With 1.0, we've changed package-level functions (neptune and management) to generally require named arguments.

No

run = neptune.init_run("workspace-name/project-name")

from neptune import management

management.trash_objects(project_name, runs_to_trash)

Yes

run = neptune.init_run(project="workspace-name/project-name")

from neptune import management

management.trash_objects(project=project_name, ids=runs_to_trash)

The methods you typically use to actually log metadata are not affected. For example, you will not have to change your upload("data_sample.csv") calls to upload(value="data_sample.csv").

System metrics not tracked in interactive sessions#

By default, monitoring of system metrics is turned off for interactive Python kernels, such as Jupyter notebooks. This includes logging of hardware consumption and standard streams (stdout and stderr).

To turn it on, you need to explicitly set the related initialization parameters to a value of True:

>>> import neptune
>>> run = neptune.init_run(
...     capture_hardware_metrics=True,
...     capture_stderr=True,
...     capture_stdout=True,
... )

In Python scripts, monitoring is still turned on by default.

Relation to pricing

We changed this with neptune 1.0.0 to avoid unintentionally using up logging hours in the background during interactive sessions.

This is no longer relevant for workspaces that follow new pricing model, which is based on the number of active projects.

Better support for parallel or distributed jobs#

Monitoring works with multiple processes out of the box: Instead of every process trying to log metrics to the same namespace ("monitoring"), we split the metrics under unique sub-namespaces per process.

We've also added three new fields under each monitoring namespace:

"monitoring/<unique hash>/pid" – process ID
"monitoring/<unique hash>/tid" – thread ID
"monitoring/<unique hash>/hostname" – host name

Before

run/
|-- monitoring/
    |-- cpu         <- Each process tries to log CPU metrics under this field
    |-- stdout      <- Each process tries to log stdout under this field
    |-- ...

After

run/
|-- monitoring
    | -- <unique hash>
        |-- cpu
        |-- stdout
        |-- hostname    <- New field: Host name
        |-- pid         <- New field: Process ID
        |-- tid         <- New field: Thread ID
        |-- ...
    |-- <unique hash>
        |-- cpu
        |-- stdout
        |-- hostname
        |-- ...

As before 1.0, you can also provide your custom monitoring namespace name by passing it to the monitoring_namespace argument at initialization:

Example

import neptune

run = neptune.init_run(monitoring_namespace="my_namespace_name")

This custom name will replace the monitoring/<unique hash> pattern. If you're logging metrics from multiple processes, ensure that your custom name is unique for each process.

Unified run states#

Run states are displayed as "Active" and "Inactive" in the web app, but have mapped to "running" and "idle" in the API. We've unified the states to "Active" and "Inactive" everywhere.

When queried through the API, the states are returned as "Active" or "Inactive".
When querying runs based on state, use the values active and inactive (case-insensitive) instead of running and idle.

Example: Fetching inactive runs from a project#

No

project = neptune.init_project(project="workspace/project")
runs_table_df = project.fetch_runs_table(state="idle").to_pandas()

Yes

project = neptune.init_project(project="workspace/project")
runs_table_df = project.fetch_runs_table(state="inactive").to_pandas()

No more implicit casting to String#

We'll no longer cast unsupported types to String when using assignment (=), append(), extend(), or log().

What does this mean?

Previously, if you attempted to log a value that is not among the supported neptune.types (such as a tuple) Neptune silently logged it as a string. As of 1.0, you're expected to use str(value) if you want an object to be logged as a string.

If you attempt to log a value of an unsupported type to a given field, Neptune will skip that value and print a warning.

Before

Works

>>> run["tuple"] = ("hi", 1)
>>> run["tuple"]
<String field at "tuple">

Works

>>> run["tuple"] = str(("hi", 1))
>>> run["tuple"]
<String field at "tuple">

After

Does not work

>>> run["tuple"] = ("hi", 1)
<warning>
>>> # tuple isn't supported

Works

>>> run["tuple"] = str(("hi", 1))
>>> run["tuple"]
<String field at "tuple">

For dictionaries that include unsupported types, you can use the stringify_unsupported() function to ensure that all nested values of an unsupported type are stored as strings.

BeforeAfter

Works

>>> run["complex_dict"] = {"tuple": ("hi", 1), "metric": 0.87}
>>> run["complex_dict/tuple"]
<String field at "complex_dict/tuple">

Works

>>> from neptune.new.utils import stringify_unsupported
>>> run["complex_dict"] = stringify_unsupported({"tuple": ("hi", 1), "metric": 0.87})
>>> run["complex_dict/tuple"]
<String field at "complex_dict/tuple">

Does not work

>>> run["complex_dict"] = {"tuple": ("hi", 1), "metric": 0.87}
<warning>
>>> # Doesn't work because the tuple isn't a supported type

Works

>>> from neptune.utils import stringify_unsupported
>>> run["complex_dict"] = stringify_unsupported({"tuple": ("hi", 1), "metric": 0.87})
>>> run["complex_dict/tuple"]
<String field at "complex_dict/tuple">

For other scenarios where you may need to work around unsupported types, see Help ≫ Unsupported type

What do I need to change in my setup?#

Performing the upgrade

Important: To smoothly upgrade to the 1.0 version of the client library, first uninstall the neptune-client library and then install neptune.

pip uninstall neptune-client

pip install neptune

To successfully update your code for neptune 1.0:

In any Neptune import statements, change neptune.new to just neptune.¹
If you're providing arguments positionally in your function calls, check if you should change them to include the argument names.
If you're still using deprecated functions or parameters, make sure to update those.
- In particular, when passing the name of your existing Neptune project or workspace to a function, use the project or workspace argument instead of name.
Instead of log(), use append() (or extend(), if you're appending multiple values at a time).
If you're using Neptune in interactive Python sessions and want to monitor hardware usage or system metrics, explicitly set the related init parameters to True.
If you have been relying on unsupported types being logged as strings, either explicitly log those as strings or look for a different way to log them with available Neptune methods.
If you're querying and filtering runs based on state, change the following:
- idle to inactive
- running to active

If you're still using the legacy API, you'll need to change your import statement to: import neptune.legacy as neptune ↩