Python SDK 2.0 and resumable uploads for large datasets
Uploads over 100 MB now resume automatically from the last acknowledged byte, and the new Python SDK unifies sync and async clients under one set of types.
By SDK Team
Resumable uploads via tus
Publishing a large robotics dataset over a flaky connection used to mean starting over every time the socket dropped. Not anymore. Uploads over 100 MB now use the tus resumable upload protocol automatically. If the connection drops, the transfer resumes from the last acknowledged byte.
There are no client changes required when you use the official SDKs — resumability is on by default. Under the hood, the SDK negotiates the upload, tracks offsets, and retries transparently.
Python SDK 2.0
The Python SDK 2.0 ships sync and async clients that share the same types, so you can move between them without relearning the surface. Streaming downloads now yield safetensors handles directly, which means you can memory-map weights instead of buffering entire files.
The upgrade is backwards-compatible: the previous surface is preserved under nodedata.legacy, so existing scripts keep running while you migrate at your own pace.
- Automatic resumable uploads for anything over 100 MB.
- Unified types across sync and async clients.
- Streaming downloads yield safetensors handles.
- Backwards-compatible re-exports under nodedata.legacy.
Full details are in the publishing-datasets and SDKs guides. Upgrade with your package manager of choice and you're done.
