streaming#
MosaicML Streaming Datasets for cloud-native model training.
Classes
Writes a streaming CSV dataset. |
|
Writes a streaming JSON dataset. |
|
A streaming dataset whose shards reside locally as a pytorch Dataset. |
|
Writes a streaming MDS dataset. |
|
A streaming data loader. |
|
A streaming pytorch IterableDataset that is also resumable mid-epoch. |
|
Writes a streaming TSV dataset. |
|
Writes a streaming XSV dataset. |