syft.frameworks.torch.fl.dataloader

Module Contents

syft.frameworks.torch.fl.dataloader.numpy_type_map

syft.frameworks.torch.fl.dataloader.default_collate(batch)

Puts each data field into a tensor with outer dimension batch size

class syft.frameworks.torch.fl.dataloader._DataLoaderIter(loader, worker_idx)

Bases: object

Iterates once over the DataLoader’s dataset, as specified by the samplers

__len__(self)

_get_batch(self)

__next__(self)

__iter__(self)

stop(self)

class syft.frameworks.torch.fl.dataloader._DataLoaderOneWorkerIter(loader, worker_idx)

Bases: object

Iterates once over the worker’s dataset, as specified by its sampler

_get_batch(self)

__next__(self)

__iter__(self)

stop(self)

class syft.frameworks.torch.fl.dataloader.FederatedDataLoader(federated_dataset, batch_size=8, shuffle=False, num_iterators=1, drop_last=False, collate_fn=default_collate, iter_per_worker=False, **kwargs)

Bases: object

Data loader. Combines a dataset and a sampler, and provides single or several iterators over the dataset.

Parameters

• federated_dataset (FederatedDataset) – dataset from which to load the data.

• batch_size (int, optional) – how many samples per batch to load (default: 1).

• shuffle (bool, optional) – set to True to have the data reshuffled at every epoch (default: False).

• collate_fn (callable, optional) – merges a list of samples to form a mini-batch.

• drop_last (bool, optional) – set to True to drop the last incomplete batch, if the dataset size is not divisible by the batch size. If False and the size of dataset is not divisible by the batch size, then the last batch will be smaller. (default: False)

• num_iterators (int) – number of workers from which to retrieve data in parallel. num_iterators <= len(federated_dataset.workers) - 1 the effect is to retrieve num_iterators epochs of data but at each step data from num_iterators distinct workers is returned.

• iter_per_worker (bool) – if set to true, __next__() will return a dictionary containing one batch per worker

__initialized = False

__iter__(self)

__next__(self)

__len__(self)