syft.serde.msgpack.native_serde

This file exists to provide one common place for all serialisation and simplify_ and _detail for all native python objects.

Module Contents

syft.serde.msgpack.native_serde._simplify_collection(worker: AbstractWorker, my_collection: Collection, shallow: bool = False) → Tuple

This function is designed to search a collection for any objects which may need to be simplified (i.e., torch tensors). It iterates through each object in the collection and calls _simplify on it. Finally, it returns the output as the tuple of simplified items of the input collection. This function is used to simplify list, set, and tuple. The reverse function, which undoes the functionality of this function is different for each of these types: _detail_collection_list, _detail_collection_set, _detail_collection_tuple.

Parameters

my_collection (Collection) – a collection of python objects

Returns

a tuple with simplified objects.

Return type

Tuple

syft.serde.msgpack.native_serde._detail_collection_list(worker: AbstractWorker, my_collection: Tuple, shallow: bool = False) → Collection

This function is designed to operate in the opposite direction of _simplify_collection. It takes a tuple of simple python objects and iterates through it to determine whether objects in the collection need to be converted into more advanced types. In particular, it converts binary objects into torch Tensors where appropriate.

Parameters

  • worker – the worker doing the deserialization

  • my_collection (Tuple) – a tuple of simple python objects (including binary).

Returns

a collection of the same type as the input where the objects

in the collection have been detailed.

Return type

Collection

syft.serde.msgpack.native_serde._detail_collection_set(worker: AbstractWorker, my_collection: Tuple, shallow: bool = False) → Collection

This function is designed to operate in the opposite direction of _simplify_collection. It takes a tuple of simple python objects and iterates through it to determine whether objects in the collection need to be converted into more advanced types. In particular, it converts binary objects into torch Tensors where appropriate.

Parameters

  • worker – the worker doing the deserialization

  • my_collection (Tuple) – a tuple of simple python objects (including binary).

Returns

a collection of the same type as the input where the objects

in the collection have been detailed.

Return type

Collection

syft.serde.msgpack.native_serde._detail_collection_tuple(worker: AbstractWorker, my_tuple: Tuple, shallow: bool = False) → Tuple

This function is designed to operate in the opposite direction of _simplify_collection. It takes a tuple of simple python objects and iterates through it to determine whether objects in the collection need to be converted into more advanced types. In particular, it converts binary objects into torch Tensors where appropriate. This is only applicable to tuples. They need special handling because msgpack is encoding a tuple as a list.

Parameters

  • worker – the worker doing the deserialization

  • my_tuple (Tuple) – a collection of simple python objects (including binary).

Returns

a collection of the same type as the input where the objects

in the collection have been detailed.

Return type

tuple

syft.serde.msgpack.native_serde._simplify_dictionary(worker: AbstractWorker, my_dict: Dict, shallow: bool = False) → Tuple

This function is designed to search a dict for any objects which may need to be simplified (i.e., torch tensors). It iterates through each key, value in the dict and calls _simplify on it. Finally, it returns the output tuple of tuples containing key/value pairs. The reverse function to this function is _detail_dictionary, which undoes the functionality of this function.

Parameters

my_dict – A dictionary of python objects.

Returns

Tuple containing tuples of simplified key/value pairs from the

input dictionary.

Return type

Tuple

syft.serde.msgpack.native_serde._detail_dictionary(worker: AbstractWorker, my_dict: Tuple, shallow: bool = False) → Dict

This function is designed to operate in the opposite direction of _simplify_dictionary. It takes a dictionary of simple python objects and iterates through it to determine whether objects in the collection need to be converted into more advanced types. In particular, it converts binary objects into torch Tensors where appropriate.

Parameters

  • worker – the worker doing the deserialization

  • my_dict (Tuple) – a simplified dictionary of simple python objects (including binary).

Returns

a collection of the same type as the input where the objects

in the collection have been detailed.

Return type

Dict

syft.serde.msgpack.native_serde._simplify_str(worker: AbstractWorker, obj: str) → tuple
syft.serde.msgpack.native_serde._detail_str(worker: AbstractWorker, str_tuple: tuple) → str
syft.serde.msgpack.native_serde._simplify_range(worker: AbstractWorker, my_range: range) → Tuple[int, int, int]

This function extracts the start, stop and step from the range.

Parameters

my_range (range) – a range object

Returns

a list defining the range parameters [start, stop, step]

Return type

list

Examples

range_parameters = _simplify_range(range(1, 3, 4))

assert range_parameters == [1, 3, 4]

syft.serde.msgpack.native_serde._detail_range(worker: AbstractWorker, my_range_params: Tuple[int, int, int]) → range

This function extracts the start, stop and step from a tuple.

Parameters

  • worker – the worker doing the deserialization (only here to standardise signature with other _detail functions)

  • my_range_params (tuple) – a tuple defining the range parameters [start, stop, step]

Returns

a range object

Return type

range

Examples

new_range = _detail_range([1, 3, 4])

assert new_range == range(1, 3, 4)

syft.serde.msgpack.native_serde._simplify_ellipsis(worker: AbstractWorker, e: Ellipsis) → Tuple[bytes]
syft.serde.msgpack.native_serde._detail_ellipsis(worker: AbstractWorker, ellipsis: bytes) → Ellipsis
syft.serde.msgpack.native_serde._simplify_slice(worker: AbstractWorker, my_slice: slice) → Tuple[int, int, int]

This function creates a list that represents a slice.

Parameters

my_slice (slice) – a python slice

Returns

a list holding the start, stop and step values

Return type

tuple

Examples

slice_representation = _simplify_slice(slice(1,2,3))

syft.serde.msgpack.native_serde._detail_slice(worker: AbstractWorker, my_slice: Tuple[int, int, int]) → slice

This function extracts the start, stop and step from a list.

Parameters

my_slice (tuple) – a list defining the slice parameters [start, stop, step]

Returns

a range object

Return type

range

Examples

new_range = _detail_range([1, 3, 4])

assert new_range == range(1, 3, 4)

syft.serde.msgpack.native_serde._simplify_ndarray(worker: AbstractWorker, my_array: numpy.ndarray) → Tuple[bin, Tuple, Tuple]
This function gets the byte representation of the array

and stores the dtype and shape for reconstruction

Parameters

my_array (numpy.ndarray) – a numpy array

Returns

a list holding the byte representation, shape and dtype of the array

Return type

list

Examples

arr_representation = _simplify_ndarray(numpy.random.random([1000, 1000])))

syft.serde.msgpack.native_serde._detail_ndarray(worker: AbstractWorker, arr_representation: Tuple[bin, Tuple, str]) → numpy.ndarray
This function reconstruct a numpy array from it’s byte data, the shape and the dtype

by first loading the byte data with the appropiate dtype and then reshaping it into the original shape

Parameters

  • worker – the worker doing the deserialization

  • arr_representation (tuple) – a tuple holding the byte representation, shape

  • dtype of the array (and) –

Returns

a numpy array

Return type

numpy.ndarray

Examples

arr = _detail_ndarray(arr_representation)

syft.serde.msgpack.native_serde._simplify_numpy_number(worker: AbstractWorker, numpy_nb: Union[numpy.int32, numpy.int64, numpy.float32, numpy.float64]) → Tuple[bin, Tuple]
This function gets the byte representation of the numpy number

and stores the dtype for reconstruction

Parameters

numpy_nb (e.g numpy.float64) – a numpy number

Returns

a list holding the byte representation, dtype of the numpy number

Return type

list

Examples

np_representation = _simplify_numpy_number(worker, numpy.float64(2.3)))

syft.serde.msgpack.native_serde._detail_numpy_number(worker: AbstractWorker, nb_representation: Tuple[bin, Tuple, str]) → Union[numpy.int32, numpy.int64, numpy.float32, numpy.float64]
This function reconstruct a numpy number from it’s byte data, dtype

by first loading the byte data with the appropiate dtype

Parameters

  • worker – the worker doing the deserialization

  • np_representation (tuple) – a tuple holding the byte representation

  • dtype of the numpy number (and) –

Returns

a numpy number

Return type

numpy.float or numpy.int

Examples

nb = _detail_numpy_number(nb_representation)

syft.serde.msgpack.native_serde.MAP_NATIVE_SIMPLIFIERS_AND_DETAILERS