syft.generic.frameworks.hook.hook_args

Module Contents

syft.generic.frameworks.hook.hook_args.hook_method_args_functions
syft.generic.frameworks.hook.hook_args.hook_method_response_functions
syft.generic.frameworks.hook.hook_args.get_tensor_type_functions
syft.generic.frameworks.hook.hook_args.base_types
syft.generic.frameworks.hook.hook_args.one
syft.generic.frameworks.hook.hook_args.get_child
syft.generic.frameworks.hook.hook_args.type_rule
syft.generic.frameworks.hook.hook_args.forward_func
syft.generic.frameworks.hook.hook_args.backward_func
syft.generic.frameworks.hook.hook_args.ambiguous_methods
syft.generic.frameworks.hook.hook_args.ambiguous_functions
syft.generic.frameworks.hook.hook_args.register_type_rule(new_type_rules: Dict)
syft.generic.frameworks.hook.hook_args.register_forward_func(new_forward_rules: Dict)
syft.generic.frameworks.hook.hook_args.register_backward_func(new_backward_rules: Dict)
syft.generic.frameworks.hook.hook_args.register_ambiguous_method(*method)
syft.generic.frameworks.hook.hook_args.register_ambiguous_function(*function)
syft.generic.frameworks.hook.hook_args.default_backward_func(tensorcls)
syft.generic.frameworks.hook.hook_args.default_register_tensor(*tensorcls)
syft.generic.frameworks.hook.hook_args.unwrap_args_from_method(attr, method_self, args, kwargs)

Method arguments are sometimes simple types (such as strings or ints) but sometimes they are custom Syft tensors such as wrappers (i.e. FrameworkTensor), LoggingTensor or some other tensor type. Complex types (which have a .child attribute) need to have arguments converted from the arg to arg.child so that the types match as the method is being called down the chain. To make this efficient, we cache which args need to be replaced with their children in a dictionary called hook_method_args_functions. However, sometimes a method (an attr) has multiple different argument signatures, such that sometimes arguments have .child objects and other times they don’t (such as x.div(), which can accept either a tensor or a float as an argument). This invalidates the cache, so we need to have a try/except which refreshes the cache if the signature triggers an error.

Parameters

  • attr (str) – the name of the method being called

  • method_self – the tensor on which the method is being called

  • args (list) – the arguments being passed to the method

  • kwargs (dict) – the keyword arguments being passed to the function (these are not hooked ie replace with their .child attr)

syft.generic.frameworks.hook.hook_args.unwrap_args_from_function(attr, args, kwargs, return_args_type=False)

See unwrap_args_from_method for details

Parameters

  • attr (str) – the name of the function being called

  • args (list) – the arguments being passed to the function

  • kwargs (dict) – the keyword arguments being passed to the function (these are not hooked ie replace with their .child attr)

  • return_args_type (bool) – return the type of the tensors in the

  • arguments (original) –

Returns

  • the arguments where all tensors are replaced with their child

  • the type of this new child

(- the type of the tensors in the arguments)

syft.generic.frameworks.hook.hook_args.build_unwrap_args_from_function(args, return_tuple=False)

Build the function f that hook the arguments: f(args) = new_args

syft.generic.frameworks.hook.hook_args.hook_response(attr, response, wrap_type, wrap_args={}, new_self=None)

When executing a command, arguments are inspected and all tensors are replaced with their child attribute until a pointer or a framework tensor is found (for example an argument could be a framework wrapper with a child being a LoggingTensor, with a child being a framework tensor). When the result of the command is calculated, we need to rebuild this chain in the reverse order (in our example put back a LoggingTensor on top of the result and then a framework wrapper). To make this efficient, we cache which elements of the response (which can be more complicated with nested tuples for example) need to be wrapped in a dictionary called hook_method_response_functions. However, sometimes a method (an attr) has multiple different response signatures. This invalidates the cache, so we need to have a try/except which refreshes the cache if the signature triggers an error.

Parameters

  • attr (str) – the name of the method being called

  • response (list or dict) – the arguments being passed to the tensor

  • wrap_type (type) – the type of wrapper we’d like to have

  • wrap_args (dict) – options to give to the wrapper (for example the

  • for the precision tensor) (precision) –

  • new_self – used for the can just below of inplace ops

syft.generic.frameworks.hook.hook_args.build_wrap_reponse_from_function(response, wrap_type, wrap_args)

Build the function that hook the response.

Example

p is of type Pointer f is the hook_response_function then f(p) = (Wrapper)>Pointer

syft.generic.frameworks.hook.hook_args.build_rule(args)

Inspect the args object to find framework or syft tensor arguments and return a rule whose structure is the same as the args object, with 1 where there was (framework or syft) tensors and 0 when not (ex: number, str, …)

Example

in: ([tensor(1, 2), Pointer@bob], 42) out: ([1, 1], 0)

syft.generic.frameworks.hook.hook_args.build_unwrap_args_with_rules(args, rules, return_tuple=False)

Build a function given some rules to efficiently replace in the args object syft tensors with their child (but not pointer as they don’t have .child), and do nothing for other type of object including framework tensors, str, numbers, bool, etc. Pointers trigger an error which can be caught to get the location for forwarding the call.

Parameters

  • args (tuple) – the arguments given to the function / method

  • rules (tuple) – the same structure but with boolean, true when there is a tensor

  • return_tuple (bool) – force to return a tuple even with a single element

Returns

a function that replace syft arg in args with arg.child

syft.generic.frameworks.hook.hook_args.build_get_tensor_type(rules, layer=None)

Build a function which uses some rules to find efficiently the first tensor in the args objects and return the type of its child.

Parameters

  • rules (tuple) – a skeleton object with the same structure as args but each tensor is replaced with a 1 and other types (int, str) with a 0

  • layer (list or None) – keep track of the path of inspection: each element in the list stand for one layer of deepness into the object, and its value for the index in the current layer. See example for details

Returns

a function returning a type

Example

Understanding the layer parameter obj = (a, [b, (c, d)], e) the layer position is for: a: [0] b: [1, 0] c: [1, 1, 0] d: [1, 1, 1] e: [2]

Global behaviour example rules = (0, [1, (0, 0), 0) - First recursion level

0 found -> do nothing list found -> recursive call with layer = [1]

  • Second recursion level 1 found -> update layer to [1, 0]

    build the function x: type(x[1][0]) break

  • Back to first recursion level save the function returned in the lambdas list 0 found -> do nothing exit loop return the first (and here unique) function

syft.generic.frameworks.hook.hook_args.one_layer(idx1)
syft.generic.frameworks.hook.hook_args.two_layers(idx1, idx2)
syft.generic.frameworks.hook.hook_args.three_layers(idx1, *ids)
syft.generic.frameworks.hook.hook_args.four_layers(idx1, *ids)
syft.generic.frameworks.hook.hook_args.get_element_at
syft.generic.frameworks.hook.hook_args.build_wrap_response_with_rules(response, rules, wrap_type, wrap_args, return_tuple=False, return_list=False)

Build a function given some rules to efficiently replace in the response object syft or framework tensors with a wrapper, and do nothing for other types of object including , str, numbers, bool, etc.

Parameters

  • response – a response used to build the hook function

  • rules – the same structure objects but with boolean, at true when is replaces a tensor

  • return_tuple – force to return a tuple even with a single element

Response:

a function to “wrap” the response

syft.generic.frameworks.hook.hook_args.zero_fold(*a, **k)
syft.generic.frameworks.hook.hook_args.one_fold(return_tuple, **kwargs)
syft.generic.frameworks.hook.hook_args.two_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.three_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.four_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.five_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.six_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.seven_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.eight_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.many_fold(lambdas, args, **kwargs)
syft.generic.frameworks.hook.hook_args.typed_identity(a)

We need to add typed identity for arguments which can be either number or tensors. If the argument changes from an int to a tensor, the assertion error triggered by typed_identity will be caught and a new signature will be computed for the command.

syft.generic.frameworks.hook.hook_args.register_response_functions
syft.generic.frameworks.hook.hook_args.register_response(attr: str, response: object, response_ids: object, owner: AbstractWorker) → object

When a remote worker execute a command sent by someone else, the response is inspected: all tensors are stored by this worker and a Pointer tensor is made for each of them.

To make this efficient, we cache which elements of the response (which can be more complicated with nested tuples for example) in the dict register_response_functions

However, sometimes a function (an attr) has multiple different response signatures. This invalidates the cache, so we need to have a try/except which refreshes the cache if the signature triggers an error.

Parameters

  • attr (str) – the name of the function being called

  • response (object) – the response of this function

  • owner (BaseWorker) – the worker which registers the tensors

syft.generic.frameworks.hook.hook_args.build_register_response_function(response: object) → Callable

Build the function that registers the response and replaces tensors with pointers.

Example

(1, tensor([1, 2]) is the response f is the register_response_function then f(p) = (1, (Wrapper)>Pointer)

syft.generic.frameworks.hook.hook_args.register_tensor(tensor: FrameworkTensorType, owner: AbstractWorker, response_ids: List = list())

Registers a tensor.

Parameters

  • tensor – A tensor.

  • owner – The owner that makes the registration.

  • response_ids – List of ids where the tensor should be stored and each id is pop out when needed.

syft.generic.frameworks.hook.hook_args.build_register_response(response: object, rules: Tuple, return_tuple: bool = False) → Callable

Build a function given some rules to efficiently replace in the response object framework tensors with a pointer after they are registered, and do nothing for other types of object including , str, numbers, bool, etc.

Parameters

  • response – the response

  • rules – the rule specifying where the tensors are

  • return_tuple – force to return a tuple even with a single element

Returns

The function to apply on generic responses