syft.messaging.plan

Package Contents

class syft.messaging.plan.func2plan(args_shape=None, state=None)

Bases: object

Decorator which converts a function to a plan.

Converts a function containing sequential pytorch code into a plan object which can be sent to any arbitrary worker.

This class should be used only as a decorator.

__call__(self, plan_function)
syft.messaging.plan.method2plan(*args, **kwargs)
class syft.messaging.plan.Plan(name: str = None, state: State = None, include_state: bool = False, is_built: bool = False, operations: List[Operation] = None, placeholders: Dict[Union[str, int], PlaceHolder] = None, forward_func=None, state_tensors=None, id: Union[str, int] = None, owner: sy.workers.BaseWorker = None, tags: List[str] = None, description: str = None)

Bases: syft.generic.object.AbstractObject, syft.generic.object_storage.ObjectStorage

A Plan stores a sequence of torch operations, just like a function.

A Plan is intended to store a sequence of torch operations, just like a function, but it allows to send this sequence of operations to remote workers and to keep a reference to it. This way, to compute remotely this sequence of operations on some remote input referenced through pointers, instead of sending multiple messages you need now to send a single message with the references of the plan and the pointers.

All arguments are optional.

Parameters

  • name – the name of the name

  • state – store the plan tensors like model parameters

  • include_state – if true, implies that the plan is a function, else a class. If true, the state is re-integrated in the args to be accessed within the function

  • is_built – state if the plan has already been built.

  • placeholders – dict of placeholders used in the plan

  • operations – list of commands (called operations)

  • forward_func – the function to be transformed into a plan

  • state_tensors – a tuple of state elements. It can be used to populate a state

  • id – plan id

  • owner – plan owner

  • tags – plan tags

  • description – plan description

get
share
static _create_placeholders(args_shape)
property _known_workers(self)
property location(self)
property readable_plan(self)
parameters(self)

This is defined to match the torch api of nn.Module where .parameters() return the model tensors / parameters

send_msg(self, *args, **kwargs)
request_obj(self, *args, **kwargs)
respond_to_obj_req(self, obj_id: Union[str, int])

Returns the deregistered object from registry.

Parameters

obj_id – A string or integer id of an object to look up.

add_placeholder(self, tensor, node_type=None)

Create and register a new placeholder if not already existing (else return the existing one).

The placeholder is tagged by a unique and incremental index for a given plan.

Parameters

  • tensor – the tensor to replace with a placeholder

  • node_type – Should be “input” or “output”, used to tag like this: #<type>-*

replace_with_placeholders(self, obj, **kw)

Replace in an object all FrameworkTensors with Placeholders

find_placeholders(self, *search_tags)

Search method to retrieve placeholders used in the Plan using tag search. Retrieve all placeholders which have a tag containing at least one search_tag.

Parameters

*search_tags – tuple of tags

Returns

A list of placeholders found

build(self, *args)

Builds the plan.

First, run the function to be converted in a plan in a context which activates the tracing and record the operations in trace.logs

Second, store the result ids temporarily to helper ordering the output placeholders at return time

Third, loop through the trace logs and replace the tensors found in the operations logged by PlaceHolders. Record those operations in plan.operations

Parameters

args – Input arguments to run the plan

copy(self)

Creates a copy of a plan.

__setattr__(self, name, value)

Add new tensors or parameter attributes to the state and register them in the owner’s registry

__call__(self, *args, **kwargs)

Calls a plan execution with some arguments.

When possible, run the original function to improve efficiency. When it’s not, for example if you fetched the plan from a remote worker, then run it from the tape of operations: - Instantiate input placeholders - for each recorded operation, run the operation on the placeholders

and use the result(s) to instantiate to appropriate placeholder.

  • Return the instantiation of all the output placeholders.

run(self, args: Tuple, result_ids: List[Union[str, int]])

Controls local or remote plan execution. If the plan doesn’t have the plan built, first build it using the original function.

Parameters

  • args – Arguments used to run plan.

  • result_ids – List of ids where the results will be stored.

send(self, *locations: AbstractWorker, force=False)

Send plan to locations.

If the plan was not built locally it will raise an exception. If force = true plan is going to be sent either way.

Parameters

  • locations – List of workers.

  • force – A boolean indicating if this operation should be forced.

get_(self)
get_pointers(self)
fix_precision_(self, *args, **kwargs)
float_precision_(self)
share_(self, *args, **kwargs)
create_pointer(self, owner, garbage_collect_data, location=None, id_at_location=None, **kwargs)
__str__(self)

Returns the string representation of Plan.

__repr__(self)
static simplify(worker: AbstractWorker, plan: Plan)

This function takes the attributes of a Plan and saves them in a tuple :param worker: the worker doing the serialization :type worker: AbstractWorker :param plan: a Plan object :type plan: Plan

Returns

a tuple holding the unique attributes of the Plan object

Return type

tuple

static detail(worker: AbstractWorker, plan_tuple: tuple)

This function reconstructs a Plan object given its attributes in the form of a tuple. :param worker: the worker doing the deserialization :param plan_tuple: a tuple holding the attributes of the Plan

Returns

a Plan object

Return type

plan

static bufferize(worker: AbstractWorker, plan: Plan)

This function takes the attributes of a Plan and saves them in a Protobuf message :param worker: the worker doing the serialization :type worker: AbstractWorker :param plan: a Plan object :type plan: Plan

Returns

a Protobuf message holding the unique attributes of the Plan object

Return type

PlanPB

static unbufferize(worker: AbstractWorker, protobuf_plan: PlanPB)

This function reconstructs a Plan object given its attributes in the form of a Protobuf message :param worker: the worker doing the deserialization :param protobuf_plan: a Protobuf message holding the attributes of the Plan

Returns

a Plan object

Return type

plan