ops – Some Common Ops and extra Ops stuff

This file contains auxiliary Ops, used during the compilation phase and Ops building class (FromFunctionOp) and decorator (as_op()) that help make new Ops more rapidly.

class theano.compile.ops.DeepCopyOp

c_code(node, name, inames, onames, sub)

Required: return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters

• node (Apply instance) – The node for which we are compiling the current c_code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as ‘fail’). WRITEME

Raises

MethodNotDefined – The subclass does not override this method.

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an ‘unversioned’ Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superceded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply()

make_node(x)

Create a “apply” nodes for the inputs in that order.

perform(node, args, outs)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.compile.ops.FromFunctionOp(fn, itypes, otypes, infer_shape)

Build a basic Theano Op around a function.

Since the resulting Op is very basic and is missing most of the optional functionalities, some optimizations may not apply. If you want to help, you can supply an infer_shape function that computes the shapes of the output given the shapes of the inputs.

Also the gradient is undefined in the resulting op and Theano will raise an error if you attempt to get the gradient of a graph containing this op.

perform(node, inputs, outputs)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.compile.ops.OutputGuard

This op is used only internally by Theano.

Only the AddDestroyHandler optimizer tries to insert them in the graph.

This Op is declared as destructive while it is not destroying anything. It returns a view. This is used to prevent destruction of the output variables of a Theano function.

There is a mechanism in Theano that should prevent this, but the use of OutputGuard adds a safeguard: it may be possible for some optimization run before the add_destroy_handler phase to bypass this mechanism, by making in-place optimizations.

TODO: find a current full explanation.

class theano.compile.ops.Rebroadcast(*axis)

Change the input’s broadcastable fields in some predetermined way.

See also

unbroadcast, addbroadcast, patternbroadcast

Notes

Works inplace and works for CudaNdarrayType.

Example

Rebroadcast((0, True), (1, False))(x) would make x broadcastable in axis 0 and not broadcastable in axis 1.

R_op(inputs, eval_points)

This method is primarily used by tensor.Rop

Suppose the op outputs

[ f_1(inputs), …, f_n(inputs) ]

Parameters

• inputs (a Variable or list of Variables) –

• eval_points – A Variable or list of Variables with the same length as inputs. Each element of eval_points specifies the value of the corresponding input at the point where the R op is to be evaluated.

Returns

rval[i] should be Rop(f=f_i(inputs),

wrt=inputs, eval_points=eval_points)

Return type

list of n elements

c_code(node, nodename, inp, out, sub)

Required: return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters

• node (Apply instance) – The node for which we are compiling the current c_code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as ‘fail’). WRITEME

Raises

MethodNotDefined – The subclass does not override this method.

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an ‘unversioned’ Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superceded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply()

make_node(x)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out_)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.compile.ops.Shape

L{Op} to return the shape of a matrix.

Notes

Non-differentiable.

R_op(inputs, eval_points)

This method is primarily used by tensor.Rop

Suppose the op outputs

[ f_1(inputs), …, f_n(inputs) ]

Parameters

• inputs (a Variable or list of Variables) –

• eval_points – A Variable or list of Variables with the same length as inputs. Each element of eval_points specifies the value of the corresponding input at the point where the R op is to be evaluated.

Returns

rval[i] should be Rop(f=f_i(inputs),

wrt=inputs, eval_points=eval_points)

Return type

list of n elements

c_code(node, name, inames, onames, sub)

Required: return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters

• node (Apply instance) – The node for which we are compiling the current c_code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as ‘fail’). WRITEME

Raises

MethodNotDefined – The subclass does not override this method.

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an ‘unversioned’ Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superceded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply()

make_node(x)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out_)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.compile.ops.Shape_i(i)

L{Op} to return the shape of a matrix.

Notes

Non-differentiable.

c_code(node, name, inames, onames, sub)

Required: return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters

• node (Apply instance) – The node for which we are compiling the current c_code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as ‘fail’). WRITEME

Raises

MethodNotDefined – The subclass does not override this method.

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an ‘unversioned’ Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superceded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply()

make_node(x)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out_, params)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.compile.ops.SpecifyShape

L{Op} that puts into the graph the user-provided shape.

In the case where this op stays in the final graph, we assert the shape. For this the output of this op must be used in the graph. This is not the case most of the time if we only take the shape of the output. Maybe there are other optimizations that will mess with this.

Notes

Maybe in the future we will never do the assert!

We currently don’t support specifying partial shape information.

TODO : test this op with sparse. Do C code for them too.

R_op(inputs, eval_points)

This method is primarily used by tensor.Rop

Suppose the op outputs

[ f_1(inputs), …, f_n(inputs) ]

Parameters

• inputs (a Variable or list of Variables) –

• eval_points – A Variable or list of Variables with the same length as inputs. Each element of eval_points specifies the value of the corresponding input at the point where the R op is to be evaluated.

Returns

rval[i] should be Rop(f=f_i(inputs),

wrt=inputs, eval_points=eval_points)

Return type

list of n elements

c_code(node, name, inames, onames, sub)

Required: return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters

• node (Apply instance) – The node for which we are compiling the current c_code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as ‘fail’). WRITEME

Raises

MethodNotDefined – The subclass does not override this method.

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an ‘unversioned’ Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superceded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply()

c_support_code_apply(node, name)

Optional: return utility code for use by an Op that will be inserted at global scope, that can be specialized for the support of a particular Apply node.

Parameters

• node (an Apply instance in the graph being compiled) –

• name (str) – A string or number that serves to uniquely identify this node. Symbol names defined by this support code should include the name, so that they can be called from the c_code, and so that they do not cause name collisions.

Notes

This function is called in addition to c_support_code and will supplement whatever is returned from there.

Raises

MethodNotDefined – Subclass does not implement this method.

make_node(x, shape)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out_)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.compile.ops.ViewOp

Returns an inplace view of the input. Used internally by Theano.

c_code(node, nodename, inp, out, sub)

Required: return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters

• node (Apply instance) – The node for which we are compiling the current c_code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding python variable that can be accessed by prepending “py_” to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as ‘fail’). WRITEME

Raises

MethodNotDefined – The subclass does not override this method.

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an ‘unversioned’ Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superceded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply()

make_node(x)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

theano.compile.ops.as_op(itypes, otypes, infer_shape=None)

Decorator that converts a function into a basic Theano op that will call the supplied function as its implementation.

It takes an optional infer_shape parameter that should be a callable with this signature:

def infer_shape(node, input_shapes):

… return output_shapes

Here input_shapes and output_shapes are lists of tuples that represent the shape of the corresponding inputs/outputs.

This should not be used when performance is a concern since the very basic nature of the resulting Op may interfere with certain graph optimizations.

Examples

@as_op(itypes=[theano.tensor.fmatrix, theano.tensor.fmatrix],

otypes=[theano.tensor.fmatrix])

def numpy_dot(a, b):

return numpy.dot(a, b)

theano.compile.ops.register_deep_copy_op_c_code(typ, code, version=())

Tell DeepCopyOp how to generate C code for a Theano Type.

Parameters

• typ (Theano type) – It must be the Theano class itself and not an instance of the class.

• code (C code) – Deep copies the Theano type ‘typ’. Use %(iname)s and %(oname)s for the input and output C variable names respectively.

• version – A number indicating the version of the code, for cache.

theano.compile.ops.register_rebroadcast_c_code(typ, code, version=())

Tell Rebroadcast how to generate C code for a Theano Type.

typTheano type

It must be the Theano class itself and not an instance of the class.

codeC code

That checks if the dimension %(axis)s is of shape 1 for the Theano type ‘typ’. Use %(iname)s and %(oname)s for the input and output C variable names respectively, and %(axis)s for the axis that we need to check. This code is put in a loop for all axes.

version

A number indicating the version of the code, for cache.

theano.compile.ops.register_shape_c_code(type, code, version=())

Tell Shape Op how to generate C code for a Theano Type.

Parameters

• typ (Theano type) – It must be the Theano class itself and not an instance of the class.

• code (C code) – Returns a vector representing the shape for the Theano type ‘typ’. Use %(iname)s and %(oname)s for the input and output C variable names respectively.

• version – A number indicating the version of the code, for cache.

theano.compile.ops.register_shape_i_c_code(typ, code, check_input, version=())

Tell Shape_i how to generate C code for a Theano Type.

Parameters

• typ (Theano type) – It must be the Theano class itself and not an instance of the class.

• code (C code) – Gets the shape of dimensions %(i)s for the Theano type ‘typ’. Use %(iname)s and %(oname)s for the input and output C variable names respectively.

• version – A number indicating the version of the code, for cache.

theano.compile.ops.register_specify_shape_c_code(typ, code, version=(), c_support_code_apply=None)

Tell SpecifyShape how to generate C code for a Theano Type.

Parameters

• typ (Theano type) – It must be the Theano class itself and not an instance of the class.

• code (C code) – Checks the shape and returns a view for the Theano type ‘typ’. Use %(iname)s and %(oname)s for the input and output C variable names respectively. %(shape)s is the vector of shape of %(iname)s. Check that its length is good.

• version – A number indicating the version of the code, for cache.

• c_support_code_apply – Extra code.

theano.compile.ops.register_view_op_c_code(type, code, version=())

Tell ViewOp how to generate C code for a Theano Type.

Parameters

• type (Theano type) – It must be the Theano class itself and not an instance of the class.

• code (C code) – Returns a view for the Theano type ‘type’. Use %(iname)s and %(oname)s for the input and output C variable names respectively.

• version – A number indicating the version of the code, for cache.

theano.compile.ops.shape_i(var, i, fgraph=None)

Equivalent of var.shape[i], but apply if possible the shape feature optimization.

This is useful in optimization that need to get the shape. This remove the need of the following shape_feature optimization that convert it. So this speed up optimization and remove Equilibrium max iteration problems.

Parameters

• var – The variable we want to take the shape of.

• i – The shape dimensions we want

• fgraph (optional) – If var.fgraph do not exist, the fgraph that have the shape_feature to introduce var in to get the optimized shape.