tensor.io – Tensor IO Ops

File operation

• Load from disk with the function load and its associated op LoadFromDisk

MPI operation

• Non-blocking transfer: isend and irecv.

• Blocking transfer: send and recv

Details

class theano.tensor.io.LoadFromDisk(dtype, broadcastable, mmap_mode=None)

An operation to load an array from disk.

See also

load

Notes

Non-differentiable.

make_node(path)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.tensor.io.MPIRecv(source, tag, shape, dtype)

An operation to asynchronously receive an array to a remote host using MPI.

See also

MPIRecv, MPIWait

Notes

Non-differentiable.

do_constant_folding(node)

This allows each op to determine if it wants to be constant folded when all its inputs are constant. This allows it to choose where it puts its memory/speed trade-off. Also, it could make things faster as constants can’t be used for inplace operations (see *IncSubtensor).

make_node()

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.tensor.io.MPIRecvWait(tag)

An operation to wait on a previously received array using MPI.

See also

MPIRecv

Notes

Non-differentiable.

make_node(request, data)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.tensor.io.MPISend(dest, tag)

An operation to asynchronously Send an array to a remote host using MPI.

See also

MPIRecv, MPISendWait

Notes

Non-differentiable.

make_node(data)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

class theano.tensor.io.MPISendWait(tag)

An operation to wait on a previously sent array using MPI.

See also

MPISend

Notes

Non-differentiable.

make_node(request, data)

Create a “apply” nodes for the inputs in that order.

perform(node, inp, out)

Required: Calculate the function on the inputs and put the variables in the output storage. Return None.

Parameters

• node (Apply instance) – Contains the symbolic inputs and outputs.

• inputs (list) – Sequence of inputs (immutable).

• output_storage (list) – List of mutable 1-element lists (do not change the length of these lists)

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a Numpy ndarray, with the right number of dimensions, and the correct dtype. Its shape and stride pattern, can be arbitrary. It not is guaranteed that it was produced by a previous call to impl. It could be allocated by another Op impl is free to reuse it as it sees fit, or to discard it and allocate new memory.

Raises

MethodNotDefined – The subclass does not override this method.

theano.tensor.io.irecv(shape, dtype, source, tag)

Non-blocking receive.

theano.tensor.io.isend(var, dest, tag)

Non blocking send.

theano.tensor.io.load(path, dtype, broadcastable, mmap_mode=None)

Load an array from an .npy file.

Parameters

• path – A Generic symbolic variable, that will contain a string

• dtype (data-type) – The data type of the array to be read.

• broadcastable – The broadcastable pattern of the loaded array, for instance, (False,) for a vector, (False, True) for a column, (False, False) for a matrix.

• mmap_mode – How the file will be loaded. None means that the data will be copied into an array in memory, ‘c’ means that the file will be mapped into virtual memory, so only the parts that are needed will be actually read from disk and put into memory. Other modes supported by numpy.load (‘r’, ‘r+’, ‘w+’) cannot be supported by Theano.

Examples

>>> from theano import *
>>> path = Variable(Generic())
>>> x = tensor.load(path, 'int64', (False,))
>>> y = x*2
>>> fn = function([path], y)
>>> fn("stored-array.npy")  
array([0, 2, 4, 6, 8], dtype=int64)

theano.tensor.io.mpi_send_wait_key(a)

Wait as long as possible on Waits, Start Send/Recvs early.

theano.tensor.io.mpi_tag_key(a)

Break MPI ties by using the variable tag - prefer lower tags first.

theano.tensor.io.recv(shape, dtype, source, tag)

Blocking receive.

theano.tensor.io.send(var, dest, tag)

Blocking send.