# neural_network.builder

Neural network builder class to construct Core ML models.

class coremltools.models.neural_network.builder.NeuralNetworkBuilder(input_features=None, output_features=None, mode=None, spec=None, nn_spec=None, disable_rank5_shape_mapping=False, training_features=None, use_float_arraytype=False)[source]

Neural network builder class to construct Core ML models.

The NeuralNetworkBuilder constructs a Core ML neural network specification layer by layer. The layers should be added in such an order that the inputs to each layer (referred to as blobs of each layer) have been previously defined. The builder can also set preprocessing steps to handle specialized input formats (such as images), and set class labels for neural network classifiers.

Refer to the protobuf messages in the specification (NeuralNetwork.proto) for more details.

Examples

from coremltools.models.neural_network import datatypes, NeuralNetworkBuilder
from coremltools.models.utils import save_spec

# Create a neural network binary classifier that classifies
# 3-dimensional data points
# Specify input and output dimensions
>>> input_dim = (3,)
>>> output_dim = (2,)

# Specify input and output features
>>> input_features = [('data', datatypes.Array(*input_dim))]
>>> output_features = [('probs', datatypes.Array(*output_dim))]

# Build a simple neural network with 1 inner product layer
>>> builder = NeuralNetworkBuilder(input_features, output_features)
>>> builder.add_inner_product(name='ip_layer', W=weights, b=bias, input_channels=3, output_channels=2,
... has_bias=True, input_name='data', output_name='probs')

# save the spec by the builder
>>> save_spec(builder.spec, 'network.mlmodel')

__init__(input_features=None, output_features=None, mode=None, spec=None, nn_spec=None, disable_rank5_shape_mapping=False, training_features=None, use_float_arraytype=False)[source]

Construct a NeuralNetworkBuilder object to build an MLModel specification with a model interface, or a NeuralNetwork protobuf message, either from scratch or using an existing specification.

Parameters
input_features: [(str, datatypes.Array)] or None

List of input feature of the network. Each feature is a (name, array) tuple, where name is the name of the feature, and array is a datatype.Array object describing the feature type.

When spec is None (building from scratch), input_features must not be None.

When spec is not None, input_features will be ignored; the input feature of the existing spec will be used instead.

output_features: [(str, datatypes.Array or None)] or None

List of output feature of the network. Each feature is a (name, array) tuple, where name is the name of the feature, and array is a datatypes.Array object describing the feature type.

The array can be None if not known.

When spec is None (building from scratch), output_features must not be None.

When spec is not None, output_features will be ignored; the output feature of the existing spec will be used instead.

mode: str (‘classifier’, ‘regressor’ or None)

Mode (one of 'classifier', 'regressor', or None).

When mode = 'classifier', a NeuralNetworkClassifier spec will be constructed. When mode = 'regressor', a NeuralNetworkRegressor spec will be constructed.

disable_rank5_shape_mapping: bool

Only applicable for neural networks.

If True, inputs are no longer forced to map to rank 5 tensors (rank is equal to the length of the shape of the tensor). Instead, for multi-array inputs "EXACT_ARRAY_MAPPING" mapping is used, whereas for image inputs "RANK4_IMAGE_MAPPING" is used. For details, see description of enums NeuralNetworkMultiArrayShapeMapping and NeuralNetworkImageShapeMapping in NeuralNetwork.proto.

When spec is not None, this argument will be ignored.

spec: None or coremltools.proto.Model_pb2

If None, a new MLModel spec will be created by the builder with input and output features.

Otherwise, the builder will continue to build on spec. This is useful when the MLModel is built incrementally.

nn_spec: None or coremltools.proto.NeuralNetwork_pb2

If None, a new, empty NeuralNetwork proto will be created for spec.

If nn_spec is not None and spec is None, the builder will build a NeuralNetwork spec without wrapping it within an MLModel. This is useful to create nested NeuralNetworks for models with control flow operations.

use_float_arraytype: bool

If true, the datatype of input/output multiarrays is set to Float32 instead of double.

Examples

# Construct a builder that builds a neural network classifier with a 299 x 299 x 3
# dimensional input and 1000 dimensional output
>>> input_features = [('data', datatypes.Array((299, 299, 3)))]
>>> output_features = [('probs', datatypes.Array((1000,)))]
>>> builder = NeuralNetworkBuilder(input_features, output_features, mode='classifier')


Add an acos layer to the model that computes element-wise arc-cosine for the input tensor. Refer to the AcosLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add an acosh layer to the model that computes element-wise inverse hyperbolic cosine for the input tensor. Refer to the AcoshLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_activation(name, non_linearity, input_name, output_name, params=None, input_rank=None, input_shape=None, output_rank=None, output_shape=None)[source]

Add an activation layer to the model. Refer to the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

non_linearity: str

The non_linearity (activation) function of this layer. It can be one of the following:

• 'RELU': Rectified Linear Unit (ReLU) function.

• 'SIGMOID': sigmoid function.

• 'TANH': tanh function.

• 'SCALED_TANH': scaled tanh function, defined as:

f(x) = alpha * tanh(beta * x)

where alpha and beta are constant scalars.

• 'SOFTPLUS': softplus function.

• 'SOFTSIGN': softsign function.

• 'SIGMOID_HARD': hard sigmoid function, defined as:

f(x) = min(max(alpha * x + beta, -1), 1)

where alpha and beta are constant scalars.

• 'LEAKYRELU': leaky relu function, defined as:

f(x) = (x >= 0) * x + (x < 0) * alpha * x

where alpha is a constant scalar.

• 'PRELU': Parametric ReLU function, defined as:

f(x) = (x >= 0) * x + (x < 0) * alpha * x

where alpha is a multi-dimensional array of same size as x.

• 'ELU': Exponential linear unit function, defined as:

f(x) = (x >= 0) * x + (x < 0) * (alpha * exp(x) - 1)

where alpha is a constant scalar.

• 'PARAMETRICSOFTPLUS': Parametric softplus function, defined as:

f(x) = alpha * log(1 + exp(beta * x))

where alpha and beta are two multi-dimensional arrays of same size as x.

• 'THRESHOLDEDRELU': Thresholded ReLU function, defined as:

f(x) = (x >= alpha) * x

where alpha is a constant scalar.

• 'LINEAR': linear function.

f(x) = alpha * x + beta

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

params: list of float or numpy.array

Parameters for the activation, depending on non_linearity.

• When non_linearity is one of ['RELU', 'SIGMOID', 'TANH', 'SCALED_TANH', 'SOFTPLUS', 'SOFTSIGN'], params is ignored.

• When non_linearity is one of ['SCALED_TANH', 'SIGMOID_HARD', 'LINEAR'], param is a list of 2 floats [alpha, beta].

• When non_linearity is one of ['LEAKYRELU', 'ELU', 'THRESHOLDEDRELU'], param is a list of 1 float [alpha].

• When non_linearity is 'PRELU', param is a list of 1 numpy array [alpha]. The shape of alpha is (C,), where C is either the number of input channels or 1. When C = 1, same alpha is applied to all channels.

• When non_linearity is 'PARAMETRICSOFTPLUS', param is a list of 2 numpy arrays [alpha, beta]. The shape of alpha and beta is (C, ), where C is either the number of input channels or 1. When C = 1, same alpha and beta are applied to all channels.

Add an add_broadcastable layer to the model that performs element-wise addition operation with broadcast support. Refer to the AddBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add an argmax layer to the model that returns the indices of the maximum value along a specified axis in the input tensor. Refer to the ArgMaxLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: int

axis along which the argmax is computed. Negative indexing is supported.

keepdims: bool, optional

if true, output rank is same as input rank, default: true.

Add an argmin layer to the model that returns the indices of the minimum value along a specified axis in the input tensor. Refer to the ArgMinLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: int

axis along which the argmin is computed. Negative indexing is supported.

keepdims: bool, optional

if true, output rank is same as input rank, default: true.

Add an argsort layer to the model. Refer to the ArgsortLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: int, optional

axis along which to compute the sorting indices

descending: bool, optional

order of sorting

Add an asin layer to the model that computes element-wise arc-sine for the input tensor. Refer to the AsinLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add an asinh layer to the model that computes element-wise inverse hyperbolic sine for the input tensor. Refer to the AsinhLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add an atan layer to the model that computes element-wise arc-tangent for the input tensor. Refer to the AtanLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add an atanh layer to the model that computes element-wise inverse hyperbolic tangent for the input tensor. Refer to the AtanhLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_batched_mat_mul(name, input_names, output_name, transpose_a=False, transpose_b=False, weight_matrix_rows=0, weight_matrix_columns=0, W=None, bias=None, int_8_dynamic_quantize=False, is_quantized_weight=False, quantization_type='linear', nbits=8, quant_scale=None, quant_bias=None, quant_lut=None)[source]

Add a N-D Batched Matrix Multiplication layer with NumPy-like broadcasting. Refer to the BatchedMatMulLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

transpose_a: bool, optional

Whether or not to transpose A, default: false.

transpose_b: bool, optional

Whether or not to transpose B, default: false.

weight_matrix_rows: int, optional

Must be equal to the last dimension of the input, default: 0.

weight_matrix_columns: int, optional

Must be equal to the last dimension of the output, default: 0.

W: float32 numpy.array or bytes(), optional

Weight matrix of shape (weight_matrix_rows, weight_matrix_columns). If W is of type bytes() (quantized to 1-8 bits), other quantization-related arguments must be provided as well (see below).

bias: float32 numpy.array, optional

Bias vector of shape (weight_matrix_columns,).

Quantization

Quantization arguments, used when W is of type bytes():

is_quantized_weight: bool, optional

Set it to true when W is of type bytes(), representing quantized weights, default: false.

quantization_type: str, optional

When weights are quantized (that is, W is of type bytes()), this should be either "linear" or "lut", default: "linear".

nbits: int, optional

Should be between 1 and 8 (inclusive). Number of bits per weight value, default: 8.

quant_scale: numpy.array(dtype=numpy.float32), optional

Scale vector to be used with linear quantization. Must be of length either 1 or weight_matrix_columns, default: None.

quant_bias: numpy.array(dtype=numpy.float32), optional

Bias vector to be used with linear quantization. Must be of length either 1 or weight_matrix_columns, default: None.

quant_lut: numpy.array(dtype=numpy.float32), optional

The LUT (look up table) to be used with LUT quantization. Must be of length 2^n bits, default: None.

int_8_dynamic_quantize: bool

Whether to quantize and dequantize before and after batched matmul, respectively. Expects byte weights, representing int8 values, if True. See NeuralNetwork.proto for other validation conditions.

add_batchnorm(name, channels, gamma, beta, mean=None, variance=None, input_name='data', output_name='out', compute_mean_var=False, instance_normalization=False, epsilon=1e-05)[source]

Add a batch normalization layer. Batch normalization operation is defined as:

y = gamma * (x - mean) / sqrt(variance + epsilon) + beta

Refer to the BatchnormLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

channels: int

Number of channels of the input blob.

gamma: numpy.array

Values of gamma. Must be numpy array of shape (channels, ).

beta: numpy.array

Values of beta. Must be numpy array of shape (channels, ).

mean: numpy.array

Means of the input blob on each channel. Must be numpy array of shape (channels, ).

variance:

Variances of the input blob on each channel. Must be numpy array of shape (channels, ).

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

compute_mean_var: bool

Set to True if mean and variance is to be computed from the input data.

instance_normalization: bool

Set compute_mean_var and this to True to perform instance normalization. That is, mean and variance are computed from the single input instance.

epsilon: float

Value of epsilon. Defaults to 1e-5 if not specified.

Add a bias layer to the model. Refer to the BiasLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

b: int or numpy.array

Bias to add to the input.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

shape_bias: list of int

List of ints that specifies the shape of the bias parameter (if present). Can be [1], [C], [1,H,W], or [C,H,W].

add_bidirlstm(name, W_h, W_x, b, W_h_back, W_x_back, b_back, hidden_size, input_size, input_names, output_names, inner_activation='SIGMOID', cell_state_update_activation='TANH', output_activation='TANH', peep=None, peep_back=None, output_all=False, forget_bias=False, coupled_input_forget_gate=False, cell_clip_threshold=50000.0)[source]

Add a Bi-directional LSTM layer to the model. Refer to the BiDirectionalLSTMLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W_h: [numpy.array]

List of recursion weight matrices for the forward layer. The ordering is [R_i, R_f, R_o, R_z], where R_i, R_f, R_o, and R_z are weight matrices at input gate, forget gate, output gate and cell gate. The shapes of these matrices are (hidden_size, hidden_size).

W_x: [numpy.array]

List of input weight matrices for the forward layer. The ordering is [W_i, W_f, W_o, W_z], where W_i, W_f, W_o, and W_z are weight matrices at input gate, forget gate, output gate and cell gate. The shapes of these matrices are (hidden_size, input_size).

b: [numpy.array]

List of biases for the forward layer. The ordering is [b_i, b_f, b_o, b_z], where b_i, b_f, b_o, and b_z are biases at input gate, forget gate, output gate and cell gate. If None, biases are ignored. Otherwise the shapes of the biases are (hidden_size, ).

W_h_back: [numpy.array]

List of recursion weight matrices for the backward layer. The ordering is [R_i, R_f, R_o, R_z], where R_i, R_f, R_o, and R_z are weight matrices at input gate, forget gate, output gate and cell gate. The shapes of these matrices are (hidden_size, hidden_size).

W_x_back: [numpy.array]

List of input weight matrices for the backward layer. The ordering is [W_i, W_f, W_o, W_z], where W_i, W_f, W_o, and W_z are weight matrices at input gate, forget gate, output gate and cell gate. The shapes of these matrices are (hidden_size, input_size).

b_back: [numpy.array]

List of biases for the backward layer. The ordering is [b_i, b_f, b_o, b_z], where b_i, b_f, b_o, and b_z are biases at input gate, forget gate, output gate and cell gate. The shapes of the biases (hidden_size).

hidden_size: int

Number of hidden units. This is equal to the number of channels of output shape.

input_size: int

Number of the number of channels of input shape.

input_names: list of str

The input blob names of this layer, in the order of [x, h_input, c_input, h_reverse_input, c_reverse_input].

output_names: list of str

The output blob names of this layer, in the order of [y, h_output, c_output, h_reverse_output, c_reverse_output].

inner_activation: str

Inner activation function used at input and forget gate. Can be one of the following options: ['RELU', 'TANH', 'SIGMOID', 'SCALED_TANH', 'SIGMOID_HARD', 'LINEAR']. Defaults to 'SIGMOID'.

cell_state_update_activation: str

Cell state update activation function used at the cell state update gate. Can be one of the following options: ['RELU', 'TANH', 'SIGMOID', 'SCALED_TANH', 'SIGMOID_HARD', 'LINEAR']. Defaults to 'TANH'.

output_activation: str

Activation function used at the output gate. Can be one of the following options: ['RELU', 'TANH', 'SIGMOID', 'SCALED_TANH', 'SIGMOID_HARD', 'LINEAR']. Defaults to 'TANH'.

peep: [numpy.array] or None

List of peephole vectors for the forward layer. The ordering is [p_i, p_f, p_o], where p_i, p_f, and p_o are peephole vectors at input gate, forget gate, and output gate. The shapes of the peephole vectors are (hidden_size,). Defaults to None.

peep_back: [numpy.array] or None

List of peephole vectors for the backward layer. The ordering is [p_i, p_f, p_o], where p_i, p_f, and p_o are peephole vectors at input gate, forget gate, and output gate. The shapes of the peephole vectors are (hidden_size,). Defaults to None.

output_all: boolean

Whether the LSTM layer should output at every time step. Defaults to False.

• If False, the output is the result after the final state update.

• If True, the output is a sequence, containing outputs at all time steps.

forget_bias: boolean

If True, a vector of 1s is added to forget gate bias. Defaults to False.

coupled_input_forget_gate: boolean
If True, the input gate and forget gate is coupled. That is, the

forget gate is not used.

Defaults to False.

cell_clip_threshold: float

The limit on the maximum and minimum values on the cell state. Defaults to 50.0.

Add a branch layer to the model that provides the functionality of branching or an if-else block. Refer to the BranchLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

if_branch: NeuralNetwork

Neural network to execute if the absolute value of the input tensor is greater than 1e-6.

else_branch: NeuralNetwork, optional

Neural network to execute if the absolute value of the input tensor is less than 1e-6.

Add a broadcast_to_dynamic layer to the model that broadcasts a tensor to a compatible shape. Refer to the BroadcastToDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a broadcast_to_like layer to the model that broadcasts a tensor to a compatible shape. Refer to the BroadcastToLikeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a broadcast_to_static layer to the model that broadcasts a tensor to a compatible shape. Refer to the BroadcastToStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

The target shape of the output tensor.

add_categorical_distribution(name, input_name, output_name, num_samples, is_logits=True, eps=1e-10, temperature=1.0, seed=- 1)[source]

Add a categorical_distribution layer to the model that fills the output tensor with random values from categorical distribution. Refer to the CategoricalDistributionLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

num_samples: int

List of dimensions for the reduce operations.

is_logits: bool, optional

If true, the input is log probabilities. If false, the input is probabilities, default: True

eps: float, optional

Epsilon parameter for categorical distribution, default 1e-10.

temperature: float, optional

Temperature parameter for categorical distribution, default 1.0.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

Add a ceil layer to the model that performs element-wise ceil operation on the input tensor that rounds the value to the smallest integer not less than x. Refer to the CeilLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a clamped relu layer to the model. Clamped relu formula is f(x) = min((x >= 0 ? x : alpha * x), beta) Refer to the ClampedReluLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

alpha: float, optional

slope of the output when input is negative, default: 0.0.

beta: float, optional

Upper bound on the output value, default: 6.0.

Add a clip layer to the model that performs element-wise clip operation. Clip the values in the input tensor to the range [min_value, max_value]. Refer to the ClipLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

min_value: float, optional

Lower bound / minimum value for clip, default: 0.0.

max_value: float, optional

Upper bound / maximum value for clip, default: 1.0.

Add a concat_nd layer to the model that performs concatenation along the given axis. Refer to the ConcatNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int

Axis to perform the concat operation on.

interleavebool

(Only available in Core ML Specification >= 5 (iOS >= 14, macOS >= 11.0) If true, concatenate by interleaving the inputs

Add a constant pad layer. Refer to the ConstantPaddingLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob name(s) of this layer.

output_name: str

The output blob name of this layer.

value: float

value to be used for padding.

if true, pad_amounts are interpreted as output shapes (see example in NeuralNetwork.proto)

must be non negative. Amount to pad in each dimension. Length of the list must be twice the input/output rank. Not required if second input is present.

Add a convolution layer to the network. Refer to the ConvolutionLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

kernel_channels: int

Number of channels for the convolution kernels.

output_channels: int

Number of filter kernels. This is equal to the number of channels in the output blob.

height: int

Height of each kernel.

width: int

Width of each kernel.

stride_height: int

Stride along the height direction.

stride_width: int

Stride along the height direction.

border_mode: str

Option for the padding type and output blob shape. Can be either ‘valid’ or ‘same’.

groups: int

Number of kernel groups. Input is divided into groups along the channel axis. Each kernel group share the same weights.

W: numpy.array or bytes() or None

Weight of the convolution kernels.

• If is_deconv is False, W should have shape (height, width, kernel_channels, output_channels), where:

kernel_channel = input_channels / groups.

• If is_deconv is True, W should have shape (height, width, kernel_channels, output_channels / groups), where:

kernel_channel = input_channels.

If W is of type bytes() (quantized), other quantization related arguments must be provided as well (see below).

For Core ML specification version >=4, W can be None. In this case, the convolution layer takes 2 inputs, where the 1st input represents the input feature map, and the 2nd input represents the weight blob.

b: numpy.array

Biases of the convolution kernels. b should have shape (outputChannels, ).

has_bias: boolean

Whether bias is ignored.

• If True, bias is not ignored.

• If False, bias is ignored.

is_deconv: boolean

Whether the convolution layer is performing a convolution or a transposed convolution (deconvolution).

• If True, the convolution layer is performing transposed convolution.

• If False, the convolution layer is performing regular convolution.

output_shape: tuple or None

Either None or a 2-tuple, specifying the output shape (output_height, output_width). Used only when is_deconv == True. When is_deconv == False, this parameter is ignored. If it is None, the output shape is calculated automatically using the border_mode.

input_name: str or list of str

The input blob name(s) of this layer.

output_name: str

The output blob name of this layer.

dilation_factors: list of int

Dilation factors across height and width directions. Must be a list of two positive integers. Defaults to [1, 1].

values of height (top, bottom) and width (left, right) padding to be used if border_more is "valid".

Type of asymmetric padding to be used when border_mode is 'same'. Can be either 'BOTTOM_RIGHT_HEAVY' or 'TOP_LEFT_HEAVY'.

Depthwise convolution

Depthwise convolution is a special case of convolution, in which:

• kernel_channels = 1 (== input_channels / groups)

• output_channels = channel_multiplier * input_channels

• groups = input_channels

• W: [Kernel_height, Kernel_width, 1, channel_multiplier * input_channels]

Quantization

Quantization arguments expected in kwargs, when W is of type bytes().

quantization_type: str

When weights are quantized (that is, W is of type bytes()), this should be either "linear" or "lut".

nbits: int

Should be between 1 and 8 (inclusive). Number of bits per weight value. Only applicable when weights are quantized.

quant_scale: numpy.array(dtype=numpy.float32)

scale vector to be used with linear quantization. Must be of length either 1 or output_channels.

quant_bias: numpy.array(dtype=numpy.float32)

bias vector to be used with linear quantization. Must be of length either 1 or output_channels.

quant_lut: numpy.array(dtype=numpy.float32)

the LUT (look up table) to be used with LUT quantization. Must be of length 2^n bits.

Add a 3 dimensional convolution layer to the network. Refer to the Convolution3DLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_channels: int

Number of input channels.

output_channels: int

Number of filter kernels. This is equal to the number of channels in the output blob.

depth: int

Depth of each kernel.

height: int

Height of each kernel.

width: int

Width of each kernel.

W: numpy.array or bytes()

Weight of the convolution kernels. - W should have shape: - If deconv is False:

(output_channels, kernel_channels, depth, height, width), where:

kernel_channels = input_channels / groups

• If deconv is True:

(output_channels / groups, kernel_channels, depth, height, width), where:

kernel_channels = input_channels

b: numpy.array

Biases of the convolution kernels. b should have shape (outputChannels, ).

has_bias: boolean

Whether bias is ignored. - If True, bias is not ignored. - If False, bias is ignored.

groups: int

Number of kernel groups. Input is divided into groups along the channel axis. Each kernel group share the same weights. Defaults to 1.

stride_depth, stride_height, stride_width: int

Stride along the depth, height, and width directions, respectively. Must all be positive integers. Defaults to 1.

dilation_depth, dilation_width, dilation_height: int

Dilation factors across depth, height, and width directions. Must all be positive integers. Defaults to 1 in each dimension.

is_deconv: bool

True if this is Convolution Transpose, otherwise False.

output_shape: None or Tuple of int

Applicable only for Deconvolution layer. None if Convolution. Tuple of length 3 if Convolution Transpose.

Option for the padding type and output blob shape. Can be 'custom', 'valid', or 'same'. Defaults to 'valid'. Case-insensitive.

Values of depth (front, back), height (top, bottom), and width (left, right) padding to be used. Must all be positive integers. All default to 0.

input_name: str or list of str

The input blob name(s) of this layer.

output_name: str

The output blob name of this layer.

Depthwise convolution

Depthwise convolution is a special case of convolution, in which:

• kernel_channels = 1 (== input_channels / groups)

• output_channels = channel_multiplier * input_channels

• groups = input_channels

• W: [Kernel_depth, Kernel_height, Kernel_width, 1, channel_multiplier * input_channels]

Add a copy layer to the model that copies its input tensor to the output tensor. Input tensor and output tensor must have distinct names. Refer to the CopyLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a cos layer to the model that computes element-wise cosine for the input tensor. Refer to the CosLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a osh layer to the model that computes element-wise hyperbolic cosine for the input tensor. Refer to the CoshLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_crop(name, left, right, top, bottom, offset, input_names, output_name)[source]

Add a cropping layer to the model. The cropping layer have two functional modes:

• When it has 1 input blob, it crops the input blob based on the 4 parameters [left, right, top, bottom].

• When it has 2 input blobs, it crops the first input blob based on the dimension of the second blob with an offset.

Refer to the CropLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

left: int

Number of elements to be cropped on the left side of the input blob. When the crop layer takes 2 inputs, this parameter is ignored.

right: int

Number of elements to be cropped on the right side of the input blob. When the crop layer takes 2 inputs, this parameter is ignored.

top: int

Number of elements to be cropped on the top of the input blob. When the crop layer takes 2 inputs, this parameter is ignored.

bottom: int

Number of elements to be cropped on the bottom of the input blob. When the crop layer takes 2 inputs, this parameter is ignored.

offset: list of int

Offset along the height and width directions when the crop layer takes 2 inputs. Must be a list of length 2. When the crop layer takes 1 input, this parameter is ignored.

input_names: list of str

The input blob names of this layer. Must be either a list of 1 string (1 input crop layer), or a list of 2 strings (2-input crop layer).

output_name: str

The output blob name of this layer.

add_crop_resize(name, input_names, output_name, target_height=1, target_width=1, mode='STRICT_ALIGN_ENDPOINTS_MODE', normalized_roi=False, box_indices_mode='CORNERS_HEIGHT_FIRST', spatial_scale=1.0)[source]

Add a crop resize layer to the model. A layer that extracts cropped spatial patches or RoIs (regions of interest) from the input and resizes them to a pre-specified size using bilinear interpolation. Note that RoI Align layer can be implemented with this layer followed by a pooling layer. Refer to the CropResizeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str
Must be a list of two names: image feature map and crop indices/RoI input.
• First input corresponds to a blob with shape [1, Batch, C, H_in, W_in]. This represents a batch of input image feature data with C channels.

• The second input shape must be [N, 1, 4, 1, 1] or [N, 1, 5, 1, 1]. This represents the bounding box coordinates for N patches/RoIs.

• N: number of patches/RoIs to be extracted

• If RoI shape = [N, 1, 4, 1, 1], the channel axis corresponds to the four coordinates specifying the bounding box. All the N~ RoIs are extracted from all the batches of the input.

• If RoI shape = [N, 1, 5, 1, 1], the first element of the channel axis specifies the input batch id from which to extract the RoI and must be in the interval [0, Batch - 1]. That is, n -th RoI is extracted from the RoI[n,0,0,0] -th input batch id. The last four elements of the channel axis specify the bounding box coordinates.

output_name: str

The output blob name of this layer.

target_height: int

Output height dimension.

target_width: int

Output width dimension.

mode: str
• The following values are supported: 'STRICT_ALIGN_ENDPOINTS_MODE', 'ALIGN_ENDPOINTS_MODE', 'UPSAMPLE_MODE', 'ROI_ALIGN_MODE'.

• This parameter determines the sampling grid used for bilinear interpolation.

normalized_roi: bool
• If true the bounding box coordinates must be in the interval [0, 1]. They are scaled by (input_height - 1), (input_width - 1); that is, based on the input spatial dimensions.

• If false the bounding box coordinates must be in the interval [0, input_height - 1] and [0, input_width - 1], respectively for height and width dimensions.

box_indices_mode: str
• The following values are supported: 'CORNERS_HEIGHT_FIRST', 'CORNERS_WIDTH_FIRST', 'CENTER_SIZE_HEIGHT_FIRST', 'CENTER_SIZE_WIDTH_FIRST'.

• Representation used to interpret the bounding box coordinates (RoI) input.
• 'CORNERS_HEIGHT_FIRST': [h_start, w_start, h_end, w_end]

• 'CORNERS_WIDTH_FIRST': [w_start, h_start, w_end, h_end]

• 'CENTER_SIZE_HEIGHT_FIRST': [h_center, w_center, box_height, box_width]

• 'CENTER_SIZE_WIDTH_FIRST': [w_center, h_center, box_width, box_height]

spatial_scale: float

Additional spatial scale that multiplies the bounding box coordinates. Generally used while implementing the RoI Align layer, which uses unnormalized RoI coordinates along with a spatial scale less than or equal to 1.

add_cumsum(name, input_names, output_name, axis=- 1, reverse=False, exclusive=False)[source]

Add a cum sum layer to the model computes the cumulative sum values of the input along a given axis. Refer to the CumSumLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int, optional

Axis to perform the operation, default: -1.

reverse: bool, optional

if true, cumsum is performed in the opposite direction, default: False.

exclusive: bool, optional

whether to perform exclusive or inclusive cumulative summation, default: False.

Add a custom layer. Refer to the CustomLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names to this layer.

output_names: list of str

The output blob names from this layer.

custom_proto_spec: CustomLayerParams

A protobuf CustomLayerParams message. This can also be left blank and filled in later.

Add a divide_broadcastable layer to the model that performs element-wise division operation with broadcast support. Refer to the DivideBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add an element-wise operation layer to the model. Refer to the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

A list of input blob names of this layer. The input blobs should have the same shape.

output_name: str

The output blob name of this layer.

mode: str

A string specifying the mode of the elementwise layer. It can be one of the following:

• 'CONCAT': Concatenate input blobs along the channel axis.

• 'SEQUENCE_CONCAT': Concatenate input blobs along the sequence axis.

• 'ADD': Perform an element-wise summation over the input blobs.

• 'MULTIPLY': Perform an element-wise multiplication over the input blobs.

• 'DOT': Compute the dot product of the two input blobs. In this mode, the length of input_names should be 2.

• 'COS': Compute the cosine similarity of the two input blobs. In this mode, the length of input_names should be 2.

• 'MAX': Compute the element-wise maximum over the input blobs.

• 'MIN': Compute the element-wise minimum over the input blobs.

• 'AVE': Compute the element-wise average over the input blobs.

alpha: float
• if mode == 'ADD' and there is only one input_name, alpha is added to the input.

• if mode == 'MULTIPLY' and there is only one input_name, alpha is multiplied to the input.

add_embedding(name, W, b, input_dim, output_channels, has_bias, input_name, output_name, is_quantized_weight=False, quantization_type='linear', nbits=8, quant_scale=None, quant_bias=None, quant_lut=None)[source]

Add an embedding layer to the model. Refer to the EmbeddingLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W: float32 numpy.array or bytes()

Weight matrix of shape (output_channels, input_dim). If W is of type bytes() (quantized to 1-8 bits), other quantization related arguments must be provided as well (see below).

b: numpy.array

Bias vector of shape (output_channels, ).

input_dim: int

Size of the vocabulary (1 + maximum integer index of the words).

output_channels: int

Number of output channels.

has_bias: boolean

Whether the bias vector of this layer is ignored in the spec.

• If True, the bias vector of this layer is not ignored.

• If False, the bias vector is ignored.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Quantization arguments expected, when W is of type bytes():
is_quantized_weight: bool

Set it to true when W is of type bytes(), representing quantized weights.

quantization_type: str

When weights are quantized (that is, W is of type bytes()), this should be either "linear" or "lut".

nbits: int

Should be between 1 and 8 (inclusive). Number of bits per weight value.

quant_scale: numpy.array(dtype=numpy.float32)

Scale vector to be used with linear quantization. Must be of length either 1 or output_channels.

quant_bias: numpy.array(dtype=numpy.float32)

Bias vector to be used with linear quantization. Must be of length either 1 or output_channels.

quant_lut: numpy.array(dtype=numpy.float32)

The LUT (look up table) to be used with LUT quantization. Must be of length 2^n bits.

add_embedding_nd(name, input_name, output_name, vocab_size, embedding_size, W, b=None, is_quantized_weight=False, quantization_type='linear', nbits=8, quant_scale=None, quant_bias=None, quant_lut=None)[source]

Add an embedding layer to the model that performs a matrix lookup and optionally adds a bias. Refer to the EmbeddingNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

vocab_size: int

Size of the vocabulary (1 + maximum integer index of the words).

embedding_size: int

Size of the embedded vector.

W: float32 numpy.array or bytes()

Weight matrix of shape (embedding_size, vocab_size). If W is of type bytes(), i.e. quantized to 1-8 bits, other quantization related arguments must be provided as well (see below).

b: numpy.array , optional

Bias vector of shape (embedding_size, ).

Quantization arguments expected, when W is of type bytes():
is_quantized_weight: bool

Set it to true when W is of type bytes(), representing quantized weights

quantization_type: str

When weights are quantized (i.e. W is of type bytes()), this should be either “linear” or “lut”.

nbits: int

Should be between 1 and 8 (inclusive). Number of bits per weight value.

quant_scale: numpy.array(dtype=numpy.float32)

scale vector to be used with linear quantization. Must be of length either 1 or embedding_size.

quant_bias: numpy.array(dtype=numpy.float32)

bias vector to be used with linear quantization. Must be of length either 1 or embedding_size.

quant_lut: numpy.array(dtype=numpy.float32)

the LUT (look up table) to be used with LUT quantization. Must be of length 2^nbits.

Add an equal layer to the model that performs the element-wise equal (=) operation. Broadcasting is supported. Refer to the EqualLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

alpha: float, optional

y = x1 != alpha, if only one input is provided, default: 0.

Add an erf function (gaussian error function) layer to the model. Refer to the ErfLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add an exp2 layer to the model that performs element-wise experiential operation. Refer to the Exp2LayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add an expand dims layer to the model that increases the rank of the input tensor by adding unit dimensions. Refer to the ExpandDimsLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int

Dimensions the operation perform on.

Add a fill_dynamic layer to the model that outputs a tensor filled with a scalar value. Refer to the FillDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

value: float, optional

A scalar value for the fill operation, default: 0.

Add a fill_like layer to the model outputs a tensor filled with a scalar value. Refer to the FillLikeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

value: float, optional

A scalar value for the fill operation, default 0.

Add a fill_static layer to the model that outputs a tensor filled with a scalar value given shape as parameter. Refer to the FillStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

The target shape of the output tensor.

value: float, optional

A scalar value for the fill operation, default 0.

Add a flatten layer. Only flattens the channel, height and width axis. Leaves the sequence axis as is. Refer to the FlattenLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

mode: int
• If mode == 0, the flatten layer is in CHANNEL_FIRST mode.

• If mode == 1, the flatten layer is in CHANNEL_LAST mode.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a flatten_to_2d layer to the model that flattens the input tensor into a 2-dimensional matrix. Refer to the FlattenTo2DLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The of input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: int, optional

Axis to perform the operation, default: 1.

Add a floor layer to the model that performs element-wise floor operation on the input tensor that rounds the value to the largest integer not greater than x. Refer to the FloorLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a floor_div_broadcastable layer to the model that performs floor division operation with broadcast support. Refer to the FloorDivBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a gather layer to the model that gathers elements or slices from data and store to a tensor whose shape is defined by indices from the input. Refer to the GatherLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int, optional

The axis the operation perform on, default: 0.

Add a gather_along_axis layer to the model that gathers elements or slices from data and store to a tensor whose shape is defined by indices from the input along the given axis into the output tensor. Refer to the GatherAlongAxisLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int, optional

The axis the operation perform on, default: 0.

Add a gather layer to the model that gathers elements or slices from data and store to a tensor whose shape is defined by indices from the input. This is the reverse operation of the scatter operation. Refer to the GatherNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a GELU (gaussian error linear unit) activation layer, which is: 0.5 * x * (1 + erf(x / sqrt(2))). Refer to the GeluLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

mode: str, optional

Gelu mode in [EXACT | TANH_APPROXIMATION | SIGMOID_APPROXIMATION], default EXACT.

Add a get_shape layer to the model. Refer to the GetShapeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a layer to pool three spatial dimensions down to one value. This behaves like a special case of Pooling3DLayerParams in which the Kernel is the size of the input and there is no padding.

Refer to the GlobalPooling3DLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

pooling_type: str

Type of pooling performed. Can either be 'MAX' OR 'AVERAGE'.

Add a greater_than layer to the model that performs the element-wise greater-than (>) operation or greater-than-or-equal-to (>=) operation. Broadcasting is supported. Refer to the GreaterThanLayerParams, GreaterEqualLayerParams messages in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

use_greater_than_equal: bool, optional

Whether or not to allow greater than or equal to, default: false.

alpha: float, optional

y = x1 != alpha, if only one input is provided, default: 0.

add_gru(name, W_h, W_x, b, hidden_size, input_size, input_names, output_names, activation='TANH', inner_activation='SIGMOID_HARD', output_all=False, reverse_input=False)[source]

Add a Gated-Recurrent Unit (GRU) layer to the model. Refer to the GRULayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W_h: [numpy.array]

List of recursion weight matrices. The ordering is [R_z, R_r, R_o], where R_z, R_r and R_o are weight matrices at update gate, reset gate and output gate. The shapes of these matrices are (hidden_size, hidden_size).

W_x: [numpy.array]

List of input weight matrices. The ordering is [W_z, W_r, W_o], where W_z, W_r, and W_o are weight matrices at update gate, reset gate and output gate. The shapes of these matrices are (hidden_size, input_size).

b: [numpy.array] or None

List of biases of the GRU layer. The ordering is [b_z, b_r, b_o], where b_z, b_r, and b_o are biases at update gate, reset gate and output gate. If None, biases are ignored. Otherwise the shapes of the biases are (hidden_size, ).

hidden_size: int

Number of hidden units. This is equal to the number of channels of output shape.

input_size: int

Number of the number of channels of input shape.

activation: str

Activation function used at the output gate. Can be one of the following options: ['RELU', 'TANH', 'SIGMOID', 'SCALED_TANH', 'SIGMOID_HARD', 'LINEAR']. Defaults to 'TANH'. See add_activation for more detailed description.

inner_activation: str

Inner activation function used at update and reset gates. Can be one of the following options: ['RELU', 'TANH', 'SIGMOID', 'SCALED_TANH', 'SIGMOID_HARD', 'LINEAR']. Defaults to 'SIGMOID_HARD'. See add_activation for more detailed description.

input_names: list of str

The input blob names list of this layer, in the order of [x, h_input].

output_names: list of str

The output blob names list of this layer, in the order of [y, h_output].

output_all: boolean

Whether the recurrent layer should output at every time step.

• If False, the output is the result after the final state update.

• If True, the output is a sequence, containing outputs at all time steps.

reverse_input: boolean

Whether the recurrent layer should process the input sequence in the reverse order.

• If False, the input sequence order is not reversed.

• If True, the input sequence order is reversed.

add_inner_product(name, W, b, input_channels, output_channels, has_bias, input_name, output_name, int_8_dynamic_quantize=False, is_quantized_weight=False, quantization_type='linear', nbits=8, quant_scale=None, quant_bias=None, quant_lut=None)[source]

Add an inner product layer to the model. Refer to the InnerProductLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W: numpy.array or bytes()

Weight matrix of shape (output_channels, input_channels). If W is of type bytes() (quantized), other quantization related arguments must be provided as well (see below).

b: numpy.array

Bias vector of shape: (output_channels, ).

input_channels: int

Number of input channels.

output_channels: int

Number of output channels.

has_bias: boolean

Whether the bias vector of this layer is ignored in the spec.

• If True, the bias vector of this layer is not ignored.

• If False, the bias vector is ignored.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Quantization arguments, used when W is of type bytes():
int_8_dynamic_quantize: boolean

Whether to quantize and dequantize before and after inner product, respectively. Expects byte weights, representing int8 values, if True. See NeuralNetwork.proto for other validation conditions.

is_quantized_weight: bool, optional

Set it to true when W is of type bytes(), representing quantized weights, default: false.

quantization_type: str

When weights are quantized (that is, W is of type bytes()), this should be either "linear" or "lut".

nbits: int

Should be between 1 and 8 (inclusive). Number of bits per weight value. Only applicable when weights are quantized.

quant_scale: numpy.array(dtype=numpy.float32)

scale vector to be used with linear quantization. Must be of length either 1 or output_channels.

quant_bias: numpy.array(dtype=numpy.float32)

bias vector to be used with linear quantization. Must be of length either 1 or output_channels.

quant_lut: numpy.array(dtype=numpy.float32)

the LUT (look up table) to be used with LUT quantization. Must be of length 2^n bits.

Add L2 normalize layer. Normalizes the input by the L2 norm, i.e. divides by the the square root of the sum of squares of all elements of the input along C, H and W dimensions. Refer to the L2NormalizeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

epsilon: float

small bias to avoid division by zero.

add_layer_normalization(name, input_name, output_name, normalized_shape, gamma, beta, eps=1e-05)[source]

Add a layer normalization layer to the model that applies layer normalization over the input tensor. Refer to the LayerNormalizationLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

normalized_shape: list of int or tuple of int

Input shape from an expected input of size.

gamma: WeightParams

Weight parameters.

beta: WeightParams

Bias parameters.

eps: float, optional

Constant value added to the denominator, default: 1e-5.

Add a less_than layer to the model that performs the element-wise less-than (<) operation or less-than-or-equal-to (<=) operation. Broadcasting is supported. Refer to the LessThanL_ayerParams, LessEqualLayerParams messages in specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

use_less_than_equal: bool, optional

Whether or not to allow less than or equal to, default: false.

alpha: float, optional

y = x1 != alpha, if only one input is provided, default: 0.

Add a load constant layer. Refer to the LoadConstantLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

constant_value: numpy.array

value of the constant as a numpy array.

shape: list of int or tuple of int

List of ints representing the shape of the constant. Must be of length 3: [C,H,W]

Add a load_constant layer that loads data as a parameter and provides it as an output. Refer to the LoadConstantNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

constant_value: numpy.array()

value of the constant as a numpy array.

shape: list of int or tuple of int

List of ints representing the shape of the constant.

Add a logical layer to the model that performs element-wise logical and/or/xor/not operation. Broadcasting is supported. Refer to the LogicalOrLayerParams, LogicalNotLayerParams, LogicalNotLayerParams, and LogicalAndLayerParam messages in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

mode: str

Logical operation mode in [AND | OR | XOR | NOT].

add_loop(name, body_network=None, input_name=None, condition=None, condition_network=None, max_iterations=None)[source]

Add a loop layer to the model that provides the functionality of a for loop, or a while loop. Refer to the LoopLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

body_network: NeuralNetwork

Neural network to execute for the body of the loop.

input_name: str

The input blob name of this layer.

condition: str, optional

Condition of the loop.

condition_network: NeuralNetwork, optional

Neural network to execute for the condition of the loop.

max_iterations: int, optional

Maximum number of iterations of the loop.

Add a loop_break layer to the model that terminates the loop that contains this layer. Must reside in the bodyNetwork of the loop layer. Refer to the LoopBreakLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

Add a loop_continue layer to the model that stops the current loop iteration and continue on the next iteration. Must reside in the bodyNetwork of the loop layer. Refer to the LoopContinueLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

Add a lower_triangular layer to the model that copies a tensor setting everything outside lower triangular to zero. Refer to the LowerTriangularLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The of input blob name of this layer.

output_name: str

The output blob name of this layer.

k: int, optional

Diagonal below which to zero elements, default: 0 (main diagonal), k < 0 is lower it and k > 0 is upper.

add_lrn(name, input_name, output_name, alpha, beta, local_size, k=1.0)[source]

Add a LRN (local response normalization) layer. Supports “across” channels normalization. Refer to the LRNLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

alpha: float

multiplicative constant in the denominator.

beta: float

exponent of the normalizing term in the denominator.

k: float

bias term in the denominator. Must be positive.

local_size: int

size of the neighborhood along the channel axis.

add_matrix_band_part(name, input_name, output_name, num_lower=- 1, num_upper=- 1)[source]

Add a matrix_band_part layer to the model that copies a tensor setting everything outside a central band in each inner-most matrix to zero. Refer to the MatrixBandPartLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The of input blob name of this layer.

output_name: str

The output blob name of this layer.

num_lower: int, optional

Number of lower sub-diagonals to keep. Default: -1 (keep entire lower triangle).

num_upper: int, optional

Number of upper sub-diagonals to keep. Default: -1 (keep entire upper triangle).

Add a max_broadcastable layer to the model that performs element-wise maximum operation with broadcast support. Refer to the MaxBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a min_broadcastable layer to the model that performs element-wise minimum operation with broadcast support. Refer to the MinBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a mod_broadcastable layer to the model that performs element-wise modular operation with broadcast support. Refer to the ModBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a multiply_broadcastable layer to the model that performs element-wise multiplication operation with broadcast support. Refer to the MultiplyBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

add_mvn(name, input_name, output_name, across_channels=True, normalize_variance=True, epsilon=1e-05)[source]

Add an MVN (mean variance normalization) layer. Computes mean, variance and normalizes the input. Refer to the MeanVarianceNormalizeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

across_channels: boolean

If False, each channel plane is normalized separately If True, mean/variance is computed across all C, H and W dimensions

normalize_variance: boolean

If False, only mean subtraction is performed.

epsilon: float

small bias to avoid division by zero.

add_nms(name, input_names, output_names, iou_threshold=0.5, score_threshold=0.0, max_boxes=1, per_class_suppression=False)[source]

Add a non maximum suppression layer. Refer to the NonMaximumSuppressionLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer. Must be at least 2, and maximum 5.

output_names: list of str

The output blob names of this layer. Must be of length 4 exactly.

iou_threshold: float

intersection over union threshold for suppression. Ignored if 3rd input is present.

score_threshold: float

threshold for selecting boxes to be used for NMS algorithm. Ignored if 4th input is present.

max_boxes: int

maximum number of boxes to output. Ignored if 5th input is present.

per_class_suppression: bool

If true, boxes are organized into classes and suppression is applied to each class group separately

Add a not_equal layer to the model that performs the element-wise not equal (!=) operation. Broadcasting is supported. Refer to the NotEqualLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

alpha: float, optional

y = x1 != alpha, if only one input is provided, default: 0.

add_one_hot(name, input_names, output_name, one_hot_vector_size=None, axis=- 1, on_value=1.0, off_value=0.0)[source]

Add a one hot layer to the model that computes the one hot representation of the input tensor. Refer to the OneHotLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

one_hot_vector_size: int > 0

size of the one hot vector.

axis: int, optional

refers to the axis in the output tensor, default: -1.

on_value: float, optional

Constant value on locations represented by first input, default: 1.0.

off_value: float, optional

Constant value at all other locations, default: 0.0.

Add optional inputs and outputs to the model spec.

Parameters
optionals_in: list of str

List of inputs that are optionals.

optionals_out: list of str

List of outputs that are optionals.

Refer to the PaddingLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

left: int

Number of elements to be padded on the left side of the input blob.

right: int

Number of elements to be padded on the right side of the input blob.

top: int

Number of elements to be padded on the top of the input blob.

bottom: int

Number of elements to be padded on the bottom of the input blob.

value: float

Value of the elements padded. Used only when padding_type = 'constant'.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Type of the padding. Can be one of 'constant', 'reflection', or 'replication'.

Add a permute layer. Assumes that the input has dimensions in the order [Seq, C, H, W] Refer to the PermuteLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

dim: tuple

The order in which to permute the input dimensions = [seq,C,H,W]. Must have length 4 and a permutation of [0, 1, 2, 3].

examples:

Lets say input has shape: [seq, C, H, W].

If dim is set to [0, 3, 1, 2], then the output has shape [W,C,H] and has the same sequence length that of the input.

If dim is set to [3, 1, 2, 0], and the input is a sequence of data with length Seq and shape [C, 1, 1], then the output is a unit sequence of data with shape [C, 1, Seq].

If dim is set to [0, 3, 2, 1], the output is a reverse of the input: [C, H, W] -> [W, H, C].

If dim is not set, or is set to [0, 1, 2, 3], the output is the same as the input.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a pooling layer to the model that performs spatial pooling. Refer to the PoolingLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

height: int

Height of pooling region.

width: int

Width of pooling region.

stride_height: int

Stride along the height direction.

stride_width: int

Stride along the width direction.

layer_type: str

Type of pooling performed. Can either be 'MAX', 'AVERAGE', or 'L2'.

Option for the type of padding and output blob shape. Can be either 'VALID', 'SAME', or 'INCLUDE_LAST_PIXEL'.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Whether to exclude padded area in the 'AVERAGE' pooling operation, default: true. - If True, the value of the padded area will be excluded. - If False, the padded area will be included. This flag is only used with average pooling.

is_global: boolean

Whether the pooling operation is global. Defaults to False. - If True, the pooling operation is global. The pooling region

is of the same size of the input blob. Parameters height, width, stride_height, and stride_width will be ignored.

• If False, the pooling operation is not global.

values of height (top, bottom) and width (left, right) padding to be used if padding type is "VALID" or "INCLUDE_LAST_PIXEL".

Type of asymmetric padding to be used when padding_type = 'SAME'. Can be either 'BOTTOM_RIGHT_HEAVY' or 'TOP_LEFT_HEAVY'.

Add a pooling layer to the model that performs spatial pooling across three dimensions. Refer to the Pooling3DLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

pooling_type: str

Type of pooling performed. Can either be 'MAX' OR 'AVERAGE'.

kernel_depth: int

Depth of the pooling region.

kernel_height: int

Height of pooling region.

kernel_width: int

Width of pooling region.

stride_depth: int

Stride along the depth direction

stride_height: int

Stride along the height direction.

stride_width: int

Stride along the width direction.

Option for the padding type and output blob shape. Can be 'VALID', 'SAME', or 'CUSTOM'.

Padding before the input in the depth direction.

Padding after the input in the depth direction.

Padding before the input in the height direction.

Padding after the input in the height direction.

Padding before the input in the width direction.

Padding after the input in the width direction.

If true, exclude zeros from padding in average pooling. Can only be true for AVERAGE padding.

Add a pow_broadcastable layer to the model that performs element-wise power operation with broadcast support. Refer to the PowBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

add_random_bernoulli_dynamic(name, input_names, output_name, prob=0.5, seed=- 1)[source]

Add a random_bernoulli_dynamic layer to the model that fills the output tensor with random values from Bernoulli distribution. Refer to the RandomBernoulliDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

prob: float, optional

Probabilities for Bernoulli distribution, default: 0.5.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

add_random_bernoulli_like(name, input_name, output_name, prob=0.5, seed=- 1)[source]

Add a random_bernoulli_like layer to the model that fills the output tensor with random values from Bernoulli distribution. Refer to the RandomBernoulliLikeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

prob: float, optional

Probabilities for Bernoulli distribution, default: 0.5.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

add_random_bernoulli_static(name, output_name, output_shape, prob=0.5, seed=- 1)[source]

Add a random_bernoulli_static layer to the model that fills the output tensor with random values from Bernoulli distribution. Refer to the RandomBernoulliStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

Target shape of the output tensor.

prob: float, optional

Probabilities for Bernoulli distribution, default: 0.5.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

add_random_normal_dynamic(name, input_names, output_name, mean=0.0, stddev=0.0, seed=- 1)[source]

Add a random_normal_dynamic layer to the model that fills the output tensor with random values from normal distribution. Refer to the RandomNormalDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

mean: float, optional

The mean of the normal distribution, default: 0.0.

stddev: float, optional

The standard deviation of the normal distribution, default: 1.0.

seed: int, optional

Used to create a random seed for the distribution. Default -1 (random).

add_random_normal_like(name, input_name, output_name, mean=0.0, stddev=0.0, seed=- 1)[source]

Add a random_normal_like layer to the model that fills the output tensor with random values from normal distribution. Refer to the RandomNormalLikeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

mean: float, optional

The mean of the normal distribution, default: 0.0.

stddev: float, optional

The standard deviation of the normal distribution, default: 1.0.

seed: int, optional

Used to create a random seed for the distribution, default -1 (random).

add_random_normal_static(name, output_name, output_shape, mean=0.0, stddev=0.0, seed=- 1)[source]

Add a random_normal_static layer to the model that fills the output tensor with random values from normal distribution. Refer to the RandomNormaStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

Target shape of the output tensor.

mean: float, optional

The mean of the normal distribution, default: 0.0.

stddev: float, optional

The standard deviation of the normal distribution, default: 1.0.

seed: int, optional

Used to create a random seed for the distribution. Default -1 (random).

add_random_uniform_dynamic(name, input_names, output_name, minval=0.0, maxval=1.0, seed=- 1)[source]

Add a random_uniform_dynamic layer to the model that fills the output tensors with random values from uniform distribution. Refer to the RandomUniformDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

minval: float, optional

Lower bound / minimum value of the uniform distribution, default: 0.0.

maxval: float, optional

Upper bound / maximum value of the uniform distribution, default: 1.0.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

add_random_uniform_like(name, input_name, output_name, minval=0.0, maxval=1.0, seed=- 1)[source]

Add a random_uniform_like layer to the model that fills the output tensors with random values from uniform distribution. Refer to the RandomUniformLikeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

minval: float, optional

Lower bound / minimum value of the uniform distribution, default: 0.0.

maxval: float, optional

Upper bound / maximum value of the uniform distribution, default: 1.0.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

add_random_uniform_static(name, output_name, output_shape, minval=0.0, maxval=1.0, seed=- 1)[source]

Add a random_uniform_static layer to the model that fills the output tensors with random values from uniform distribution. Refer to the RandomUniformStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

Target shape of the output tensor.

minval: float, optional

Lower bound / minimum value of the uniform distribution, default: 0.0.

maxval: float, optional

Upper bound / maximum value of the uniform distribution, default: 1.0.

seed: int, optional

Used to create a random seed for the distribution. default -1 (random).

Add a range_dynamic layer that returns a tensor that contains evenly spaced values. This layer has up to three inputs or no input and three parameters. Refer to the RangeDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names. If input size == 1: end is input, start and step are read from parameters If input size == 2: end, start are inputs, step is read from parameters If input size == 3: start, end, step are all inputs, none of the parameters are used.

output_name: str

The output blob name of this layer.

start: int, optional

Range parameter: start. Ignored if start is provided as input, default: 0.

step: int, optional

Range parameter: step. Ignored if step is provided as input, default: 1.

add_range_static(name, output_name, input_names=None, end=1, start=0, step=1)[source]

Add a range_static layer that returns a tensor that contains evenly spaced values. This layer has no input and three parameters. Refer to the RangeStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

output_name: str

The output blob name of this layer.

input_names: list of str

The input blob names of this layer.

end: int, optional

Range parameter: end, default: 1.

start: int, optional

Range parameter: start, default: 0.

step: int, optional

Range parameter: step size, default: 1.

Add a rank_preserving_reshape layer to the model that reshapes the input tensor without altering the rank of the tensor. Refer to the RankPreservingReshapeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

Determines the shape of the output blob. 0: copy the dimension of the input to output -1: calculate dimensions from the rest of the shape

add_reduce(name, input_name, output_name, axis, mode, epsilon=1e-06)[source]

Add a reduce layer. Applies the function specified by the parameter mode, along dimension(s) specified by the parameter axis. Refer to the ReduceLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: str

dimensions along which the reduction operation is applied. Allowed values: ‘CHW’, ‘HW’, ‘C’, ‘H’, ‘W’

mode: str

Reduction operation to be applied. Allowed values: ‘sum’, ‘avg’, ‘prod’, ‘logsum’, ‘sumsquare’, ‘L1’, ‘L2’, ‘max’, ‘min’, ‘argmax’. ‘argmax’ is only supported with axis values ‘C’, ‘H’ and ‘W’.

epsilon: float

number that is added to the input when ‘logsum’ function is applied.

add_reduce_l1(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_l1 layer to the model that reduces the input tensor using l1_normalization(elements across given dimensions). Refer to the ReduceL1LayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_l2(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_l2 layer to the model that reduces the input tensor using l2_normalization(elements across given dimensions). Refer to the ReduceL2LayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_logsum(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_logsum layer to the model that reduces the input tensor using log(sum(elements across given dimensions)). Refer to the ReduceLogSumLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_logsumexp(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_logsumexp layer to the model that computes log(sum(exp(tensor))) and reduces along the given axis. Refer to the ReduceLogSumExpLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_max(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_max layer to the model that reduces the input tensor using max(elements across given dimensions). Refer to the ReduceMaxLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_mean(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_mean layer to the model that reduces the input tensor using mean(elements across given dimensions). Refer to the ReduceMeanLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_min(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_min layer to the model that reduces the input tensor using min(elements across given dimensions). Refer to the ReduceMinLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_prod(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_prod layer to the model that reduces the input tensor using prod(elements across given dimensions). Refer to the ReduceProdLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes. If axes list is empty, it will be set to true, default: false.

add_reduce_sum(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_sum layer to the model that reduces the input tensor using sum(elements across given dimensions). Refer to the ReduceSumLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all).

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

add_reduce_sumsquare(name, input_name, output_name, axes=None, keepdims=True, reduce_all=False)[source]

Add a reduce_sumsquare layer to the model that reduces the input tensor using sum(square(elements across given dimensions)). Refer to the ReduceSumSquareLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

List of dimensions for the reduce operations. Each should be in range [-rank(input), rank(input)), default: None (reduce_all)

keepdims: bool, optional

Whether or not to retain the reduced dimensions with length 1, default: true.

reduce_all: bool, optional

Whether or not to reduce on all axes, default: false.

Add a data reorganization layer of type “SPACE_TO_DEPTH” or “DEPTH_TO_SPACE”. Refer to the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

mode: str
• If mode == ‘SPACE_TO_DEPTH’: data is moved from the spatial to the channel dimension. Input is spatially divided into non-overlapping blocks of size block_size X block_size and data from each block is moved to the channel dimension. Output CHW dimensions are: [C * block_size * block_size, H/block_size, C/block_size].

• If mode == ‘DEPTH_TO_SPACE’: data is moved from the channel to the spatial dimension. Reverse of the operation ‘SPACE_TO_DEPTH’. Output CHW dimensions are: [C/(block_size * block_size), H * block_size, C * block_size].

• If mode == ‘PIXEL_SHUFFLE’: data is moved from the channel to the spatial dimension. Reverse of the operation ‘SPACE_TO_DEPTH’. Output CHW dimensions are: [C/(block_size * block_size), H * block_size, C * block_size].

block_size: int

Must be greater than 1. Must divide H and W, when mode is ‘SPACE_TO_DEPTH’. (block_size * block_size) must divide C when mode is ‘DEPTH_TO_SPACE’ or ‘PIXEL_SHUFFLE’.

Add a reshape layer. Refer to the ReshapeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

target_shape: tuple

Shape of the output blob. The product of target_shape must be equal to the shape of the input blob. Can be either length 3 (C,H,W) or length 4 (Seq,C,H,W).

mode: int
• If mode == 0, the reshape layer is in CHANNEL_FIRST mode.

• If mode == 1, the reshape layer is in CHANNEL_LAST mode.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a reshape_dynamic layer to the model that reshapes a tensor. Refer to the ReshapeDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a reshape_like layer to the model that reshapes a tensor. Refer to the ReshapeLikeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a reshape_static layer to the model that reshapes a tensor. Refer to the ReshapeStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

output_shape: list of int or tuple of int

Target shape of the output tensor.

add_resize_bilinear(name, input_name, output_name, target_height=1, target_width=1, mode='ALIGN_ENDPOINTS_MODE')[source]

Add a resize bilinear layer to the model. A layer that resize the input to a given spatial size using bilinear interpolation. Refer to the ResizeBilinearLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

target_height: int

Output height dimension.

target_width: int

Output width dimension.

mode: str

Following values are supported: ‘STRICT_ALIGN_ENDPOINTS_MODE’, ‘ALIGN_ENDPOINTS_MODE’, ‘UPSAMPLE_MODE’, ‘ROI_ALIGN_MODE’. This parameter determines the sampling grid used for bilinear interpolation.

Add a reverse layer to the model that reverses specific dimensions of the input tensor. Refer to the ReverseLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

reverse_dim: list of int or tuple of int

Reverse along the dimension, default [1].

add_reverse_sequence(name, input_names, output_name, batch_axis=0, seq_axis=- 1)[source]

Add a reverse sequence layer to the model that reverses variable length slices. Refer to the ReverseSeqLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

batch_axis: int, optional

Slices input along the dimension batch_axis, default 0.

seq_axis: int, optional

Reverse along the dimension seq_axis, default: -1.

Add a round layer to the model that performs element-wise round operation on the input tensor that rounds the value to the nearest integer. Refer to the RoundLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_scale(name, W, b, has_bias, input_name, output_name, shape_scale=None, shape_bias=None)[source]

Add a scale layer to the model. Refer to the ScaleLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W: int or numpy.array

Scale of the input.

b: int or numpy.array

Bias to add to the input.

has_bias: boolean

Whether the bias vector of this layer is ignored in the spec.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

shape_scale: list of int or tuple of int

List of ints that specifies the shape of the scale parameter. Can be [1], [C], [1,H,W], or [C,H,W].

shape_bias: list of int

List of ints that specifies the shape of the bias parameter (if present). Can be [1], [C], [1,H,W], or [C,H,W].

Add a scatter layer to the model that scatters data into a new tensor according to indices from the input. Refer to the ScatterLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int

The axis the operation perform on, default: 0.

mode: str, optional

Scatter accumulation mode in [UPDATE | ADD | SUB | MUL | DIV | MAX | MIN], default: UPDATE.

Add a scatter_along_axis layer to the model that scatters data into a new tensor according to indices from the input along the given axis into the output tensor. Refer to the ScatterAlongAxisLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int

The axis to perform on, default: 0.

mode: str, optional

Scatter accumulation mode in [UPDATE | ADD | SUB | MUL | DIV | MAX | MIN], default: UPDATE

Add a scatter layer to the model that scatters data into a new tensor according to indices from input. This is the reverse operation of the gather operation. Refer to the ScatterNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

mode: str, optional

Scatter accumulation mode in [UPDATE | ADD | SUB | MUL | DIV | MAX | MIN], default: UPDATE

Add a sequence repeat layer to the model. Refer to the SequenceRepeatLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

nrep: int

Number of repetitions of the input blob along the sequence axis.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a sign layer to the model that performs element-wise sign operation (+1 for positive values, -1 for negative values, 0 for zeroes). Refer to the SignLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_simple_rnn(name, W_h, W_x, b, hidden_size, input_size, activation, input_names, output_names, output_all=False, reverse_input=False)[source]

Add a simple recurrent layer to the model. Refer to the SimpleRecurrentLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W_h: numpy.array

Weights of the recurrent layer’s hidden state. Must be of shape (hidden_size, hidden_size).

W_x: numpy.array

Weights of the recurrent layer’s input. Must be of shape (hidden_size, input_size).

b: numpy.array or None

Bias of the recurrent layer’s output. If None, bias is ignored. Otherwise it must be of shape (hidden_size, ).

hidden_size: int

Number of hidden units. This is equal to the number of channels of output shape.

input_size: int

Number of the number of channels of input shape.

activation: str

Activation function name. Can be one of the following option: ['RELU', 'TANH', 'SIGMOID', 'SCALED_TANH', 'SIGMOID_HARD', 'LINEAR']. See add_activation for more detailed description.

input_names: list of str

The input blob names list of this layer, in the order of [x, h_input].

output_names: list of str

The output blob names list of this layer, in the order of [y, h_output].

output_all: boolean

Whether the recurrent layer should output at every time step.

• If False, the output is the result after the final state update.

• If True, the output is a sequence, containing outputs at all time steps.

reverse_input: boolean

Whether the recurrent layer should process the input sequence in the reverse order.

• If False, the input sequence order is not reversed.

• If True, the input sequence order is reversed.

Add a sin layer to the model that computes element-wise sine for the input tensor. Refer to the SinLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a sinh layer to the model that computes element-wise hyperbolic sine for the input tensor. Refer to the SinhLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_slice(name, input_name, output_name, axis, start_index=0, end_index=- 1, stride=1)[source]

Add a slice layer. Equivalent to to numpy slice [start_index:end_index:stride], start_index is included, while end_index is exclusive. Refer to the SliceLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: str

axis along which input is sliced. allowed values: ‘channel’, ‘height’, ‘width’

start_index: int

must be non-negative.

end_index: int

negative indexing is supported.

stride: int

must be positive.

Add a slice layer. Equivalent to to numpy slice [start_index: start_index+size], Input is list of str which is [input_tensor, begin_id].

Assume input_tensor has shape (2, 3, 4), and axis=1, size=2. This would produce input_tensor[:, begin_id:begin_id+2, :]

Refer to the SliceBySizeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int

axis along which input is sliced.

size: int

The size of which input will be taken

Add a slice_dynamic layer to the model that extracts a slice of size (end - begin) / stride from the given input tensor. Refer to the SliceDynamicLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

end_ids: list of int or tuple of int, optional

End offsets for slice layer, default: [1].

strides: list of int or tuple of int, optional

Strides for slice layer, default: [1].

Boolean masks for begin offsets, default: [false].

Boolean masks for end offsets, default: [false].

Boolean masks for squeezing axis, default: [false].

Add a slice_static layer to the model that extracts a slice of size (end - begin) / stride from the given input tensor. Refer to the SliceStaticLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

begin_ids: list of int or tuple of int

Begin offsets for slice layer.

end_ids: list of int or tuple of int

End offsets for slice layer.

strides: list of int or tuple of int

Strides for slice layer.

add_sliding_windows(name, input_name, output_name, axis, window_size, step=1)[source]

Add a sliding_windows layer to the model that returns a tensor containing all windows of size window_size * separated by step along the dimension axis. Refer to the SlidingWindowsLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The of input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: int

Axis to perform the operation.

window_size: int

Number of elements in the sliding window.

step: int, optional

The stride of the input elements in the sliding window, default: 1.

Add a softmax layer to the model. Refer to the SoftmaxLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a softmax_nd layer to the model that performs softmax operation along the given axis. Refer to the SoftmaxNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axis: int

Axis to perform the softmax operation on.

Add a split layer that uniformly splits the input along the channel dimension to produce multiple outputs. Refer to the SplitLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_names: list of str

List of output blob names of this layer.

add_split_nd(name, input_name, output_names, axis, num_splits=2, split_sizes=None)[source]

Add a split layer to the model that splits the input tensor into multiple output tensors. Either uniformly split the input tensor into num_splits tensors, or split into given size list split_sizes output tensors. Refer to the SplitNDLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_names: list of str

The output blob names of this layer.

axis: int

Axis to perform split on.

num_splits: int, optional

Number of splits, default: 2.

split_sizes: list of int or tuple of int, optional

List of size to split, default [] or None.

Add a squeeze layer to the model that decrease the rank of the input tensor by removing unit dimensions. Refer to the SqueezeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

axes: list of int or tuple of int, optional

Dimensions to perform the operation, default: None (squeeze_all).

squeeze_all: bool, optional

If true, all dimensions that are 1 are squeezed, default: false.

Add a stack layer to the model that performs stack operation on a list of tensors into one rank+1 tensor on the given axis. Refer to the StackLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

axis: int, optional

The axis to perform stack operation, default: 0.

Add a subtract_broadcastable layer to the model that performs element-wise subtraction operation with broadcast support. Refer to the SubtractBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a tan layer to the model that computes element-wise tangent for the input tensor. Refer to the TanLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a tanh layer to the model that computes element-wise hyperbolic tangent for the input tensor. Refer to the TanhLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

Add a tile layer to the model that construct a tensor by repeating the input tensor multiple number of times. Refer to the TileLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str or list[str]

The input blob name of this layer. If second input is provided, reps parameter is ignored.

output_name: str

The output blob name of this layer.

reps: list of int or tuple of int

Number of times to replicate. If input_name provides two inputs, second input is used as reps and this parameter is ignored.

add_topk(name, input_names, output_names, k=0, axis=0, use_bottom_k=False)[source]

Add a topk layer to the model that returns top or bottom k values and the corresponding indices of the input tensor along a given axis. Refer to the TopKLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer. It must be of length 1 or 2. The optional second input corresponds to value of K.

output_names: list of str

The output blob names of this layer. First and second correspond to values and indices, respectively.

k: int, optional

number of values/indices to be computed along the axis. Need not be given of there are two inputs, default: 0.

axis: int, optional

axis along which the topk values/indices are computed. negative indexing is supported, default: 0

use_bottom_k: bool, optional

if true, bottom k values are computed instead, default: false.

Add a N-D transpose layer with axes as a parameter. Refer to the TransposeLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

axes: list of int or tuple of int

The list containing a permutation of “[0,1,2,…,N-1]” where N is the rank of input/output tensor.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

add_unary(name, input_name, output_name, mode, alpha=1.0, shift=0, scale=1.0, epsilon=None)[source]

Add a Unary layer. Applies the specified function (mode) to all the elements of the input. Prior to the application of the function the input can be scaled and shifted by using the ‘scale’, ‘shift’ parameters. Refer to the UnaryFunctionLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

mode: str

Unary function. Allowed values: ‘sqrt’, ‘rsqrt’, ‘inverse’, ‘power’, ‘exp’, ‘log’, ‘abs’, threshold’.

alpha: float

constant used in with modes ‘power’ and ‘threshold’.

shift, scale: float

input is modified by scale and shift prior to the application of the unary function.

epsilon: float

small bias to prevent division by zero.

add_unilstm(name, W_h, W_x, b, hidden_size, input_size, input_names, output_names, inner_activation='SIGMOID', cell_state_update_activation='TANH', output_activation='TANH', peep=None, output_all=False, forget_bias=False, coupled_input_forget_gate=False, cell_clip_threshold=50000.0, reverse_input=False)[source]

Add a Uni-directional LSTM layer to the model. Refer to the UniDirectionalLSTMLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

W_h: [numpy.array]

List of recursion weight matrices. The ordering is [R_i, R_f, R_o, R_z], where R_i, R_f, R_o, R_z are weight matrices at input gate, forget gate, output gate and cell gate. The shapes of these matrices are (hidden_size, hidden_size).

W_x: [numpy.array]

List of input weight matrices. The ordering is [W_i, W_f, W_o, W_z], where W_i, W_f, W_o, W_z are weight matrices at input gate, forget gate, output gate and cell gate. The shapes of these matrices are (hidden_size, input_size).

b: [numpy.array] or None

List of biases. The ordering is [b_i, b_f, b_o, b_z], where b_i, b_f, b_o, b_z are biases at input gate, forget gate, output gate and cell gate. If None, biases are ignored. Otherwise the shapes of the biases are (hidden_size, ).

hidden_size: int

Number of hidden units. This is equal to the number of channels of output shape.

input_size: int

Number of the number of channels of input shape.

input_names: list of str

The input blob names list of this layer, in the order of [x, h_input, c_input].

output_names: list of str

The output blob names list of this layer, in the order of [y, h_output, c_output].

inner_activation: str

Inner activation function used at input and forget gate. Can be one of the following option: [‘RELU’, ‘TANH’, ‘SIGMOID’, ‘SCALED_TANH’, ‘SIGMOID_HARD’, ‘LINEAR’].

cell_state_update_activation: str

Cell state update activation function used at the cell state update gate. [‘RELU’, ‘TANH’, ‘SIGMOID’, ‘SCALED_TANH’, ‘SIGMOID_HARD’, ‘LINEAR’].

output_activation: str

Activation function used at the output gate. Can be one of the following option: [‘RELU’, ‘TANH’, ‘SIGMOID’, ‘SCALED_TANH’, ‘SIGMOID_HARD’, ‘LINEAR’].

peep: [numpy.array] or None

List of peephole vectors. The ordering is [p_i, p_f, p_o], where p_i, p_f, and p_o are peephole vectors at input gate, forget gate, output gate. The shapes of the peephole vectors are (hidden_size,).

output_all: boolean

Whether the LSTM layer should output at every time step.

• If False, the output is the result after the final state update.

• If True, the output is a sequence, containing outputs at all time steps.

forget_bias: boolean

If True, a vector of 1s is added to forget gate bias.

coupled_input_forget_gate: boolean

If True, the input gate and forget gate is coupled. i.e. forget gate is not used.

cell_clip_threshold: float

The limit on the maximum and minimum values on the cell state. If not provided, it is defaulted to 50.0.

reverse_input: boolean

Whether the LSTM layer should process the input sequence in the reverse order.

• If False, the input sequence order is not reversed.

• If True, the input sequence order is reversed.

Add a upper_triangular layer to the model that copies a tensor setting everything outside upper triangular to zero. Refer to the UpperTriangularLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The of input blob name of this layer.

output_name: str

The output blob name of this layer.

k: int, optional

Diagonal above which to zero elements, default: 0 (main diagonal), k < 0 is lower it and k > 0 is upper.

add_upsample(name, scaling_factor_h, scaling_factor_w, input_name, output_name, mode='NN', linear_upsample_mode='DEFAULT')[source]

Add an upsample layer to the model. Refer to the UpsampleLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

scaling_factor_h: int or float

Scaling factor on the vertical direction. Float values only supported with BILINEAR and ALIGN_CORNERS_*.

scaling_factor_w: int or float

Scaling factor on the horizontal direction. Float values only supported with BILINEAR and ALIGN_CORNERS_*.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

mode: str

Overall interpolation mode. The following values are supported:

• 'NN': nearest neighbour

• 'BILINEAR': bilinear interpolation

linear_upsample_mode: str

Specifies the behavior for linear upsampling. Only valid when Interpolation Mode is BILINEAR.

If input grid is [0, Xin-1] (corresponding to an input size of Xin), and if the output size is Xout, then the grid points are sampled in the following manner:

‘DEFAULT’:
• spacing = (Xin-Xin/Xout) / (Xout-1)

• grid_point[i] = min(Xin-1, max(0, i * spacing)), for i = 0,1,2,..,Xout-1

‘ALIGN_CORNERS_TRUE’:
• spacing = (Xin-1) / (Xout-1)

• grid_point[i] = min(Xin-1, max(0, i * spacing)), for i = 0,1,2,..,Xout-1

‘ALIGN_CORNERS_FALSE’:
• spacing = Xin / Xout

• grid_point[i] = min(Xin-1, max(0, i * spacing + 0.5 * spacing - 0.5)), for i = 0,1,2,..,Xout-1

Add a where_broadcastable layer to the model that returns the elements either from tensor x or tensor y, depending on the value in the condition tensor. Refer to the WhereBroadcastableLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_names: list of str

The input blob names of this layer.

output_name: str

The output blob name of this layer.

Add a where_nonzero layer to the model that returns a tensor containing the indices of all non-zero elements of input tensor. Refer to the WhereNonZeroLayerParams message in the specification (NeuralNetwork.proto) for more details.

Parameters
name: str

The name of this layer.

input_name: str

The input blob name of this layer.

output_name: str

The output blob name of this layer.

inspect_conv_channels(layer_name)[source]

Prints the output and kernel channels of a convolution layer.

inspect_innerproduct_channels(layer_name)[source]

Prints the output and kernel channels of an innerProduct layer.

inspect_input_features()[source]

Prints the name and type of input features.

inspect_layers(last=- 1, verbose=False)[source]

Prints the summary for last “last” number of layers.

Parameters
last: int

The numbers of layers to inspect, starting from the last one.

verbose: bool

Whether to display layer-specific parameters or not.

inspect_loss_layers()[source]

Prints the summary for the loss layer.

inspect_optimizer()[source]

Prints the summary for the optimizer.

inspect_output_features()[source]

Prints the name and type of output features.

inspect_updatable_layers()[source]

Prints all updatable layers with their inputs and outputs.

make_updatable(trainables)[source]

Make the builder’s NeuralNetwork spec updatable.

Parameters
trainables: list of str

List of layer names to be set trainable.

set_categorical_cross_entropy_loss(name, input)[source]

Categorical Cross Entropy is used for single label categorization (only one category is applicable for each data point).

Parameters
name: The name of the loss layer
input: The name of the input

The input should be a vector of length N representing the distribution over N categories. This must be the output of a softmax.

Notes

$Loss_ {CCE}(input, target) = -\sum_{i = 1} ^ {N}(target == i) log(input[i]) = - log(input[target])$
set_class_labels(class_labels, predicted_feature_name='classLabel', prediction_blob='')[source]

Set class labels to the model spec to make it a neural network classifier.

Parameters
class_labels: list of int or list of str

A list of integers or strings that map the index of the output of a neural network to labels in a classifier.

predicted_feature_name: str

Name of the output feature for the class labels exposed in the Core ML neural network classifier, defaults: 'classLabel'.

prediction_blob: str

If provided, then this is the name of the neural network blob which generates the probabilities for each class label (typically the output of a softmax layer). If not provided, then the last output layer is assumed.

set_input(input_names, input_dims)[source]

Set the inputs of the network spec.

Parameters
input_names: list of str

The input names of the network.

input_dims: [tuple]

The input dimensions of the network. The ordering of input_dims is the same as input_names.

Examples

# Set the neural network spec inputs to be 3 dimensional vector data1 and
# 4 dimensional vector data2.
>>> builder.set_input(input_names=['data1', 'data2'], input_dims=[(3,), (4,)])

set_mean_squared_error_loss(name, input_feature=None)[source]
input_feature: [(str, datatypes.Array)] or None

The input feature of the loss layer. Each feature is a (name, array) tuple, where name is the name of the model’s tensor our loss will be attached to, and array is a datatypes.Array object describing the shape of that tensor. Both the name and the array’s shape must be provided in the tuple.

Examples

>>> feature = [('output_tensor', datatypes.Array((299, 299, 3)))]

set_optional_input(input_idx, value=None, format='float')[source]

Marks given input as optional input. Optionally, sets default value for optional input if value is not None.

Parameters
input_idx: int

Index of input to be marked and fill with default value.

value: int/double/float/None

Value to be fill as default value.

format: str

Format of default value. Must be one of 'float', 'double', or 'int'.

set_output(output_names, output_dims)[source]

Set the outputs of the network spec.

Parameters
output_names: list of str

The output names of the network.

output_dims: [tuple]

The output dimensions of the network. The ordering of output_dims is the same as output_names.

Examples

# Set the neural network spec outputs to be 3 dimensional vector feature1 and
# 4 dimensional vector feature2.
>>> builder.set_output(output_names=['feature1', 'feature2'], output_dims=[(3,), (4,)])

set_pre_processing_parameters(image_input_names=None, is_bgr=False, red_bias=0.0, green_bias=0.0, blue_bias=0.0, gray_bias=0.0, image_scale=1.0, image_format='NCHW')[source]

Add a pre-processing parameters layer to the neural network object.

Parameters
image_input_names: list of str

Name of input blobs that are images

is_bgr: boolean or dict()

Channel order for input blobs that are images. BGR if True else RGB. To specify a different value for each image input, provide a dictionary with input names as keys.

red_bias: float or dict()

Image re-centering parameter (red channel)

blue_bias: float or dict()

Image re-centering parameter (blue channel)

green_bias: float or dict()

Image re-centering parameter (green channel)

gray_bias: float or dict()

Image re-centering parameter (for grayscale images)

image_scale: float or dict()

Value by which to scale the images.

image_format: str

Image format, either ‘NCHW’ / ‘NHWC’

set_training_input(training_input)[source]

Set the training inputs of the network spec.

Parameters
training_input: [tuple]

The training input names and type of the network.

Examples

# Set the neural network spec training inputs to be 3 dimensional vector for 'input' and
# Double for 'target'.
>>> builder.set_training_input([('input', datatypes.Array(3)), ('target', 'Double')])


# neural_network.flexible_shape_utils

Utilities to annotate Neural Network Features with flexible shape information. Only available in coremltools 2.0b1 and onwards

class coremltools.models.neural_network.flexible_shape_utils.NeuralNetworkImageSize(height=None, width=None)[source]

An object representing a size for an image feature inside a neural network. Valid sizess for height and width are > 0.

class coremltools.models.neural_network.flexible_shape_utils.NeuralNetworkImageSizeRange(height_range=None, width_range=None)[source]

An object representing a range of sizes for an image feature inside a neural network. Valid ranges for height and width are > 0. A “-1” upper bound value for either width or height represents an unbounded size for that dimension.

class coremltools.models.neural_network.flexible_shape_utils.NeuralNetworkMultiArrayShape(channel=None, height=None, width=None)[source]

An object representing a shape for a multiArray feature in a neural network. Valid shapes must have have only the Channel [C] shape or the Channel, Height and Width [C, H, W] shapes populated

class coremltools.models.neural_network.flexible_shape_utils.NeuralNetworkMultiArrayShapeRange(input_ranges=None)[source]

An object representing a range of shapes for a multiArray feature in a neural network. Valid shape ranges must have have only the Channel [C] range or the Channel, Height and Width [C, H, W] ranges populated. A “-1” value in an upper bound represents an unbounded range.

isFlexible()[source]

Returns true if any one of the channel, height, or width ranges of this shape allow more than one input value.

Annotate an input or output image feature in a Neural Network spec to to accommodate a list of enumerated image sizes

Parameters
• spec – MLModel The MLModel spec containing the feature

• feature_name – str The name of the image feature for which to add size information. If the feature is not found in the input or output descriptions then an exception is thrown

• sizes – [] | NeuralNetworkImageSize A single or a list of NeuralNetworkImageSize objects which encode valid size information for a image feature

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import flexible_shape_utils
>>> image_sizes = [flexible_shape_utils.NeuralNetworkImageSize(128, 128)]
>>> image_sizes.append(flexible_shape_utils.NeuralNetworkImageSize(256, 256))

Returns

None. The spec object is updated

Annotate an input or output multiArray feature in a Neural Network spec to to accommodate a list of enumerated array shapes

Parameters
• spec – MLModel The MLModel spec containing the feature

• feature_name – str The name of the image feature for which to add shape information. If the feature is not found in the input or output descriptions then an exception is thrown

• shapes – [] | NeuralNetworkMultiArrayShape A single or a list of NeuralNetworkImageSize objects which encode valid size information for a image feature

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import flexible_shape_utils
>>> array_shapes = [flexible_shape_utils.NeuralNetworkMultiArrayShape(3)]
>>> second_shape = flexible_shape_utils.NeuralNetworkMultiArrayShape()
>>> second_shape.set_channel_shape(3)
>>> second_shape.set_height_shape(10)
>>> second_shape.set_width_shape(15)
>>> array_shapes.append(second_shape)

Returns

None. The spec object is updated

Annotate an input or output MLMultiArray feature in a Neural Network spec to accommodate a range of shapes. Add provided enumerated shapes to the list of shapes already present. This method is different from “add_enumerated_multiarray_shapes”, which is applicable for rank 5 mapping, SBCHW, arrays.

Parameters
• spec – MLModel The MLModel spec containing the feature

• feature_name – str The name of the feature for which to add shape range information. If the feature is not found in the input or output descriptions then an exception is thrown

• enumerated_shapes – List[Tuple(int)] list of shapes, where each shape is specified as a tuple of integers.

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import flexible_shape_utils
>>> # say, the default shape of "my_multiarray_featurename" is (2,3)

Returns

None. The spec is updated

coremltools.models.neural_network.flexible_shape_utils.set_multiarray_ndshape_range(spec, feature_name, lower_bounds, upper_bounds)[source]

Annotate an input or output MLMultiArray feature in a Neural Network spec to accommodate a range of shapes. This is different from “update_multiarray_shape_range”, which works with rank 5 SBCHW mapping.

Parameters
• spec – MLModel The MLModel spec containing the feature

• feature_name – str The name of the feature for which to add shape range information. If the feature is not found in the input or output descriptions then an exception is thrown

• lower_bounds – List[int] list of integers specifying the lower bounds of each dimension. Length must be same as the rank (length of shape) of the feature_name.

• upper_bounds – List[int] list of integers specifying the upper bounds of each dimension. -1 corresponds to unbounded range. Length must be same as the rank (length of shape) of the feature_name.

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import flexible_shape_utils
>>> # say, the default shape of "my_multiarray_featurename" is (2,3)
>>> flexible_shape_utils.set_multiarray_ndshape_range(spec, feature_name='my_multiarray_featurename', lower_bounds=[1,2], upper_bounds=[10,-1])

Returns

None. The spec is updated

coremltools.models.neural_network.flexible_shape_utils.update_image_size_range(spec, feature_name, size_range)[source]

Annotate an input or output Image feature in a Neural Network spec to to accommodate a range of image sizes

Parameters
• spec – MLModel The MLModel spec containing the feature

• feature_name – str The name of the Image feature for which to add shape information. If the feature is not found in the input or output descriptions then an exception is thrown

• size_range – NeuralNetworkImageSizeRange A NeuralNetworkImageSizeRange object with the populated image size range information.

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import flexible_shape_utils
>>> img_size_ranges = flexible_shape_utils.NeuralNetworkImageSizeRange()
>>> flexible_shape_utils.update_image_size_range(spec, feature_name='my_multiarray_featurename', size_range=img_size_ranges)

Returns

None. The spec object is updated

coremltools.models.neural_network.flexible_shape_utils.update_multiarray_shape_range(spec, feature_name, shape_range)[source]

Annotate an input or output MLMultiArray feature in a Neural Network spec to accommodate a range of shapes

Parameters
• spec – MLModel The MLModel spec containing the feature

• feature_name – str The name of the feature for which to add shape range information. If the feature is not found in the input or output descriptions then an exception is thrown

• shape_range – NeuralNetworkMultiArrayShapeRange A NeuralNetworkMultiArrayShapeRange object with the populated shape range information. The shape_range object must either contain only shape information for channel or channel, height and width. If the object is invalid then an exception is thrown

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import flexible_shape_utils
>>> shape_range = flexible_shape_utils.NeuralNetworkMultiArrayShapeRange()
>>> flexible_shape_utils.update_multiarray_shape_range(spec, feature_name='my_multiarray_featurename', shape_range=shape_range)

Returns

None. The spec is updated

# neural_network.quantization_utils

Utilities to compress Neural Network Models. Only available in coremltools 2.0b1 and onwards

Quantized layer selector allowing the user to specify some types of layers to skip during quantization process and the minimum size parameters in quantized convolution layers.

Examples

from coremltools.models.neural_network.quantization_utils import AdvancedQuantizedLayerSelector
skip_layer_types=['batchnorm', 'bias', 'depthwiseConv'],
minimum_conv_kernel_channels=4,
minimum_conv_weight_count=4096)
quantized_model = quantize_weights(model, 8, selector=selector)

do_quantize(layer, weight_param=None)[source]

weight_param - should be name of the WeightParam field

class coremltools.models.neural_network.quantization_utils.MatrixMultiplyLayerSelector(minimum_weight_count=1, minimum_input_channels=1, minimum_output_channels=1, maximum_input_channels=None, maximum_output_channels=None, include_layers_with_names=None)[source]

Layer selector object that allows users to select matrix multiplication layers with one of the matrices being constant, based on some criterions like total numbers of parameters/weights, number of input or output channels and/or layer names. If any of the criterion is not valid, the corresponding layer is not selected.

do_quantize(layer, weight_param=None)[source]

weight_param - should be name of the WeightParam field

class coremltools.models.neural_network.quantization_utils.ModelMetrics(spec)[source]

A utility class to hold evaluation metrics

class coremltools.models.neural_network.quantization_utils.OutputMetric(name, type)[source]

Utility class to calculate and hold metrics between two model outputs

class coremltools.models.neural_network.quantization_utils.QuantizedLayerSelector[source]

This is the base class to implement custom selectors to skip certain layers during quantization. To implement a custom selector, create a class that inherits this class and override do_quantize() method.

Examples

class MyLayerSelector(QuantizedLayerSelector):
def __init__(self):
super(MyLayerSelector, self).__init__()

def do_quantize(self, layer, **kwargs):
ret = super(MyLayerSelector, self).do_quantize(layer)
if not ret or layer.name == 'dense_2':
return False
return True

selector = MyLayerSelector()
quantized_model = quantize_weights(mlmodel, 8, quantization_mode='linear', selector=selector)

coremltools.models.neural_network.quantization_utils.activate_int8_int8_matrix_multiplications(spec, selector=None)[source]

Utility function that takes in either a full precision (float) spec or an nbit quantized spec to selectively enable int8 activation + weight quantization of matrix multiplication operations where the second matrix represents a constant weight.

spec: MLModel.get_spec()

Currently conversion for only neural network models is supported. If a pipeline model is passed in then all embedded neural network models embedded within will be modified.

selector: (optional) MatrixMultiplyLayerSelector

A MatrixMultiplyLayerSelector object that enables int8 activation + weight quantization only on those layers for which the user-specified criterion on the minimum/maximum number of size/channels in constant weight parameters is met. It can also be derived to provide custom selection.

coremltools.models.neural_network.quantization_utils.compare_models(full_precision_model, quantized_model, sample_data)[source]

Utility function to compare the performance of a full precision vs quantized model

full_precision_model: MLModel

The full precision model with float32 weights

quantized_model: MLModel

Quantized version of the model with quantized weights

sample_data: str | [dict]

Data used to characterize performance of the quantized model in comparison to the full precision model. Either a list of sample input dictionaries or an absolute path to a directory containing images. Path to a directory containing images is only valid for models with one image input. For all other models a list of sample inputs must be provided.

Returns

None. Performance metrics are printed out

coremltools.models.neural_network.quantization_utils.quantize_weights(full_precision_model, nbits, quantization_mode='linear', sample_data=None, **kwargs)[source]

Utility function to convert a full precision (float) MLModel to a nbit quantized MLModel (float16).

full_precision_model: MLModel

Model which will be converted to half precision. Currently conversion for only neural network models is supported. If a pipeline model is passed in then all embedded neural network models embedded within will be converted.

nbits: int
Number of bits per quantized weight. Only 16-bit float point and

1-8 bit is supported

quantization_mode: str

One of the following:

“linear”:

Linear quantization with scale and bias assuming the range of weight values is [A, B], where A = min(weight), B = max(weight)

“linear_lut”:

Simple linear quantization represented as a lookup table

“kmeans_lut”:

LUT based quantization, where LUT is generated by K-Means clustering

“custom_lut”:

LUT quantization where LUT and quantized weight params are calculated using a custom function. If this mode is selected then a custom function must be passed in kwargs with key lut_function. The function must have input params (nbits, wp) where nbits is the number of quantization bits and wp is the list of weights for a given layer. The function should return two parameters (lut, qw) where lut is an array of length (2^n bits)containing LUT values and qw is the list of quantized weight parameters. See _get_linear_lookup_table_and_weight for a sample implementation.

“linear_symmetric”:

Linear quantization with scale and bias assuming the range of weight values is [-A, A], where A = max(abs(weight)).

sample_data: str | [dict]

Data used to characterize performance of the quantized model in comparison to the full precision model. Either a list of sample input dictionaries or an absolute path to a directory containing images. Path to a directory containing images is only valid for models with one image input. For all other models a list of sample inputs must be provided.

kwargs: keyword arguments
lut_function(callable function)

A callable function provided when quantization mode is set to _QUANTIZATION_MODE_CUSTOM_LOOKUP_TABLE. See quantization_mode for more details.

selector: QuantizedLayerSelector

A QuanatizedLayerSelector object that can be derived to provide custom quantization selection.

Returns
model: MLModel

The quantized MLModel instance if running on macOS 10.14 or later, otherwise the quantized model specification is returned

Examples

>>> import coremltools
>>> from coremltools.models.neural_network import quantization_utils
>>> model = coremltools.models.MLModel('my_model.mlmodel')
>>> quantized_model = quantization_utils.quantize_weights(model, 8, "linear")
`

# neural_network.update_optimizer_utils

Neural Network optimizer utilities.

class coremltools.models.neural_network.update_optimizer_utils.AdamParams(lr=0.01, batch=10, beta1=0.9, beta2=0.999, eps=1e-08)[source]

Adam - A Method for Stochastic Optimization.

Attributes
lr: float

The learning rate that controls learning step size. Adjustable in progress, default: 0.01.

batch: int

The mini-batch size, number of examples used to compute single gradient step, default: 10.

beta1: float

Controls the exponential decay rate for the first moment estimates, default: 0.9.

beta2: float

Controls the exponential decay rate for the second moment estimates, default: 0.999.

eps: float

The epsilon, a very small number to prevent any division by zero in the implementation, default: 1e-8.

Methods

 set_lr(value, min, max) Set value for learning rate. set_batch(value, allow_set) Set value for batch size. set_beta1(value, min, max) Set value for beta1. set_beta2(value, min, max) Set value for beta2. set_eps(value, min, max) Set value for epsilon.
class coremltools.models.neural_network.update_optimizer_utils.Batch(value, allowed_set=None)[source]

Batch optimizer.

Attributes
value: float
allowed_set: float
class coremltools.models.neural_network.update_optimizer_utils.RangeParam(value, min=0, max=1)[source]

Range Parameter optimizer.

Attributes
value: float
min: float
max: float
class coremltools.models.neural_network.update_optimizer_utils.SgdParams(lr=0.01, batch=10, momentum=0)[source]

SGD - Stochastic Gradient Descent optimizer.

Attributes
lr: float

The learning rate that controls learning step size. Adjustable in progress, default: 0.01.

batch: int

The mini-batch size, number of examples used to compute single gradient step, default: 10.

momentum: float

The momentum factor that helps accelerate gradients vectors in the right direction, default 0.

Methods

 set_lr(value, min, max) Set value for learning rate. set_batch(value, allow_set) Set value for batch size. set_momentum(value, min, max) Set value for momentum.