ngclearn.components.synapses.convolution package

Submodules

ngclearn.components.synapses.convolution.convSynapse module

class ngclearn.components.synapses.convolution.convSynapse.ConvSynapse(*args, **kwargs)

Bases: JaxComponent

A base convolutional synaptic cable.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals

filters - current value tensor of filter/kernel efficacies

biases - current base-rate/bias efficacies

Parameters:

• name – the string name of this cell

• x_shape – 2d shape of input map signal (component currently assumess a square input maps)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• bias_init – kernel to drive initialization of bias/base-rate values (Default: None, which turns off/disables biases)

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((K @ in) * resist_scale) + b where @ denotes convolution

• batch_size – batch size dimension of this component

advance_state()

classmethod help()

reset()

ngclearn.components.synapses.convolution.deconvSynapse module

class ngclearn.components.synapses.convolution.deconvSynapse.DeconvSynapse(*args, **kwargs)

Bases: JaxComponent

A base deconvolutional (transposed convolutional) synaptic cable.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals

filters - current value tensor of filter/kernel efficacies

biases - current base-rate/bias efficacies

Parameters:

• name – the string name of this cell

• x_shape – 2d shape of input map signal (component currently assumess a square input maps)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• bias_init – kernel to drive initialization of bias/base-rate values (Default: None, which turns off/disables biases)

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((W @.T Rscale) * in) + b where @.T denotes deconvolution

• batch_size – batch size dimension of this component

advance_state()

classmethod help()

reset()

ngclearn.components.synapses.convolution.hebbianConvSynapse module

class ngclearn.components.synapses.convolution.hebbianConvSynapse.HebbianConvSynapse(*args, **kwargs)

Bases: ConvSynapse

A specialized synaptic convolutional cable that adjusts its efficacies via a two-factor Hebbian adjustment rule.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by filters)

filters - current value matrix of synaptic filter efficacies

biases - current value vector of synaptic bias values

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

pre - pre-synaptic signal to drive first term of Hebbian update (takes in external signals)

post - post-synaptic signal to drive 2nd term of Hebbian update (takes in external signals)

dWeights - delta tensor containing changes to be applied to synaptic filter efficacies

dBiases - delta tensor containing changes to be applied to bias values

dInputs - delta tensor containing back-transmitted signal values (“backpropagating pulse”)

opt_params - locally-embedded optimizer statisticis (e.g., Adam 1st/2nd moments if adam is used)

Parameters:

• name – the string name of this cell

• x_shape – 2d shape of input map signal (component currently assumess a square input maps)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• eta – global learning rate (default: 0)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• bias_init – kernel to drive initialization of bias/base-rate values (Default: None, which turns off/disables biases)

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((K @ in) * resist_scale) + b where @ denotes convolution

• w_bound – maximum weight to softly bound this cable’s value matrix to; if set to 0, then no synaptic value bounding will be applied

• is_nonnegative – enforce that synaptic efficacies are always non-negative after each synaptic update (if False, no constraint will be applied)

• w_decay – degree to which (L2) synaptic weight decay is applied to the computed Hebbian adjustment (Default: 0); note that decay is not applied to any configured biases

• sign_value – multiplicative factor to apply to final synaptic update before it is applied to synapses; this is useful if gradient descent style optimization is required (as Hebbian rules typically yield adjustments for ascent)

• optim_type –

  optimization scheme to physically alter synaptic values once an update is computed (Default: “sgd”); supported schemes include “sgd” and “adam”

  Note:

  technically, if “sgd” or “adam” is used but signVal = 1, then the ascent form of each rule is employed (signVal = -1) or a negative learning rate will mean a descent form of the optim_scheme is being employed

• batch_size – batch size dimension of this component

backtransmit()

evolve()

classmethod help()

reset()

ngclearn.components.synapses.convolution.hebbianDeconvSynapse module

class ngclearn.components.synapses.convolution.hebbianDeconvSynapse.HebbianDeconvSynapse(*args, **kwargs)

Bases: DeconvSynapse

A specialized synaptic deconvolutional (transposed convolutional) cable that adjusts its efficacies via a two-factor Hebbian adjustment rule.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by filters)

filters - current value matrix of synaptic filter efficacies

biases - current value vector of synaptic bias values

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

pre - pre-synaptic signal to drive first term of Hebbian update (takes in external signals)

post - post-synaptic signal to drive 2nd term of Hebbian update (takes in external signals)

dWeights - delta tensor containing changes to be applied to synaptic filter efficacies

dBiases - delta tensor containing changes to be applied to bias values

dInputs - delta tensor containing back-transmitted signal values (“backpropagating pulse”)

opt_params - locally-embedded optimizer statisticis (e.g., Adam 1st/2nd moments if adam is used)

Parameters:

• name – the string name of this cell

• x_shape – dimension of input signal (assuming a square input)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• eta – global learning rate (default: 0)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• bias_init – kernel to drive initialization of bias/base-rate values (Default: None, which turns off/disables biases)

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale –

  a fixed (resistance) scaling factor to apply to synaptic

  transform (Default: 1.), i.e., yields: out = ((W @.T Rscale) * in) + b where @.T denotes deconvolution

  w_bound: maximum weight to softly bound this cable’s value matrix to; if

  set to 0, then no synaptic value bounding will be applied

• is_nonnegative – enforce that synaptic efficacies are always non-negative after each synaptic update (if False, no constraint will be applied)

• w_decay – degree to which (L2) synaptic weight decay is applied to the computed Hebbian adjustment (Default: 0); note that decay is not applied to any configured biases

• sign_value – multiplicative factor to apply to final synaptic update before it is applied to synapses; this is useful if gradient descent style optimization is required (as Hebbian rules typically yield adjustments for ascent)

• optim_type –

  optimization scheme to physically alter synaptic values once an update is computed (Default: “sgd”); supported schemes include “sgd” and “adam”

  Note:

  technically, if “sgd” or “adam” is used but signVal = 1, then the ascent form of each rule is employed (signVal = -1) or a negative learning rate will mean a descent form of the optim_scheme is being employed

• batch_size – batch size dimension of this component

backtransmit()

evolve()

classmethod help()

reset()

ngclearn.components.synapses.convolution.ngcconv module

Calculation toolbox that drives conv/deconv operations in the ngc-learn convolution components sub-branch; this contains routines/co-routines for ngclearn.components.synapses.convolution.

ngclearn.components.synapses.convolution.ngcconv.calc_dK_conv(x, d_out, delta_shape, stride_size=1, padding=((0, 0), (0, 0)))

Calculate the gradient with respect to the kernel for a convolution operation.

Parameters:

• x (ndarray) – The input array.

• d_out (ndarray) – The gradient with respect to the output.

• delta_shape (tuple) – The shape difference (deX, deY) between the input and output.

• stride_size (int) – The stride size for the convolution. Defaults to 1.

• padding (tuple) – Padding to apply to the input. Defaults to ((0, 0), (0, 0)).

Returns:

The gradient with respect to the kernel.

Return type:

ndarray

ngclearn.components.synapses.convolution.ngcconv.calc_dK_deconv(x, d_out, delta_shape, stride_size=1, out_size=2, padding='SAME')

Calculate the gradient with respect to the kernel for a deconvolution operation.

Parameters:

• x (ndarray) – The input array.

• d_out (ndarray) – The gradient with respect to the output.

• delta_shape (tuple) – The shape difference (deX, deY) between the input and output.

• stride_size (int) – The stride size for the deconvolution. Defaults to 1.

• out_size (int) – The output size for the deconvolution.

• padding (str) – Padding to apply to the input. Defaults to “SAME”.

Returns:

The gradient with respect to the kernel.

Return type:

ndarray

ngclearn.components.synapses.convolution.ngcconv.calc_dX_conv(K, d_out, delta_shape, stride_size=1, anti_padding=None)

ngclearn.components.synapses.convolution.ngcconv.calc_dX_deconv(K, d_out, delta_shape, stride_size=1, padding=((0, 0), (0, 0)))

Wrapper function to calculate the gradient with respect to the input (dX) from the gradient with respect to the output (d_out) using the kernel (K). This version takes into account the shape difference (delta_shape) between the input and the output of the convolution.

Parameters:

• K (ndarray) – The convolution kernel.

• d_out (ndarray) – The gradient with respect to the output of the convolution.

• delta_shape (tuple) – The shape difference (deX, deY) between the input and output.

• stride_size (int) – The stride size for the deconvolution. Defaults to 1.

• padding (tuple) – Padding to apply to the input. Defaults to ((0, 0), (0, 0)).

Returns:

The gradient with respect to the input.

Return type:

dx (ndarray)

ngclearn.components.synapses.convolution.ngcconv.conv2d(inputs, filters, stride_size=1, rhs_dilation=(1, 1), lhs_dilation=(1, 1), padding=((0, 0), (0, 0)))

ngclearn.components.synapses.convolution.ngcconv.deconv2d(inputs, filters, stride_size=1, rhs_dilation=(1, 1), padding=((0, 0), (0, 0)))

ngclearn.components.synapses.convolution.ngcconv.get_same_conv_padding(lhs, rhs, stride_size=1, rhs_dilation=(1, 1), lhs_dilation=(1, 1))

ngclearn.components.synapses.convolution.ngcconv.get_valid_conv_padding(lhs, rhs, stride_size=1, rhs_dilation=(1, 1), lhs_dilation=(1, 1))

ngclearn.components.synapses.convolution.ngcconv.rot180(tensor)

Rotate the input tensor by 180 degrees.

Parameters:

tensor (ndarray) – The input tensor to be rotated.

Returns:

The tensor rotated by 180 degrees.

Return type:

ndarray

ngclearn.components.synapses.convolution.staticConvSynapse module

class ngclearn.components.synapses.convolution.staticConvSynapse.StaticConvSynapse(*args, **kwargs)

Bases: ConvSynapse

A static convolutional synaptic cable; no form of synaptic evolution/adaptation is in-built to this component.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals

weights - current value tensor of kernel efficacies

biases - current base-rate/bias efficacies

Parameters:

• name – the string name of this cell

• x_shape – 2d shape of input map signal (component currently assumess a square input maps)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• bias_init – kernel to drive initialization of bias/base-rate values (Default: None, which turns off/disables biases)

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((K @ in) * resist_scale) + b where @ denotes convolution

• batch_size – batch size dimension of this component

ngclearn.components.synapses.convolution.staticDeconvSynapse module

class ngclearn.components.synapses.convolution.staticDeconvSynapse.StaticDeconvSynapse(*args, **kwargs)

Bases: DeconvSynapse

A static deconvolutional (transposed convolutional) synaptic cable; no form of synaptic evolution/adaptation is in-built to this component.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals

weights - current value tensor of kernel efficacies

biases - current base-rate/bias efficacies

Parameters:

• name – the string name of this cell

• x_shape – dimension of input signal (assuming a square input)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• bias_init – kernel to drive initialization of bias/base-rate values (Default: None, which turns off/disables biases)

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((K @ in) * resist_scale) + b where @ denotes convolution

• batch_size – batch size dimension of this component

ngclearn.components.synapses.convolution.traceSTDPConvSynapse module

class ngclearn.components.synapses.convolution.traceSTDPConvSynapse.TraceSTDPConvSynapse(*args, **kwargs)

Bases: ConvSynapse

A specialized synaptic convolutional cable that adjusts its filter efficacies via a trace-based form of spike-timing-dependent plasticity (STDP).

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by filters)

filters - current value matrix of synaptic filter efficacies

biases - current value vector of synaptic bias values

eta - learning rate global scale

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

preSpike - pre-synaptic spike to drive 1st term of STDP update (takes in external signals)

postSpike - post-synaptic spike to drive 2nd term of STDP update (takes in external signals)

preTrace - pre-synaptic trace value to drive 1st term of STDP update (takes in external signals)

postTrace - post-synaptic trace value to drive 2nd term of STDP update (takes in external signals)

dWeights - delta tensor containing changes to be applied to synaptic filter efficacies

dInputs - delta tensor containing back-transmitted signal values (“backpropagating pulse”)

Parameters:

• name – the string name of this cell

• x_shape – 2d shape of input map signal (component currently assumess a square input maps)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• A_plus – strength of long-term potentiation (LTP)

• A_minus – strength of long-term depression (LTD)

• eta – global learning rate (default: 0)

• pretrace_target – controls degree of pre-synaptic disconnect, i.e., amount of decay (higher -> lower synaptic values)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((K @ in) * resist_scale) + b where @ denotes convolution

• w_bound – maximum weight to softly bound this cable’s value matrix to; if set to 0, then no synaptic value bounding will be applied

• w_decay – degree to which (L2) synaptic weight decay is applied to the computed STDP adjustment (Default: 0)

• batch_size – batch size dimension of this component

backtransmit()

evolve()

classmethod help()

reset()

ngclearn.components.synapses.convolution.traceSTDPDeconvSynapse module

class ngclearn.components.synapses.convolution.traceSTDPDeconvSynapse.TraceSTDPDeconvSynapse(*args, **kwargs)

Bases: DeconvSynapse

A specialized synaptic deconvolutional (transposed convolutional) cable that adjusts its filter efficacies via a trace-based form of spike-timing-dependent plasticity (STDP).

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by filters)

filters - current value matrix of synaptic filter efficacies

biases - current value vector of synaptic bias values

eta - learning rate global scale

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

preSpike - pre-synaptic spike to drive 1st term of STDP update (takes in external signals)

postSpike - post-synaptic spike to drive 2nd term of STDP update (takes in external signals)

preTrace - pre-synaptic trace value to drive 1st term of STDP update (takes in external signals)

postTrace - post-synaptic trace value to drive 2nd term of STDP update (takes in external signals)

dWeights - delta tensor containing changes to be applied to synaptic filter efficacies

dInputs - delta tensor containing back-transmitted signal values (“backpropagating pulse”)

Parameters:

• name – the string name of this cell

• x_shape – dimension of input signal (assuming a square input)

• shape – tuple specifying shape of this synaptic cable (usually a 4-tuple with number filter height x filter width x input channels x number output channels); note that currently filters/kernels are assumed to be square (kernel.width = kernel.height)

• A_plus – strength of long-term potentiation (LTP)

• A_minus – strength of long-term depression (LTD)

• eta – global learning rate (default: 0)

• pretrace_target – controls degree of pre-synaptic disconnect, i.e., amount of decay (higher -> lower synaptic values)

• filter_init – a kernel to drive initialization of this synaptic cable’s filter values

• stride – length/size of stride

• padding – pre-operator padding to use – “VALID” (none), “SAME”

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((K @ in) * resist_scale) + b where @ denotes convolution

• w_bound – maximum weight to softly bound this cable’s value matrix to; if set to 0, then no synaptic value bounding will be applied

• w_decay – degree to which (L2) synaptic weight decay is applied to the computed STDP adjustment (Default: 0)

• batch_size – batch size dimension of this component

backtransmit()

evolve()

classmethod help()

reset()

Module contents