ngclearn.components.synapses.patched package

Submodules

ngclearn.components.synapses.patched.hebbianPatchedSynapse module

class ngclearn.components.synapses.patched.hebbianPatchedSynapse.HebbianPatchedSynapse(*args, **kwargs)

Bases: PatchedSynapse

A synaptic cable that adjusts its efficacies via a two-factor Hebbian adjustment rule.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by synapses)

weights - current value matrix of synaptic efficacies

biases - current value vector of synaptic bias values

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

pre - pre-synaptic signal to drive first term of Hebbian update (takes in external signals)

post - post-synaptic signal to drive 2nd term of Hebbian update (takes in external signals)

dWeights - current delta matrix containing changes to be applied to synaptic efficacies

dBiases - current delta vector containing changes to be applied to bias values

opt_params - locally-embedded optimizer statisticis (e.g., Adam 1st/2nd moments if adam is used)

Parameters:

• name – the string name of this cell

• shape – tuple specifying shape of this synaptic cable (usually a 2-tuple with number of inputs by number of outputs)

• n_sub_models – The number of submodels in each layer (Default: 1 similar functionality as DenseSynapse)

• stride_shape – Stride shape of overlapping synaptic weight value matrix (Default: (0, 0))

• eta – global learning rate

• weight_init – a kernel to drive initialization of this synaptic cable’s values; typically a tuple with 1st element as a string calling the name of initialization to use

• bias_init – a kernel to drive initialization of biases for this synaptic cable (Default: None, which turns off/disables biases)

• block_mask – weight mask matrix

• w_bound – maximum weight to softly bound this cable’s value matrix to; if set to 0, then no synaptic value bounding will be applied

• is_nonnegative – enforce that synaptic efficacies are always non-negative after each synaptic update (if False, no constraint will be applied)

• prior – a kernel to drive prior of this synaptic cable’s values; typically a tuple with 1st element as a string calling the name of prior to use and 2nd element as a floating point number calling the prior parameter lambda (Default: (None, 0.)) currently it supports “l1” or “lasso” or “l2” or “ridge” or “l1l2” or “elastic_net”. usage guide: prior = (‘l1’, 0.01) or prior = (‘lasso’, lmbda) prior = (‘l2’, 0.01) or prior = (‘ridge’, lmbda) prior = (‘l1l2’, (0.01, 0.01)) or prior = (‘elastic_net’, (lmbda, l1_ratio))

• sign_value – multiplicative factor to apply to final synaptic update before it is applied to synapses; this is useful if gradient descent style optimization is required (as Hebbian rules typically yield adjustments for ascent)

• optim_type –

  optimization scheme to physically alter synaptic values once an update is computed (Default: “sgd”); supported schemes include “sgd” and “adam”

  Note:

  technically, if “sgd” or “adam” is used but signVal = 1, then the ascent form of each rule is employed (signVal = -1) or a negative learning rate will mean a descent form of the optim_scheme is being employed

• pre_wght – pre-synaptic weighting factor (Default: 1.)

• post_wght – post-synaptic weighting factor (Default: 1.)

• resist_scale – a fixed scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((W * Rscale) * in) + b

• p_conn – probability of a connection existing (default: 1.); setting this to < 1. will result in a sparser synaptic structure

• batch_size – the size of each mini batch

batched_reset(batch_size)

evolve()

classmethod help()

reset()

ngclearn.components.synapses.patched.patchedSynapse module

class ngclearn.components.synapses.patched.patchedSynapse.PatchedSynapse(*args, **kwargs)

Bases: JaxComponent

A patched dense synaptic cables that creates multiple small dense synaptic cables; no form of synaptic evolution/adaptation is in-built to this component.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by synapses)

weights - current value matrix of synaptic efficacies

biases - current value vector of synaptic bias values

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

pre - pre-synaptic signal to drive first term of Hebbian update (takes in external signals)

post - post-synaptic signal to drive 2nd term of Hebbian update (takes in external signals)

dWweights - current delta matrix containing changes to be applied to synaptic efficacies

dBiases - current delta vector containing changes to be applied to bias values

opt_params - locally-embedded optimizer statisticis (e.g., Adam 1st/2nd moments if adam is used)

Parameters:

• name – the string name of this cell

• shape – tuple specifying shape of this synaptic cable (usually a 2-tuple with number of inputs by number of outputs)

• n_sub_models – The number of submodels in each layer (Default: 1 similar functionality as DenseSynapse)

• stride_shape – Stride shape of overlapping synaptic weight value matrix (Default: (0, 0))

• eta – global learning rate

• weight_init – a kernel to drive initialization of this synaptic cable’s values; typically a tuple with 1st element as a string calling the name of initialization to use

• bias_init – a kernel to drive initialization of biases for this synaptic cable (Default: None, which turns off/disables biases)

• w_masks – weight mask matrix

• pre_wght – pre-synaptic weighting factor (Default: 1.)

• post_wght – post-synaptic weighting factor (Default: 1.)

• resist_scale – a fixed scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((W * Rscale) * in) + b

• p_conn – probability of a connection existing (default: 1.); setting this to < 1. will result in a sparser synaptic structure

advance_state()

batched_reset(batch_size)

classmethod help()

reset()

ngclearn.components.synapses.patched.staticPatchedSynapse module

class ngclearn.components.synapses.patched.staticPatchedSynapse.StaticPatchedSynapse(*args, **kwargs)

Bases: PatchedSynapse

A static dense synaptic cable; no form of synaptic evolution/adaptation is in-built to this component.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output

weights - current value matrix of synaptic efficacies

Parameters:

• name – the string name of this cell

• shape – tuple specifying shape of this synaptic cable (usually a 2-tuple with number of inputs by number of outputs)

• n_sub_models – tuple specifying the number of sub models (the number of fully connected/dense synapses) or the number of patches of this synaptic cable (usually a 2-tuple with number of inputs by number of outputs)

• stride_shape – tuple specifying the stride (overlap) between sub models (patches) (Default: (0, 0))

• (Default (w_mask= Helps in creating patches) – None)

• weight_init – a kernel to drive initialization of this synaptic cable’s values; typically a tuple with 1st element as a string calling the name of initialization to use

• bias_init – a kernel to drive initialization of this synaptic bias’s values; typically a tuple with 1st element as a string calling the name of initialization to use

• resist_scale – a fixed (resistance) scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((W * Rscale) * in)

• p_conn – probability of a connection existing (default: 1.); setting this to < 1 and > 0. will result in a sparser synaptic structure (lower values yield sparse structure)

Module contents