ngclearn.components.synapses.modulated package

Submodules

ngclearn.components.synapses.modulated.MSTDPETSynapse module

class ngclearn.components.synapses.modulated.MSTDPETSynapse.MSTDPETSynapse(*args, **kwargs)

Bases: TraceSTDPSynapse

A synaptic cable that adjusts its efficacies via trace-based form of three-factor learning, i.e., modulated spike-timing-dependent plasticity (M-STDP) or modulated STDP with eligibility traces (M-STDP-ET).

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (transformation induced by synapses)

weights - current value matrix of synaptic efficacies

modulator - external modulatory signal values (e.g., a reward value)

key - JAX PRNG key

— Synaptic Plasticity Compartments: —

preSpike - pre-synaptic spike to drive 1st term of STDP update (takes in external signals)

postSpike - post-synaptic spike to drive 2nd term of STDP update (takes in external signals)

preTrace - pre-synaptic trace value to drive 1st term of STDP update (takes in external signals)

postTrace - post-synaptic trace value to drive 2nd term of STDP update (takes in external signals)

dWeights - current delta matrix containing (MS-STDP/MS-STDP-ET) changes to be applied to synaptic efficacies

eligibility - current state of eligibility trace

eta - global learning rate (applied to change in weights for final MS-STDP/MS-STDP-ET adjustment)

References:

Florian, Răzvan V. “Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity.”

Neural computation 19.6 (2007): 1468-1502.

Morrison, Abigail, Ad Aertsen, and Markus Diesmann. “Spike-timing-dependent

plasticity in balanced random networks.” Neural computation 19.6 (2007): 1437-1467.

Bi, Guo-qiang, and Mu-ming Poo. “Synaptic modification by correlated

activity: Hebb’s postulate revisited.” Annual review of neuroscience 24.1

(2001): 139-166.

Parameters:

• name – the string name of this cell

• shape – tuple specifying shape of this synaptic cable (usually a 2-tuple with number of inputs by number of outputs)

• A_plus – strength of long-term potentiation (LTP)

• A_minus – strength of long-term depression (LTD)

• eta – global learning rate initial value/condition (default: 1)

• mu – controls the power scale of the Hebbian shift

• pretrace_target – controls degree of pre-synaptic disconnect, i.e., amount of decay (higher -> lower synaptic values)

• tau_elg – eligibility trace time constant (default: 0); must be >0, otherwise, the trace is disabled and this synapse evolves via M-STDP

• elg_decay – eligibility decay constant (default: 1)

• tau_w – amount of synaptic decay to augment each MSTDP/MSTDP-ET update with

• weight_init – a kernel to drive initialization of this synaptic cable’s values; typically a tuple with 1st element as a string calling the name of initialization to use

• resist_scale – a fixed scaling factor to apply to synaptic transform (Default: 1.), i.e., yields: out = ((W * Rscale) * in)

• p_conn – probability of a connection existing (default: 1.); setting this to < 1. will result in a sparser synaptic structure

• w_bound – maximum value/magnitude any synaptic efficacy can be (default: 1)

evolve(dt, t)

classmethod help()

reset()

ngclearn.components.synapses.modulated.REINFORCESynapse module

class ngclearn.components.synapses.modulated.REINFORCESynapse.REINFORCESynapse(*args, **kwargs)

Bases: DenseSynapse

A stochastic synapse implementing the REINFORCE algorithm (policy gradient method). This synapse uses Gaussian distributions for generating actions and performs gradient-based updates.

— Synapse Compartments: —

inputs - input (takes in external signals)

outputs - output signals (sampled actions from Gaussian distribution)

weights - current value matrix of synaptic efficacies (contains both mean and log-std parameters)

dWeights - current delta matrix containing changes to be applied to synaptic efficacies

rewards - reward signals used to modulate weight updates (takes in external signals)

objective - scalar value of the current loss/objective

accumulated_gradients - exponential moving average of gradients for tracking learning progress

step_count - counter for number of learning steps

learning_mask - binary mask determining when learning occurs

seed - JAX PRNG key for random sampling

Parameters:

• name – the string name of this component

• shape – tuple specifying shape of this synaptic cable (usually a 2-tuple with number of inputs by number of outputs)

• eta – learning rate for weight updates (Default: 1e-4)

• decay – decay factor for computing exponential moving average of gradients (Default: 0.99)

• weight_init – a kernel to drive initialization of this synaptic cable’s values; typically a tuple with 1st element as a string calling the name of initialization to use

• resist_scale – a fixed scaling factor to apply to synaptic transform (Default: 1.)

• act_fx – activation function to apply to inputs (Default: “identity”)

• p_conn – probability of a connection existing (default: 1.); setting this to < 1. will result in a sparser synaptic structure

• w_bound – upper bound for weight clipping (Default: 1.)

• batch_size – batch size dimension of this component (Default: 1)

• seed – random seed for reproducibility (Default: 42)

• mu_act_fx – activation function to apply to the mean of the Gaussian distribution (Default: “identity”)

evolve(dt)

classmethod help()

reset()

Module contents