ngclearn.modules.regression package

Submodules

ngclearn.modules.regression.elastic_net module

class ngclearn.modules.regression.elastic_net.Iterative_ElasticNet(key, name, sys_dim, dict_dim, batch_size, weight_fill=0.05, lr=0.01, lmbda=0.0001, l1_ratio=0.5, optim_type='adam', threshold=0.05, epochs=100)

Bases: object

A neural circuit implementation of the iterative Elastic Net (L1 and L2) algorithm using a Hebbian learning update rule.

The circuit implements sparse regression through Hebbian synapses with Elastic Net regularization.

The specific differential equation that characterizes this model is dW_reg (for adjusting W, given dW (the gradient of loss/energy function), it adds lmbda * dW_reg to the dW)

dW_reg = (jnp.sign(W) * l1_ratio) + (W * (1-l1_ratio)/2)

dW/dt = dW + lmbda * dW_reg

— Circuit Components: —

W - HebbianSynapse for learning regularized dictionary weights

err - GaussianErrorCell for computing prediction errors

— Component Compartments —

W.inputs - input features (takes in external signals)

W.pre - pre-synaptic activity for Hebbian learning

W.post - post-synaptic error signals

W.weights - learned dictionary coefficients

err.mu - predicted outputs

err.target - target signals (target vector)

err.dmu - error gradients

err.L - loss/energy values

Parameters:

• key – JAX PRNG key for random number generation

• name – string name for this solver

• sys_dim – dimensionality of the system/target space

• dict_dim – dimensionality of the dictionary/feature space/the number of predictors

• batch_size – number of samples to process in parallel

• weight_fill – initial constant value to fill weight matrix with (Default: 0.05)

• lr – learning rate for synaptic weight updates (Default: 0.01)

• lmbda – elastic net regularization lambda parameter (Default: 0.0001)

• optim_type – optimization type for updating weights; supported values are “sgd” and “adam” (Default: “adam”)

• threshold – minimum absolute coefficient value - values below this are set to zero during thresholding (Default: 0.001)

• epochs – number of training epochs (Default: 100)

batch_set(batch_size)

clamp(y_scaled, X)

fit(y, X)

thresholding(scale=1.0)

ngclearn.modules.regression.lasso module

class ngclearn.modules.regression.lasso.Iterative_Lasso(key, name, sys_dim, dict_dim, batch_size, weight_fill=0.05, lr=0.01, lasso_lmbda=0.0001, optim_type='adam', threshold=0.001, epochs=100)

Bases: object

A neural circuit implementation of the iterative Lasso (L1) algorithm using a Hebbian learning update rule.

The circuit implements sparse coding-like regression through Hebbian synapses with L1 regularization.

The specific differential equation that characterizes this model is adding lmbda * sign(W) to the dW (the gradient of loss/energy function): | dW/dt = dW + lmbda * sign(W)

— Circuit Components: —

W - HebbianSynapse for learning sparse dictionary weights

err - GaussianErrorCell for computing prediction errors

— Component Compartments —

W.inputs - input features (takes in external signals)

W.pre - pre-synaptic activity for Hebbian learning

W.post - post-synaptic error signals

W.weights - learned dictionary coefficients

err.mu - predicted outputs

err.target - target signals (target vector)

err.dmu - error gradients

err.L - loss/energy values

Parameters:

• key – JAX PRNG key for random number generation

• name – string name for this solver

• sys_dim – dimensionality of the system/target space

• dict_dim – dimensionality of the dictionary/feature space/the number of predictors

• batch_size – number of samples to process in parallel

• weight_fill – initial constant value to fill weight matrix with (Default: 0.05)

• lr – learning rate for synaptic weight updates (Default: 0.01)

• lasso_lmbda – L1 regularization lambda parameter (Default: 0.0001)

• optim_type – optimization type for updating weights; supported values are “sgd” and “adam” (Default: “adam”)

• threshold – minimum absolute coefficient value - values below this are set to zero during thresholding (Default: 0.001)

• epochs – number of training epochs (Default: 100)

batch_set(batch_size)

clamp(y_scaled, X)

fit(y, X)

thresholding(scale=2)

ngclearn.modules.regression.ridge module

class ngclearn.modules.regression.ridge.Iterative_Ridge(key, name, sys_dim, dict_dim, batch_size, weight_fill=0.05, lr=0.01, ridge_lmbda=0.0001, optim_type='adam', threshold=0.001, epochs=100)

Bases: object

A neural circuit implementation of the iterative Ridge (L2) algorithm using a Hebbian learning update rule.

This circuit implements sparse regression through Hebbian synapses with L2 regularization.

The specific differential equation that characterizes this model is adding lmbda * W to the dW (the gradient of loss/energy function): | dW/dt = dW + lmbda * W

— Circuit Components: —

W - HebbianSynapse for learning regularized dictionary weights

err - GaussianErrorCell for computing prediction errors

— Component Compartments —

W.inputs - input features (takes in external signals)

W.pre - pre-synaptic activity for Hebbian learning

W.post - post-synaptic error signals

W.weights - learned dictionary coefficients

err.mu - predicted outputs

err.target - target signals (target vector)

err.dmu - error gradients

err.L - loss/energy values

Parameters:

• key – JAX PRNG key for random number generation

• name – string name for this solver

• sys_dim – dimensionality of the system/target space

• dict_dim – dimensionality of the dictionary/feature space/the number of predictors

• batch_size – number of samples to process in parallel

• weight_fill – initial constant value to fill weight matrix with (Default: 0.05)

• lr – learning rate for synaptic weight updates (Default: 0.01)

• ridge_lmbda – L2 regularization lambda parameter (Default: 0.0001)

• optim_type – optimization type for updating weights; supported values are “sgd” and “adam” (Default: “adam”)

• threshold – minimum absolute coefficient value - values below this are set to zero during thresholding (Default: 0.001)

• epochs – number of training epochs (Default: 100)

batch_set(batch_size)

clamp(y_scaled, X)

fit(y, X)

thresholding(scale=2)

Module contents