GNCN-t1-SC (Olshausen & Field, 1996)¶
This circuit implements the sparse coding model proposed in (Olshausen & Field, 1996) [1].
Specifically, this model is unsupervised and can be used to process sensory
pattern (row) vector(s) x
to infer internal latent states. This class offers,
beyond settling and update routines, a projection function by which ancestral
sampling may be carried out given the underlying directed generative model
formed by this NGC system.
The GNCN-t1-SC is graphically depicted by the following graph:
- class ngclearn.museum.gncn_t1_sc.GNCN_t1_SC(args)[source]
Structure for constructing the sparse coding model proposed in:
Olshausen, B., Field, D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).
Note this model imposes a factorial (Cauchy) prior to induce sparsity in the latent activities z1 (the latent codebook). Synapses initialized from a (fan-in) scaled uniform distribution. This model would be named, under the NGC computational framework naming convention (Ororbia & Kifer 2022), as the GNCN-t1/SC (SC = sparse coding) or GNCN-t1/Olshausen.
Node Name Structure:p(z1) ; z1 -(z1-mu0)-> mu0 ;e0; z0Cauchy prior applied for p(z1)Note: You can also recover the model learned through ISTA by using, instead of a factorial prior over latents, a thresholding function such as the “soft_threshold”. (Make sure you set “prior” to “none” in this case.) This results in the GNCN-t1/SC emulating a system similar to that proposed in:
Daubechies, Ingrid, Michel Defrise, and Christine De Mol. “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.” Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences 57.11 (2004): 1413-1457.
- Parameters
args – a Config dictionary containing necessary meta-parameters for the GNCN-t1/SC
DEFINITION NOTE:args should contain values for the following:* batch_size - the fixed batch-size to be fed into this model* z_dim - # of latent variables in layers z1* x_dim - # of latent variables in layer z0 or sensory x* seed - number to control determinism of weight initialization* beta - latent state update factor* leak - strength of the leak variable in the latent states (Default = 0)* prior - type of prior to use (Default = “cauchy”)* lmbda - strength of the prior applied over latent state activities (only if prior != “none”)* threshold - type of threshold to use (Default = “none”)* thr_lmbda - strength of the threshold applied over latent state activities (only if threshold != “none”)* n_group - must be > 0 if lat_type != None and s.t. (z_dim mod n_group) == 0* K - # of steps to take when conducting iterative inference/settling* act_fx - activation function for layers z1 (Default = identity)* out_fx - activation function for layer mu0 (prediction of z0) (Default: identity)- project(z_sample)[source]
Run projection scheme to get a sample of the underlying directed generative model given the clamped variable z_sample
- Parameters
z_sample – the input noise sample to project through the NGC graph
- Returns
x_sample (sample(s) of the underlying generative model)
- settle(x, K=- 1, cold_start=True, calc_update=True)[source]
Run an iterative settling process to find latent states given clamped input and output variables
- Parameters
x – sensory input to reconstruct/predict
K – number of steps to run iterative settling for
cold_start – start settling process states from zero (Leave this to True)
calc_update – if True, computes synaptic updates @ end of settling process (Default = True)
- Returns
x_hat (predicted x)
- calc_updates(avg_update=True)[source]
Calculate adjustments to parameters under this given model and its current internal state values
- Returns
delta, a list of synaptic matrix updates (that follow order of .theta)
- clear()[source]
Clears the states/values of the stateful nodes in this NGC system
References:
[1] Olshausen, B., Field, D. Emergence of simple-cell receptive field properties
by learning a sparse code for natural images. Nature 381, 607–609 (1996).