ngclearn.utils package

Submodules

ngclearn.utils.config module

class ngclearn.utils.config.Config(fname=None)[source]

Bases: object

Simple configuration object to house named arguments for experiments (to be built from a .cfg file on disk).

File format is:
# Comments start with pound symbol
arg_name = arg_value
arg_name = arg_value # side comment that will be stripped off
Parameters

fname – source file name to build configuration object from (suffix = .cfg)

getArg(arg_name)[source]

Retrieve argument from current configuration

Parameters

arg_name – the string name of the argument to retrieve from this config

Returns

the value of the named argument queried

hasArg(arg_name)[source]

Check if argument exists (or if it is known by this config object)

Parameters

arg_name – the string name of the argument to check for the existence of

Returns

True if this config contains this argument, False otherwise

setArg(arg_name, arg_value)[source]

Sets an argument directly

Parameters

  • arg_name – the string name of the argument to set within this config

  • arg_value – the value name of the argument to set within this config

ngclearn.utils.data_utils module

Data functions and utilies.

class ngclearn.utils.data_utils.DataLoader(design_matrices, batch_size, disable_shuffle=False, ensure_equal_batches=True)[source]

Bases: object

A data loader object, meant to allow sampling w/o replacement of one or more named design matrices. Note that this object is iterable (and implements an __iter__() method).

Parameters

  • design_matrices – list of named data design matrices - [(“name”, matrix), …]

  • batch_size – number of samples to place inside a mini-batch

  • disable_shuffle – if True, turns off sample shuffling (thus no sampling w/o replacement)

  • ensure_equal_batches – if True, ensures sampled batches are equal in size (Default = True). Note that this means the very last batch, if it’s not the same size as the rest, will reuse random samples from previously seen batches (yielding a batch with a mix of vectors sampled with and without replacement).

ngclearn.utils.data_utils.binarized_shuffled_omniglot(out_dir)[source]

Specialized function for the omniglot dataset.

Note: this function has not been tested/fully integrated yet

ngclearn.utils.data_utils.generate_patch_set(imgs, patch_shape, batch_size, center_patch=True)[source]

Generates a set of patches from an array/list of image arrays (via random sampling with replacement).

Parameters

  • imgs – the array of image arrays to sample from

  • patch_shape – a 2-tuple of the form (pH = patch height, pW = patch width)

  • batch_size – how many patches to extract/generate from source images

  • center_patch – centers each patch by subtracting the patch mean (per-patch)

Returns

an array (D x (pH * pW)), where each row is a flattened patch sample

ngclearn.utils.io_utils module

Utility I/O functions.

ngclearn.utils.io_utils.deserialize(fname)[source]

De-serializes a object from disk

Parameters

fname – filename of object to load - /path/to/fname_of_model

Returns

the deserialized model object

ngclearn.utils.io_utils.parse_simulation_info(sim_info)[source]

Parses a simulation information dictionary into a human-readable string.

Parameters

sim_info – simulation info dictionary

Returns

a string presenting the simulation information

ngclearn.utils.io_utils.plot_img_grid(samples, fname, nx, ny, px, py, plt, rotNeg90=False)[source]
ngclearn.utils.io_utils.plot_sample_img(x_s, px, py, fname, plt, rotNeg90=False)[source]

Plots a (1 x (px * py)) array as a (px x py) gray-scale image and saves this image to disk.

Parameters

  • x_s – the numpy image array

  • px – number of pixels in the row dimension

  • py – number of pixels in the column dimension

  • fname – the filename of the image to save to disk

  • plt – a matplotlib plotter object

  • rotNeg90 – rotates the image -90 degrees before saving to disk

ngclearn.utils.io_utils.serialize(fname, object)[source]

Serializes an object to disk.

Parameters

  • fname – filename of object to save - /path/to/fname_of_model

  • model – model object to serialize

ngclearn.utils.metric_utils module

General mathematical measurement/metric functions/utilities file.

ngclearn.utils.metric_utils.bce(p, x, offset=1e-07)[source]

Calculates the negative Bernoulli log likelihood or binary cross entropy (BCE).

Parameters

  • p – predicted probabilities of shape (N x D)

  • x – target binary values (data) of shape (N x D)

Returns

an (N x 1) column vector, where each row is the BCE(p, x) for that row’s datapoint

ngclearn.utils.metric_utils.calc_ACC(T)[source]

Calculates the average accuracy (ACC) given a task matrix T.

Parameters

T – task matrix (containing accuracy values)

Returns

scalar ACC for T

ngclearn.utils.metric_utils.calc_BWT(T)[source]

Calculates the backward(s) transfer (BWT) given a task matrix T

Parameters

T – task matrix (containing accuracy values)

Returns

scalar BWT for T

ngclearn.utils.metric_utils.cat_nll(p, x, epsilon=1e-07)[source]

Measures the negative Categorical log likelihood

Parameters

  • p – predicted probabilities

  • x – true one-hot encoded targets

Returns

an (N x 1) column vector, where each row is the Cat.NLL(x_pred, x_true) for that row’s datapoint

ngclearn.utils.metric_utils.fast_log_loss(probs, y_ind)[source]

Calculates negative Categorical log likelihood / cross entropy via a fast indexing approach (assumes targets/labels are integers or class indices for single-class one-hot encoding).

Parameters

  • probs – predicted label probability distributions (one row per label)

  • y_ind – label indices - can be either (D,) vector or (Dx1) column vector

Returns

the scalar value of Cat.NLL(x_pred, x_true)

ngclearn.utils.metric_utils.mse(mu, x)[source]

Measures mean squared error (MSE), or the negative Gaussian log likelihood with variance of 1.0.

Parameters

  • mu – predicted values (mean)

  • x – target values (x/data)

Returns

an (N x 1) column vector, where each row is the MSE(x_pred, x_true) for that row’s datapoint

ngclearn.utils.stat_utils module

Statistical functions/utilities file.

ngclearn.utils.stat_utils.ainv(A)[source]

Computes the inverse of matrix A

Parameters

A – matrix to invert

Returns

the inversion of A

ngclearn.utils.stat_utils.calc_covariance(X, mu_=None, weights=None, bias=True)[source]

Calculate the covariance matrix of X

Parameters

  • X – an (N x D) data design matrix to measure log density over (1 row vector - 1 data point)

  • mu – a pre-computed (1 x D) vector mean of the Gaussian distribution (Default = None)

  • weights – a (N x 1) weighting column vector, one row is weight applied to one sample in X (Default = None)

  • bias – (only applies if weights is None), if True, compute the biased estimator of covariance

Returns

a (D x D) covariance matrix

ngclearn.utils.stat_utils.calc_gKL(mu_p, sigma_p, mu_q, sigma_q)[source]

Calculate the Gaussian Kullback-Leibler (KL) divergence between two multivariate Gaussian distributions, i.e., KL(p||q).

Parameters

  • mu_p – (1 x D) vector mean of distribution p

  • sigma_p – (D x D) covariance matrix of distributon p

  • mu_q – (1 x D) vector mean of distribution q

  • sigma_q – (D x D) covariance matrix of distributon q

Returns

the scalar KL divergence

ngclearn.utils.stat_utils.calc_list_moments(data_list, num_dec=3)[source]

Compute the mean and standard deviation from a list of data values. This is for simple scalar measurements/metrics that will be printed to I/O.

Parameters

  • data_list – list of data values, each element should be (1 x 1)

  • num_dec – number of decimal points to round values to (Default = 3)

Returns

(mu, sigma), where mu = mean and sigma = standard deviation

ngclearn.utils.stat_utils.calc_log_gauss_pdf(X, mu, cov)[source]

Calculates the log Gaussian probability density function (PDF)

Parameters

  • X – an (N x D) data design matrix to measure log density over

  • mu – the (1 x D) vector mean of the Gaussian distribution

  • cov – the (D x D) covariance matrix of the Gaussian distribution

Returns

a (N x 1) column vector w/ each row containing log density value per sample

ngclearn.utils.stat_utils.convert_to_spikes(x_data, gain=1.0, offset=0.0, n_trials=1)[source]
ngclearn.utils.stat_utils.sample_bernoulli(p)[source]

Samples a multivariate Bernoulli distribution

Parameters

p – probabilities to samples of shape (n_s x D)

Returns

an (n_s x D) (binary) matrix of Bernoulli samples (one vector sample per row)

ngclearn.utils.stat_utils.sample_gaussian(n_s, mu=0.0, sig=1.0, n_dim=- 1)[source]

Samples a multivariate Gaussian assuming a diagonal covariance or scalar variance (shared across dimensions) in the form of a standard deviation vector/scalar.

Parameters

  • n_s – number of samples to draw

  • mu – (1 x D) mean of the Gaussian distribution

  • sig – (1 x D) or (1 x 1) standard deviation of the Gaussian distribution

  • n_dim – dimensionality of the sample space

Returns

an (n_s x n_dim) matrix of uniform samples (one vector sample per row)

ngclearn.utils.stat_utils.sample_uniform(n_s, n_dim)[source]

Samples a multivariate Uniform distribution

Parameters

  • n_s – number of samples to draw

  • n_dim – dimensionality of the sample space

Returns

an (n_s x n_dim) matrix of uniform samples (one vector sample per row)

ngclearn.utils.transform_utils module

A mathematical transformation utilities function file. This file contains activation functions and other relevant data transformation tools/utilities.

ngclearn.utils.transform_utils.bernoulli(x)[source]
ngclearn.utils.transform_utils.binarize(data, threshold=0.5)[source]

Converts the vector data to its binary equivalent

Parameters

  • data – the data to binarize (real-valued)

  • threshold – the cut-off point for 0, i.e., if threshold = 0.5, then any number/value inside of data < 0.5 is set to 0, otherwise, it is set to 1.0

Returns

the binarized equivalent of “data”

ngclearn.utils.transform_utils.binary_flip(x_b)[source]

Flips the bit values within binary vector x_b

Parameters

x_b – the binary vector to flip

Returns

the flipped binary vector form of x_b

ngclearn.utils.transform_utils.bkwta(x, K=10)[source]

Binarized k-winners-take-all competitive activation function

ngclearn.utils.transform_utils.calc_modulatory_factor(W)[source]

Calculate modulatory matrix W_M for W

Note: this is NOT fully tested/integrated yet

ngclearn.utils.transform_utils.calc_zca_whitening_matrix(X)[source]

Calculates a ZCA whitening matrix via the Mahalanobis whitening method.

Note: this is NOT fully tested/integrated yet

Parameters

X – a design matrix of shape (M x N), where rows -> features, columns -> observations

Returns

the resultant (M x M) ZCA matrix

ngclearn.utils.transform_utils.clip_fx(x)[source]
ngclearn.utils.transform_utils.convert_to_spikes_(x_data, max_spike_rate, dt, sp_div=4.0)[source]

Converts a vector x_data to its approximate Poisson spike equivalent.

Note: this function is NOT fully tested/integrated yet.

Parameters

  • max_spike_rate – firing rate (in Hertz)

  • dt – integraton time constant (in milliseconds or ms)

  • sp_div – to denominator to convert input data values to a firing frequency

Returns

the binary spike vector form of x_data

ngclearn.utils.transform_utils.create_block_bin_matrix(shape, n_ones_per_row)[source]
ngclearn.utils.transform_utils.create_competiion_matrix(z_dim, lat_type, beta_scale, alpha_scale, n_group, band)[source]

This function creates a particular matrix to simulate competition via self-excitatory and inhibitory synaptic signals.

Parameters

  • z_dim – dimensionality of neural group to apply competition to

  • lat_type – type of competiton pattern. “lkwta” sets a column/group based form of k-WTA style competition and “band” sets a matrix band-based form of competition.

  • beta_scale – the strength of the cross-unit inhibiton

  • alpha_scale – the strength of the self-excitation

  • n_group –

    if lat_type is set to “lkwta”, then this ensures that only a certain number of neurons are within a competitive group/column

    Note

    z_dim should be divisible by n_group

  • band – the band parameter (Note: not fully tested)

Returns

a (z_dim x z_dim) competition matrix

ngclearn.utils.transform_utils.create_mask_matrix(n_col_m, nrow, ncol)[source]
ngclearn.utils.transform_utils.d_elu(z, alpha=1.0)[source]
ngclearn.utils.transform_utils.d_identity(x)[source]
ngclearn.utils.transform_utils.d_ltanh(z)[source]
ngclearn.utils.transform_utils.d_relu(x)[source]
ngclearn.utils.transform_utils.d_relu6(x)[source]
ngclearn.utils.transform_utils.d_sigmoid(x)[source]
ngclearn.utils.transform_utils.d_softplus(x)[source]
ngclearn.utils.transform_utils.d_tanh(x)[source]
ngclearn.utils.transform_utils.decide_fun(fun_type)[source]

A selector function that generates a physical activation function and its first-order (element-wise) derivative funciton given a description fun_type. Note that some functions do not come with a proper derivative (and thus set to the identity function derivative – see list below).

Currently supported functions (for given fun_type) include:
* “tanh” - hyperbolic tangent
* “ltanh” - LeCun-style hyperbolic tangent
* “sigmoid” - logistic link function
* “kwta” - K-winners-take-all
* “softmax” - the softmax function (derivative not generated)
* “identity” - the identity function
* “relu” - rectified linear unit
* “lrelu” - leaky rectified linear unit
* “softplus” - the softplus function
* “relu6” - the relu but upper bounded/capped at 6.0
* “elu” - exponential linear unit
* “erf” - the error function (derivative not generated)
* “binary_flip” - bit-flipping function (derivative not generated)
* “bkwta” - binary K-winners-take-all (derivative not generated)
* “sign” - signum (derivative not generated)
* “clip_fx” - clipping function (derivative not generated)
* “heaviside” - Heaviside function (derivative not generated)
* “bernoulli” - the Bernoulli sampling function (derivative not generated)
Parameters

fun_type – a string stating the name of activation function and its 1st elementwise derivative to generate

Returns

(fx, d_fx), where fx is the physical activation function and d_fx its derivative

ngclearn.utils.transform_utils.drop_out(input, rate=0.0, seed=69)[source]

Custom drop-out function – returns output as well as binary mask

ngclearn.utils.transform_utils.elu(z, alpha=1.0)[source]
ngclearn.utils.transform_utils.erf(x)[source]
ngclearn.utils.transform_utils.filter(x_t, x_f, dt, a, filter_type='var_trace')[source]

Applies a filter to data x_t.

Note: this function is NOT fully tested/integrated yet.

Parameters

  • x_t –

  • x_f –

  • dt –

  • a –

  • filter_type – (Default = “var_trace”)

Returns

the filtered vector form of x_t

ngclearn.utils.transform_utils.global_contrast_normalization(Xin, s, lmda, epsilon)[source]
ngclearn.utils.transform_utils.gte(x, val=0.0)[source]
ngclearn.utils.transform_utils.identity(z)[source]
ngclearn.utils.transform_utils.init_weights(kernel, shape, seed)[source]

Randomly generates/initializes a matrix/vector according to a kernel pattern.

Currently supported/tested patterns include:
* “he_uniform”
* “he_normal”
* “classic_glorot”
* “glorot_normal”
* “glorot_uniform”
* “orthogonal”
* “truncated_gaussian” (alternative: “truncated_normal”)
* “gaussian” (alternative: “normal”)
* “uniform”
Parameters

  • kernel – a tuple denoting the pattern by which a matrix is initialized Note that the first item of kernel MUST contain a string specifying the initlialization pattern/scheme to use. Other elements, for tuples of length > 1 can contain pattern-specific hyper-paramters.

  • shape – a 2-tuple specifying (N x M), a matrix of N rows by M columns

  • seed – value to control determinism in initializer

Returns

an (N x M) matrix randomly initialized to a chosen scheme

ngclearn.utils.transform_utils.inverse_logistic(x, clip_bound=0.03)[source]

Inverse logistic link - logit function

ngclearn.utils.transform_utils.inverse_tanh(x)[source]

Inverse hyperbolic tangent

ngclearn.utils.transform_utils.kwta(x, K=50)[source]

k-winners-take-all competitive activation function

ngclearn.utils.transform_utils.lrelu(x)[source]
ngclearn.utils.transform_utils.ltanh(z)[source]
ngclearn.utils.transform_utils.mellowmax(x, omega=1.0, axis=1)[source]
ngclearn.utils.transform_utils.mish(x)[source]
ngclearn.utils.transform_utils.normalize_by_norm(param, proj_mag=1.0, param_axis=0)[source]
ngclearn.utils.transform_utils.normalize_image(image)[source]

Maps image array first to [0, image.max() - image.min()] then to [0, 1]

Arg:

image: the image numpy.ndarray

Returns

image array mapped to [0, 1]

ngclearn.utils.transform_utils.relu(x)[source]
ngclearn.utils.transform_utils.relu6(x)[source]
ngclearn.utils.transform_utils.scale_feat(x, a=- 1.0, b=1.0)[source]

Applies the min-max feature scaling function to input x.

Parameters

  • a – the lower bound to scale x w/in

  • b – the upper bound to scale x w/in

Returns

the scaled version of x, w/ each value in range [a,b]

ngclearn.utils.transform_utils.sech(x)[source]
ngclearn.utils.transform_utils.sech_sqr(x)[source]
ngclearn.utils.transform_utils.shrink(a, b)[source]
ngclearn.utils.transform_utils.sigmoid(x)[source]
ngclearn.utils.transform_utils.sign(x)[source]
ngclearn.utils.transform_utils.softmax(x, tau=0.0)[source]

Softmax function with overflow control built in directly. Contains optional temperature parameter to control sharpness (tau > 1 softens probs, < 1 sharpens –> 0 yields point-mass)

Parameters

  • x – a (N x D) input argument (pre-activity) to the softmax operator

  • tau – probability sharpening/softening factor

Returns

a (N x D) probability distribution output block

ngclearn.utils.transform_utils.softplus(x)[source]
ngclearn.utils.transform_utils.tanh(x)[source]
ngclearn.utils.transform_utils.threshold_cauchy(x, lmda)[source]
ngclearn.utils.transform_utils.threshold_soft(x, lmda)[source]
ngclearn.utils.transform_utils.to_one_hot(idx, depth)[source]

Converts an integer or integer array into a binary one-hot encoding.

Parameters

  • idx – an integer or integer list representing the index/indices of the chosen category/categories

  • depth – total number of actual categories (the dimension K of the encoding)

Returns

a binary one-of-K encoding of the input idx (an N x K vector if len(idx) = N)

ngclearn.utils.transform_utils.whiten(X)[source]

Whitens image X via ZCA whitening

Note: this is NOT fully tested/integrated yet

Module contents