Cable

A Cable represents one of the fundamental building blocks of an NGC system. These particular objects are meant to serve as the connectors between Node(s), passing along or transforming signals from the source point (a compartment, or receiving area, within a particular node) to a destination point (another compartment in a different node) and transforming such signals through synaptic parameters.

Cable Model

The Cable class serves as a root class for the wire building block objects of an NGC system/graph. This is a core modeling component of general NGC computational systems. Cable sub-classes within ngc-learn inherit from this base class.

class ngclearn.engine.cables.cable.Cable(cable_type, inp, out, name=None, seed=69)

Base cable element (class from which other cable types inherit basic properties from)

Parameters

• cable_type – the string concretely denoting this cable’s type

• inp –

  2-Tuple defining the nodal points that this cable will connect. The value types inside each slot of the tuple are specified below:

  input_node (Tuple[0])

  the source/input Node object that this cable will carry signal information from

  input_compartment (Tuple[1])

  the compartment within the source/input Node that signals will extracted and transmitted from

• out –

  2-Tuple defining the nodal points that this cable will connect. The value types inside each slot of the tuple are specified below:

  input_node (Tuple[0])

  the destination/output Node object that this cable will carry signal information to

  input_compartment (Tuple[1])

  the compartment within the destination/output Node that signals transmitted and deposited into

• name – the string name of this cable (Default = None which creates an auto-name)

• seed – integer seed to control determinism of any underlying synapses associated with this cable

propagate()

Internal transmission function that computes the correct transformation of a source node to a destination node

Returns

the resultant transformed signal (transformation f information from “node”)

set_update_rule(preact=None, postact=None, update_rule=None, gamma=1.0, use_mod_factor=False, param=None, decay_kernel=None)

Sets the synaptic adjustment rule for this cable (currently a 2-factor local synaptic Hebbian update rule).

Parameters

• preact –

  2-Tuple defining the pre-activity/source node of which the first factor the synaptic update rule will be extracted from. The value types inside each slot of the tuple are specified below:

  preact_node (Tuple[0])

  the physical node that offers a pre-activity signal for the first factor of the synaptic/cable update

  preact_compartment (Tuple[1])

  the component in the preact_node to extract the necessary signal to compute the first factor the synaptic/cable update

• postact –

  2-Tuple defining the post-activity/source node of which the second factor the synaptic update rule will be extracted from. The value types inside each slot of the tuple are specified below:

  postact_node (Tuple[0])

  the physical node that offers a post-activity signal for the second factor of the synaptic/cable update

  postact_compartment (Tuple[1])

  the component in the postact_node to extract the necessary signal to compute the second factor the synaptic/cable update

• update_rule – a specific update rule to use with the parameters of this cable

• gamma – scaling factor for the synaptic update

• use_mod_factor –

  if True, triggers the modulatory matrix weighting factor to be applied to the resultant synaptic update

  Note

  This is un-tested/not fully integrated

• param – a list of strings, each containing named parameters that are to be learned w/in this cable

• decay_kernel –

  2-Tuple defining the type of weight decay to be applied to the synapses. The value types inside each slot of the tuple are specified below:

  decay_type (Tuple[0])

  string indicating which type of weight decay to use, “l2” will trigger L2-penalty decay, while “l1” will trigger L1-penalty decay

  decay_coefficient (Tuple[1])

  scalar/float to control magnitude of decay applied to computed local updates

calc_update()

Calculates the updates to the internal synapses that compose this cable given this cable’s pre-configured synaptic update rule.

Parameters

clip_kernel – radius of Gaussian ball to constrain computed update matrices by (i.e., clipping by Frobenius norm)

DCable Model

The DCable class extends from the base Cable class and represents a dense transform of signals from one nodal point to another. Signals that travel across it through a set of synaptic parameters (and potentially a base-rate/bias shift parameter). In conjunction with the corresponding SCable class, this serves as the core modeling component of a higher-level NGCGraph class used in simulation.

class ngclearn.engine.cables.dcable.DCable(inp, out, init_kernels=None, shared_param_path=None, clip_kernel=None, constraint_kernel=None, seed=69, name=None)

A dense cable that transforms signals that travel across via a bundle of synapses. (In other words, a linear projection followed by an optional base-rate/bias shift.)

Note: a dense cable only contains two possible learnable parameters, “A” and “b” each with only two terms for their local Hebbian updates.

Parameters

• inp –

  2-Tuple defining the nodal points that this cable will connect. The value types inside each slot of the tuple are specified below:

  input_node (Tuple[0])

  the source/input Node object that this cable will carry signal information from

  input_compartment (Tuple[1])

  the compartment within the source/input Node that signals will extracted and transmitted from

• out –

  2-Tuple defining the nodal points that this cable will connect. The value types inside each slot of the tuple are specified below:

  input_node (Tuple[0])

  the destination/output Node object that this cable will carry signal information to

  input_compartment (Tuple[1])

  the compartment within the destination/output Node that signals transmitted and deposited into

• w_kernel –

  an N-Tuple defining type of scheme to randomly initialize weights.

  scheme (Tuple[0])

  triggers the type of initalization scheme, for example, “gaussian” will apply an elementwise Gaussian initialization. (See the documentation for init_weights() in ngclearn.utils.transform_utils for details on all the types of initializations and their string codes that can be used.)

  scheme_arg1 (Tuple[1])

  first argument to control the initialization (for many schemes, setting this value to 1.0 or even omitting it is acceptable given that this parameter is ignored, for example, in “unif_scale”, the second argument would be ignored.) (See the documentation for init_weights() in ngclearn.utils.transform_utils for details on all the types of initializations and their extra arguments.)

  scheme_arg2 (Tuple[2])

  second argument to control the initialization – this is generally only necessary to set in the case of lateral competition initialization schemes, such as in the case of “lkwta” which requires a 3-Tuple specified as follows: (“lkwta”,alpha_scale,beta_scale) where alpha_scale controls the strength of self-excitation and beta_scale controls the strength of the cross-unit inhibition.

• b_kernel –

  2-Tuple defining type of scheme to randomly initialize weights.

  scheme (Tuple[0])

  triggers the type of initalization scheme, for example, “gaussian” will apply an elementwise Gaussian initialization. (See the documentation for init_weights() in ngclearn.utils.transform_utils for details on all the types of initializations and their string codes that can be used.)

  scheme_arg1 (Tuple[1])

  first argument to control the initialization (for many schemes, setting this value to 1.0 or even omitting it is acceptable given that this parameter is ignored, for example, in “unif_scale”, the second argument would be ignored.) (See the documentation for init_weights() in ngclearn.utils.transform_utils for details on all the types of initializations and their extra arguments.)

• shared_param_path –

• clip_kernel –

  3-Tuple defining type of clipping to apply to calculated synaptic adjustments.

  clip_type (Tuple[0])

  type of clipping constraint to apply. If “hard_clip” is set, then a hard-clipping routine is applied (ignoring “clip_axis”) while “norm_clip” clips by checking if the norm exceeds “clip_value” along “clip_axis”. Note that “hard_clip” will also be applied to biases (while “clip_norm” is not).

  clip_value (Tuple[1])

  the magnitude of the worse-case bounds of the clip to apply/enforce.

  clip_axis (Tuple[2])

  the axis along which the clipping is to be applied (to each matrix).

  Note

  specifying None will mean no clipping is applied to this cable’s calculated updates

• constraint_kernel –

  Dict defining the constraint type to be applied to the learnable parameters of this cable. The expected keys and corresponding value types are specified below:

  ’clip_type’

  type of clipping constraint to be applied to learnable parameters/synapses. If “norm_clip” is specified, then norm-clipping will be applied (with a check if the norm exceeds “clip_mag”), and if “forced_norm_clip” then norm-clipping will be applied regardless each time apply_constraint() is called.

  ’clip_mag’

  the magnitude of the worse-case bounds of the clip to apply/enforce.

  ’clip_axis’

  the axis along which the clipping is to be applied (to each matrix).

  Note

  specifying None will mean no constraints are applied to this cable’s parameters

• name – the string name of this cable (Default = None which creates an auto-name)

• seed – integer seed to control determinism of any underlying synapses associated with this cable

propagate()

Internal transmission function that computes the correct transformation of a source node to a destination node

Returns

the resultant transformed signal (transformation f information from “node”)

calc_update()

Calculates the updates to the internal synapses that compose this cable given this cable’s pre-configured synaptic update rule.

Parameters

clip_kernel – radius of Gaussian ball to constrain computed update matrices by (i.e., clipping by Frobenius norm)

SCable Model

The SCable class extends from the base Cable class and represents a simple carry-over of signals from one nodal point to another. Signals that travel across it can either be carried directly (an identity transform) or multiplied by a scalar amplification coefficient. In conjunction with the corresponding DCable class, this serves as the core modeling component of a higher-level NGCGraph class used in simulation.

class ngclearn.engine.cables.scable.SCable(inp, out, coeff=1.0, name=None, seed=69)

A simple cable that, at most, applies a scalar amplification of signals that travel across it. (Otherwise, this cable works like an identity carry-over.)

Parameters

• inp –

  2-Tuple defining the nodal points that this cable will connect. The value types inside each slot of the tuple are specified below:

  input_node (Tuple[0])

  the source/input Node object that this cable will carry signal information from

  input_compartment (Tuple[1])

  the compartment within the source/input Node that signals will extracted and transmitted from

• out –

  2-Tuple defining the nodal points that this cable will connect. The value types inside each slot of the tuple are specified below:

  input_node (Tuple[0])

  the destination/output Node object that this cable will carry signal information to

  input_compartment (Tuple[1])

  the compartment within the destination/output Node that signals transmitted and deposited into

• coeff – a scalar float to control any signal scaling associated with this cable

• name – the string name of this cable (Default = None which creates an auto-name)

• seed – integer seed to control determinism of any underlying synapses associated with this cable

propagate()

Internal transmission function that computes the correct transformation of a source node to a destination node

Returns

the resultant transformed signal (transformation f information from “node”)