The Base_Node Class

The principal characteristics of the Base_Node class are:

1. It processes input and produces output.

2. It does not define how the node object processes information; the task is done in the subclasses.

3. It maintains lists of connections to the associated nodes. Two linked lists are used to store associated nodes: one maintains the nodes that bring input to the node, the other the list of nodes that receive the output of this neuron.

The Base_Link objects (explained later) do the connections among Base_Node objects.

The Base_Link objects store two pointers, one to the source and one to the destination neuron. Therefore, the Base_Node objects point only to the links. Figure 3.38 shows an example of network formed by Base_Link (rectangles) and Base_Node (circles) objects.

The creation of those links will generate an additional overhead that is softened by the flexibility of creating any architecture and the functionality capability associated to this structure once the Base_Link objects also store the connection weight and can process information.

The CANN Neuron class (see Section 3.2.2.1) does not have a list of input and output connections. It has only one list of input connections, e.g. a list of synapses that bring activation to the neuron. The neuron does not need to know what neurons are associated to its input because each synapse connected to it knows its source and destination neurons. There is no list of synapses associated to the neuron output. Once the synapse knows to which neurons it is associated, it is able to request the neuron output value whenever necessary. The list of synapses is maintained by Neuron as a Java Vector object.

Figure 3.38 – Object representation of network topology (Rogers, 1997)

The disadvantage of Rogers’ solution is the necessity of managing many pointers.

When implementing in C++ keeping those pointers consistently allocated generates a programming overhead. The programmer is responsible for allocating and deallocating those pointers being a highly error-prune solution.

Base_Node

Input_Node Feed_Forward_Node SON_Node

Adaline_Node ADALINE_Network

Base_Network

Bias_Node BP_Node

BAM_Ouput_Node BAM_Input_Node

BP_Output_Node BP_Middle_Node

BP_Network SON_Network BAM_Network

Generic_BP_Network BAM_System

Figure 3.39 – Neural Network Node hierarchy (Rogers, 1997)

The Base_Node class hierarchy is shown in Figure 3.39 and its definition is shown in Code 3.10. The Base_Node stores a set of numeric “values” that is useful to store the results of any processing done inside the neuron. This set of “values” was created to make the class generic and to make it useful in several ANN architectures. It also stores “error” results in the same way. There are getter and setter methods for “values” and “errors”. This class also stores the node name and ID. The ID is used for certain iteration controls and for the object serialization.

The Neuron class in CANN does not store a set of values but a single value, which is the neuron activation called currentActivation. The neurons also do not store name and ID because they are expensive data in terms of memory footprint. It would be waste of memory space to allocate a string to the name of each neuron in a CNM network where there could exist millions of combinatorial neurons.

The ID is not necessary once the neurons are stored in Java Vectors, which are classes that have iteration facilities via indexes. Furthermore, the Java serialization and reflection engines guarantee the unique access to the objects not being necessary to have IDs to identify them. Another important contribution of Java is the error handling mechanism that allows a seamless treatment of exceptions in classes and methods. Therefore it is not necessary to store error codes inside the classes.

Rogers’ solution stores learning parameters in the node values instead of storing them in the ANN model structure. The GetValue and SetValue methods are used to get and set the necessary learning parameters for all nodes of the model. Sometimes some learning parameters are, in fact, necessary inside the neuron to be able to do the activation calculation.

However those values should not be stored inside the neurons because it causes a waste of memory by allocating space for storing the same value as many times as node instances exist.

The natural solution would be to store those values at the ANN level and pass them by parameter to the nodes whenever necessary.

Base_Node has constructor and destructor methods for allocating and deallocating the object’s internal variables. It also provides methods for saving its state to the disk - methods save and load that operate over an iostream, - and a print method to print its state to the standard IO.

In the Java solution there is no necessity to implement the class destruction method because the destruction is provided automatically by the Java’s garbage collection engine. It is also not necessary to implement save and load methods for neurons and connections because the Java Persistence facility provides this automatically. Classes are only required to implement the Serializable interface in order to be automatically enabled to be serialized. In this case, instance variables that shall not be persistent shall be declared transient.

The most important methods defined by Base_Node are certainly Learn, Run and Epoch.

Those methods are abstract so that they are implemented only by the subclasses. The Run method defines the node operation when evaluating an input in “normal” operation, typically testing. The Learn method is used during the training process and the third method Epoch is important for unusual situations to reset values or to do special operations.

Code 3.10 - The Base_Node class (Rogers, 1997).

friend void Connect( Base_Node &from_node, Base_Node &to_node, Base_Link *link );

friend void Connect( Base_Node &from_node, Base_Node &to_node, Base_Link &link );

The CANN Neuron class has only one abstract method for both Run and Learn operations defined in Rogers’ Base_Node class because both do the same thing, e.g. calculate the neuron activation. The calculated activation is the same independently whether the ANN is in the learning or testing process. There is no similar operation like the one defined by the Epoch method.

In case of the CANN Neuron class, the process of information is not left to the subclasses. The creation of subclasses of Neuron and Synapse is frequently done to implement the specialties of each ANN model. But the implementation of the activation function is done in separate subclasses of the class ComputationStrategy, which avoids the creation of nested subclasses for the same model just implementing various activation functions. Using the ComputationStrategy solution the activation

functions could be defined even at runtime by “plugging” in a new instance of a specific ComputationStrategy. That is the use of the separation metapattern instead of the unification metapattern.

Finally there are “static” methods called Connect that implement the facility of connecting Base_Node objects through Base_Link objects. There is also a Disconnect method to undo such a connection.

The “static” Connect methods defined by Rogers’ Base_Node class are also unnecessary in the CANN Neuron definition. The connection among neurons is done using a more generic method called generateSynapses that implements the connection of the input synapses to the neuron in the appropriate way the ANN model requires. The Disconnect method implemented by Rogers was not implemented in CANN Neuron class but it is certainly a good idea to have the chance to disconnect neurons on the fly.