xor_pytorch

A classical XOR neural network using pytorch in python3

xor.py tested an implementation and the training of
a simple neural network using pytorch.
The implemented neural network evaluates XOR for
two noisy inputs, A and B. The classical network
consists of one input and one hidden layers.

>>> net = XORNet()
>>> net
XORNet (
  (fc0): Linear (2 -> 2)
  (fc1): Linear (2 -> 1)
)

The sigmoidal activation mediates all the node inter-connections.
The simple classical two-layer network permits an output which is
basically quadratic in linear combinations of A and B. The
quaduatic properites arise from the symmetry of XOR in A and B.
However, there are two quadratic solutions corresponding to the
ambiguity of XOR for A = B = 0.5 for a simple two-layer
network.

solution 0	solution 1
XOR = 0 along A - B = 0	XOR = 1 along A + B = 1

The gif outputs are plots of the XOR(A,B)
evaluated for a mesh grid of A and B during the training.

Training

>>> xor = XOR()
>>> xor.net.load_state_dict(state_dict)
>>> # training for solution 0
>>> xor.training(nbatch=25,delta=0.2,table=XORData.TABLE0,save='t0')
>>> xor.net.load_state_dict(state_dict) # reset the start state
>>> # training for solution 1
>>> xor.training(nbatch=25,delta=0.2,table=XORData.TABLE1,save='t1')

xor.training plots and produces a png file for each batch procssed.
state_dict is a set of random weights and biases to initial the
network elements. XORData.TABLE0 and XORData.TABLE1 are
XORData.TRUTHTABLE with the additions of XOR(0.5,0.5)=0 and
XOR(0.5,0.5)=1 to prime the network to solution 0 and solution 1
respectively. The random noise in A and B is uniform in the range
[-delta,+delta].

Animations

$ convert -geometry 400x400 -delay 30 t0_*.png t0.gif
$ convert -geometry 400x400 -delay 30 t1_*.png t1.gif

produced the gif files with help of convert of
imagemagick

Analysis

xor.gp is a gnuplot script to illustrate the mathematical
solutions of the network.

Higher order solutions

Higher order networks seek to introduce higher symmetry to the
solution. A third layer would perform linear combinations of outputs
from the two-layer feeder networks below. Consider

class XORNet3(nn.Modules):
def __init__(self):
    super(XORNet, self).__init__()
    self.fc0 = nn.Linear(2,8)
    self.fc1 = nn.Linear(8,8)
    self.fc2 = nn.Linear(8,1)
def forward(self,x):
    x = F.sigmoid(self.fc0(x))
    x = F.sigmoid(self.fc1(x))
    x = F.sigmoid(self.fc2(x))
    return x

XORNet3 is a simple upgrade of XORNet of xor.py.
Training can use XORDATA.TRUTHTABLE un-modified unlike the
case of the two-layer network. The noise in A and B takes
the full span of [-0.5,0.5]. (*note* XORNet3 is
not included in the repository)

XORNet3 training