Micro-Modular Neural Network

Background

Artificial neural networks are an interesting field of information processing systems.

Artificial neural networks can be created as computer simulations or as real circuits.

Creating neural networks as computer simulations is relatively quick which is

advantageous because a large number of neural networks can be easily created and
tested. Unfortunately simulations of artificial neural networks suffer from two
problems: a) they are only a rough approximation of a real neural network; and b)
simulations are not scalable, i.e. as more nodes are added to the simulation, the
computing time rises rapidly.

Creating neural networks with real circuits has the advantages of being real (i.e. not
an approximation) and being scalable. The disadvantage with real neural networks is
the difficulty of in building them.

The difficulty in building real neural networks can be overcome by using

programmable circuitry (e.g. FPGA). However, programmable logic also suffers two
scalability issues. Firstly, adding more nodes to programmable logic increases the
amount of programming circuitry necessary. Secondly, programmable circuits are
limited to micro chip size.

What is needed is a method of building non-simulated artificial neural networks that

is reasonable simple and scalable.

Introduction

The basic idea is a system of building neural networks with an array of modules or
blocks. Each block contains basic circuitry which a) communicates input and output
signals to adjacent modules; and b) performs basic information processing operations
on the inputs to form the outputs.

A 2d array of modules is made by photolithography, etching and deposition on a thin

film. The thin film is broken into the individual modules. The separated modules are
then packed together into a 3d array which forms a neural network.

The modules are to be made on the micrometre scale, approximately 50 to 100µm in

diameter. A large number of modules of this size can be packed into a relatively small
space. So for example, one million 100µm diameter modules fit in 1cm3, 1 billion in a
10 cm3, etc.

Challenges

Although the idea is simple, there are many problems that need to be solved to make
it possible. I discuss some of the challenges below.
Module shape
The modules must be made in a shape that is easy to manufacture, manipulate and
stack. The modules are preferably interchangeable. This means they are all the same
size and shape. It also means that they can be used in any orientation.

The simplest form to manufacture and stack is a cubic arrangement. However,

manipulating and accurately positioning the cube modules would be difficult. What is
needed is a way of quickly stacking the modules. My idea is to use a 3D cross shape
or an extruded cross shape. The radiating arms of the cross shape help to accurately
position and stack the modules.

Module construction
I envision that the modules would be made using thin-film semiconductor techniques.
Constructing electrodes on all 6 faces of the module may be difficult. The 3D cross
shaped modules could be made in the following way:

 Make cube shape modules with soluble regions

 Vapour deposit metal over all exposed surfaces
 Dissolve soluble regions (thus ‘undermining’ the metal layer in some
areas)
 Any metal layer not attached to the remaining structure will readily
break and fall off

a) vapour metal deposited b) soluble regions dissolved

The electrode on each face would be a flat surface of metal. The modules would
connect simply be being pressed surface-to-surface.
Module alignment
The cross shaped modules would be dropped over an array of columns. The columns
act to align the crosses as they fall into place. The columns could then be slid out after
the modules are stacked; the 3D array of modules could then be compressed to ensure
good electrical contact between modules. The resulting 3D module array would be
porous which would allow gas or liquid to flow and cool the network if necessary.

Interface circuitry
The modules are to be made so they can be placed anywhere in the neural network.

Each of the 6 outermost faces of the modules would have a single electrode and use
an electrical circuit to split electrical signals into an input and an output.

input/output splitting circuit

Power source
The power to the electronic circuit in each module could be provided wirelessly. The
wireless power would be provided by a high frequency magnetic field. Each module
would have a simple resonant circuit tuned to the magnetic field. Because the tuned
circuits are resonating they would relay power from one module to the next. In one
embodiment the inductor is made of superconducting material which provides
superior magnetic field transmitting and receiving properties (but obviously there are
many difficulties in cooling a circuit to low temperatures).

wireless power receiving circuit

Module circuit
Inside each module would be a circuit which performs a function upon the inputs to
create an output(s). This circuit could be digital or analogue; so for example, a digital
module could contain an AND circuit while an analogue module could contain a
summing amplifier.

To form a neural network, there would need to be several different types of module
circuit. In a preferred embodiment there would be two types of module circuit: 1) a
‘neuron’ type and 2) a ‘dendrite’ type. The neuron type module would act to combine
several signal inputs, while the dendrite type module would act as communication
channels which would electrically connect up the neuron type modules. The dendrite
could be made of ‘memristor’ material. In this way, electrical connections between
neurons would strengthen with use (in an analogous way that a brain learns).

Schematic circuit of neuron module Schematic circuit of dendrite module

connecting the modules

Although manufacturing the modules would be difficult, the advantage of the idea is
the ease in which the modules can be connected. The modules would be dropped over
the alignment columns, and slide into a neatly stacked arrangement.

Some difficulties may arise in the process. For example, at the microscopic scale the
modules may have a tendency to electrostatically stick to the columns rather than
slide down them. This could be overcome by, for example, giving the columns the
same electrostatic charge as the modules. Cleanliness would obviously be another
problem; even small amounts of dust could clog the stack of modules.

The design of neural circuits would probably be an iterative process (such as a genetic
algorithm). Thus it would be preferable if the modules could be reused. One simple
way of deconstructing and sorting a neural network would be to make the different
module types of different density; the collection of modules would be placed in a
liquid where one module type would float while the other would sink.

Further remarks
I see this idea as a good platform for investigating neural networks. I have posted and
discussed this idea online at the following URLs:
 http://www.halfbakery.com/idea/Modular_20neural_20network#11795
79418
 http://www.halfbakery.com/idea/Notes_20on_20Modular_20Neural_2
0Networks#1209601967
 http://www.halfbakery.com/idea/Recognising_20artificial_20intelligen
ce#1212792229