Implementation of edge detection on FPGA
Abin Thomas Mathew Authors Name/s per 2nd Affiliation (Author)
Intelligent Transportation System
Alpen-Adria University, Klagenfurt
Austria.
athomasm@edu.uni-klu.ac.at
Abstract— Edge detection is a computer vision algorithm that
is very processor intensive. It is possible to increase the speed of
the algorithm by using hardware parallelism. This paper
presents an implementation of edge detection in an FPGA, the
Altera development kit. The paper focuses on providing the often
missing link from the algorithm development to the FPGA
implementation. The implementation is a co-operative work
between multiple software tools. The Canny filter with the
Prewitt kernel is the co-processor to Nios 2 based system for the
task mentioned above. In addition the testing software and
firmware development is described. The results show how a
highly parallel algorithm can run faster on a 50 MHz FPGA then
a modern PC in the GHz.. (Abstract)
Keywords—Edge detection, canny filter, prewitt kernel, SOPC
builder, Nios II processor, DSP builder, Altera Cyclone 2 FPGA
I. INTRODUCTION
Edge detection is a fundamental process used for image
processing application which acts as a precursor to extract
information from the frames, later used for feature extraction
and object segmentation. It is also used to remove the blur in
noisy images. Edge detection is seen in most of the application
related to image processing. The simulation of this algorithm
has been performed using most of the mathematical tools and
programmed using various languages. The hardware
implementation of the same was performed on GPP, DSP and
microcontrollers. The results from these experiments have
proven to be quite fast compared to the simulation, but it is not
fast enough.
FPGA or reprogrammable computation device is a boon to
engineers in various fields and image processing is one of
these fields of research. The work seems to be much easier
when the project is implemented on a FPGA when compared
to various contemporary computational devices. A specific
task will faster on a FPGA running on a 50MHz clock than a
PC which works on a clock in GHz [1]. Edge detection is
implemented on an FPGA with the help of Simulink1 for
modeling the Prewitt Kernel co-processor and SOPC builder2
which will create a soft-core processor.
1
Simulink is an environment for multidomain simulation
and Model-Based Design for dynamic and embedded
systems. It is a product of the The Mathwork family
2 4
SOPC Builder is a product from Altera Corp, is a pioneer in
programmable logic solutions, which is used for system
line 1 (of Affiliation): dept. name of organization
line 2—name of organization, acronyms acceptable
line 3—City, Country
line 4—e-mail address if desired
The first question which might arise is; why FPGA, when
dedicated DSPs are available to do the same work? The
answer to this question lies in the time taken to perform the
same task on a DSP and the FPGA. The first and foremost
advantage for the FPGA is that all the processes and
instructions are executed in parallel which those in DSP are
executed in sequence. Some of the processes can be assigned
to work in parallel but, not the entire stretch of code.
The main objectives of this internship are
 Learn and work with the software tools like DSP Builder3
in MATLAB, SOPC builder and Nios 2 processor in
Quartus 2 development suite and Nios 2 embedded
development suite.
 Implement the two dimensional filter with Prewitt kernel
in Simulink using DSP builder
 Create a system using NIOS II/s processor for a standard
application
 Merge the Prewitt co-processor into the system
 Execute the software codes for the system built using
Nios 2 embedded development suite
Real-time video and image processing is used in a wide
variety of applications from video surveillance and traffic
management to medical imaging applications. These
operations typically require very high computation power.
Standard definition NTSC video is digitized at 720x480 or full
D1 resolution at 30 frames per second, which results in a
31MHz pixel rate. With multiple adaptive convolution stages
to detect or eliminate different features within the image, the
filtering operation receives input data at a rate of over 1 giga
samples per second. Coupled with new high-resolution
standards and multi-channel environments, processing
requirements can be even higher. Achieving this level of
processing power using programmable DSP requires multiple
processors. A single FPGA with an embedded soft processor4
development tool used to design and generate a System On a
Programmable Chip.
3
DSP Builder, SOPC builder, quartus 2 and Nios 2
development suites are software tools from Altera
Corporation.
An embedded soft processor is a reconfigurable processor
which lies on the FPGA fabric
can deliver the requisite level of computing power more cost-
effectively, while simplifying board complexity.
Real-time image processing requires high computation
power. For example, the NTSC video standard requires
30 frames per second, with approximately .25 mega
pixels per frame, resulting in a total of 7.5 mega pixels
per second. PAL video standard has a similar
processing load with 25 frames per second, but the
frame size is larger. The amount of processing required
per pixel depends on the image processing algorithm.
In the case of the Canny edge-detection algorithm the
processing requirement is a combination of the four
stages of Gaussian smoothing for noise reduction,
finding zero crossings using the derivative of Gaussian,
non-maximal suppression to thin the edges, and finally
hysteresis thresholding.
Depending on the size of the kernels, the processing
requirements for the first two convolution stages can
change. Using the assumption of a kernel size of 7*7 for
the Gaussian and 5*5 for the derivative of the Gaussian,
the two stages require approximately 62 operations per
pixel. This implementation requires less complexity than
most typical 2-D convolution processes because of the
symmetric and separable characteristics of the Gaussian
masks. The non-maximal suppression stage requires 27
operations per pixel. Due to the recursive nature of the
hysteresis thresholding process, this stage requires an
additional 40 operations per pixel. In total, the Canny
algorithm requires approximately 130 operations per
pixel. Coupled with running at multiple resolutions, serial
implementation of the algorithm using a programmable
DSP with a clock speed of 600MHz can only process 4.6
mega pixels per second. This is not sufficient to support
real-time video streams at high resolution. As high-
resolution video standards become more prevalent, the
processing requirements will increase even further. The
high-resolution video standards typically have 4 times
more pixels per frame. The computation load, therefore
is approximately 4 times higher. These image
processing applications with high computation loads can
be implemented with multiple DSPs or a single, very
expensive high-end DSP. In this scenario, FPGAs offer
an alternative real-time image processing platform. An
FPGA efficiently supports high levels of parallel-
processing data-flow structures, which are important for
efficient implementation of image processing algorithms.
II. HARDWARE COMPONENTS
The previous section provided a brief insight on the
reasons to why an FPGA is used over a DSP for edge
detection. This chapter starts with an introduction to both the
hardware components. The discussion continues with DSP and
FPGA being widely used in most of the fields of engineering
for its ease of use and computation speed. Both these
components have their own trade off and some of them are
mentioned in the forthcoming sections.
A. Digital Signal Processor (DSP):
A digital signal processor, or DSP, is similar to a general-
purpose processor (GPP) in many aspects. It has fixed logic,
that is, the connections between logic gates cannot be
changed. It provides a fixed instruction set (ISA) to the
programmer, and it expects a program in this ISA that it will
then execute in a sequential manner (as opposed to dataflow-
driven) (Kisacanin, Bhattacharyya, & Chai, 2008). Most DSP
ISAs exhibit a similar structure as GPP ISAs, complete with
arithmetic and logic instructions, memory access, registers,
control flow, and so on. Distinguishing it from general-
purpose CPUs, a DSP‘s instruction set is optimized for matrix
operations, particularly multiplication and accumulation
(MAC), traditionally in fixed-point arithmetic, but
increasingly also for double-precision floating point
arithmetic. DSPs exhibit deep pipelining and thus expect a
very linear program flow with infrequent conditional jumps.
They provide for SIMD (single instruction, multiple data)
instructions, assuming a large amount of data that has to be
processed by the same, relatively simple, mathematical
program. SIMD programs exploit instruction-level
parallelism, executing the exact same instruction
simultaneously on multiple data. VLIW (very long instruction
word) relaxes this constraint by allowing different instructions
(op-codes) to be packed together in a VLIW, and every
instruction therein processes a different data concurrently.
Many DSPs are VLIW architectures. The types of instructions
that are allowed together within one VLIW (and thus will be
executed in parallel) depend on the function units that can
operate in parallel. For example, if a DSP has two fixed-point
MAC units and two floating point MAC units, then at most
two fixed-point MAC operations can be placed into the same
VLIW. This constraint is relaxed even further in so-called
MIMD machines (multiple instruction, multiple data), where
multiple identical processors can independently execute
arbitrary instructions on non-dependent data. You might note
that modern CPUs and their multiple-dispatch (superscalar)
pipelines do exactly that—schedule multiple instructions
concurrently. With DSPs, however, there is no such intelligent
pipeline. Instead, the burden of scheduling is on the compiler:
it has to co-schedule instructions for independent data
operations and optimize the packing of instructions in width
(e.g., four instructions per word) and in sequence (control
flow). DSPs do not perform such complex CPU operations as
branch prediction or instruction reordering. Here, too, the
compiler has to perform the optimizations. DSP programs are
relatively small programs (tens or hundreds of LOC), with few
branch and control instructions, as opposed to entire operating
systems running on general-purpose CPUs. Frequently, a
single, tight, and heavily optimized loop is executed once for
every data element or set thereof. Fig. 1.6 shows how the
serialized pixel data is streamed through a DSP with a simple
one-dimensional smoothing filter.
Since DSPs usually execute small programs on huge amounts
or endless streams of data, these two pieces of information are
stored in separate memory blocks, often accessible through
separate buses. This is called Harvard architecture, as
opposed to the GPP‘s von Neumann architecture, in which
both program and data are stored in the same memory.
Because the program does not change (firmware!), many
DSPs provide on-chip ROM (typically in the order of 10 kB)
for program storage, and a small but efficient RAM hierarchy
for data storage. Frequently, an embedded system also
includes a separate non-volatile memory chip such as an
EEPROM or flash memory. There are several high-
performance DSPs available, including several that have
multi-core DSP-with-general-purpose-CPU system on a chip
structure. Of particular interest are the DSPs with
specialization toward video and image processing, also known
as media processors. These tend to include multiple (perhaps
as many as 64) enhanced DMA units and multiple dedicated
I/O streams adapted toward the movement of pixels onto and
off the chip. Media processors are a common choice for video
applications owing to characteristics that make them equally
attractive for embedded vision systems: programmability,
direct memory access (DMA) architectures, some level of
parallelism (VLIW or SIMD), low power and low cost.
Example vision systems using DSPs are discussed in Chapters
4 and 9.Manufacturers of DSPs include Agere Systems,
Analog Devices, Infineon, Lucent Technologies, Equator
Technologies, Freescale Semiconductor (formerly part of
Motorola), NXP (formerly part of Philips Electronics), Texas
Instruments, and Zilog. Most of these manufacturers also offer
DSP development boards specific for image processing and
computer vision, complete with the required auxiliary chips,
with cameras and software. Those are an ideal starting point
for an experimental embedded vision system. Bruno Paillard
wrote a good introduction to DSPs that can be found at [12]. A
good textbook resource is Lynn and Fuerst‘s Introductory
Digital Signal Processing with Computer Applications. The
USENET group comp.dsp might also be of interest to the
reader.
B. Field Programmable Gate Array (FPGA):
A field-programmable gate array, or FPGA, is a
semiconductor in which the actual logic circuit can be
modified to the application builder‘s needs. The chip is a
relatively inexpensive, off-the-shelf device that can be
programmed in the ―field‖ and not the semiconductor
fabrication. It is important to note the difference between
software programming and logic programming, or logic
design as it is usually called: a software program always needs
to run on some microcontroller with an appropriate instruction
set architecture (ISA), whereas a logic program is the
microcontroller. In fact, this logic program can specify a
controller that accepts as input a particular ISA, for example,
the ISA of an ARM CPU, effectively turning the FPGA into
an ARM CPU.
This is a so-called soft core, built from general-purpose
logic blocks. These soft cores, or better the right to use the
intellectual property, can be purchased from companies such
as Xilinx, Inc., and Altera Corporation. They are then
―downloaded‖ to the FPGA where they implement the desired
functionality. Some of the modern FPGAs integrate platform-
or hard multipurpose processors on the logic such as a
PowerPC, ARM, or a DSP architecture. Other common hard
and soft modules include multipliers, interface logic, and
memory blocks. The logic design determines the FPGA‘s
functionality. This configuration is written to the device and is
retained until it is erased. To be precise, there are three types
of FGPAs: antifuse, SRAM, and FLASH. Antifuse chips are
not reprogrammable. FLASH (EPROM) is also nonvolatile,
meaning that the logic design stays on the chip through power
cycles. It can be erased and reprogrammed many times.
SRAM programming on the other hand is volatile; it has to be
programmed at power on. The huge benefit of an FPGA is the
great flexibility in logic, offering extreme parallelism in data
flow and processing to vision applications. One can, for
example, create 320 parallel accumulation buffers and ALUs,
summing up an entire 320×240 image in 240 clock cycles.
Another example would be to place a region of interest in the
FPGA and then perform pixel operations on the entire region
simultaneously (see Fig. 1.7). FPGAs can achieve speeds
close to DSPs and ASICs, require a bit more power than an
ASIC, have much lower non-recurring engineering (NRE)
costs, but higher volume prices than ASICs.
Algorithm developers and software engineers are usually
trained on a sequential model. That and the flexibility of logic
circuitry make parallel designs on FPGAs a challenging task,
particularly because the best implementation is often not
intuitively apparent. One of the first difficulties is dividing the
responsibilities between the FPGA and a GPP, and between
the FPGA‘s core CPU and into possible other chips on the
platform. Common ―hardware description languages‖ for logic
design are Verilog and VHDL and the topic of many
engineering courses and books. FPGAs are a great resource
for parallelism and offer tremendous flexibility in the
embedded vision processing system. On the other hand, large
FPGAs are quite power hungry and their clock rates are lower
than a typical DSPs‘ clock rate. A wide variety of field-
programmable array chips are available on the commercial
market today. The optimal choice for an embedded vision
system is a combination of a single FPGA with sufficient
general-purpose I/O resources to handle the imager‘s
incoming and outgoing interfaces plus a 64-bit interface
to/from the DSP. An equally important selection criterion will
be the amount of embedded FPGA memory as well as the
package. Most FPGAs have an abundance of I/O capability as
compared with internal logic so it is probable that an optional
32-bit SDRAM interface may also be possible. Such an
interface in an embedded vision system would provide the
FPGA with private access to its own storage area at the cost of
access time and added FPGA complexity. The plentiful I/O
resources are also used to let FPGAs control the input gates of
other onboard chips. Example vision systems using FPGAs
are discussed in Chapters 6 and 7. FPGA manufacturers
include Achronix semiconductor, Actel, Altera, AMI
Semiconductor, Atmel, Cypress Semiconductor, Lattice
Semiconductor, QuickLogic, and Xilinx. Most of these
manufacturers also offer FPGA development boards specific
for image processing and computer vision, usually with at
least one DSP onboard, and complete with cameras and
software. Those are an ideal starting point for
experimentation.
 Both these devices are good for specific functions and
hence when used practically for a particular application, we
would be able to see some trade-offs. Some of them are
mentioned below:
Fig: Illustration of edge in an image 1
 Choice between the two is very dependent on the context
of usage
 In case power is a bit of constraint, then DSP is
preferred despite all the overhead of pain in
programming
 Besides the I/O stuff that you mentioned, DSPs are not
so bad :)
 In case you do not have much of power constraints,
like you want to do an FFT on an image put on a
security camera plugged on to a socket, then FPGA by
all means is a wise choice.
 P+R on FPGA is stochastic!
 This is very bad in a real time system, where we need
to ensure that every place to try to program it comes
the same way
 Embedded Systems
 Well here, we also need that we lower power and very
low overhead for reconfiguration (in case of FPGA)
and loading a new program (in case of a DSP).
 This gives FPGA an advantage market-wise
III. EDGE DETECTION
Edge in a layman‘s term is a change of altitude or height or
an angle like the edge of a table. In terms of images the edge
will be defined in terms of intensity. A slight change in the
intensity of light from one pixel to the other will be an edge.
The intensity of light in a pixel is represented with a value in
the range 0-255, where 0 will represent ―Black‖ and 255 will
represent ―White‖. Hence edges are seen at locations where
there is an intensity contrast in an image. An edge signifies the
boundary of an object.
5 7 6 4 152 142 146 147
Fig: Illustration of edge in an image 2
The mentioned illustration shows the change of intensity from
pixel to pixel. The first row in the table gives the intensity
level at each pixel. Between the first and the second pixel
though there is a change in intensity, it is not that noticeable.
This becomes a criterion when setting the threshold for
detecting an edge. The intensity at first four pixels seems to be
quite similar due to a very minute change of intensity. We can
clearly see the change of intensity between the fourth and fifth
pixel. The values just above them show the difference too.
This contrast gives us an edge in an image.
Edge detection is the fundamental tool which is used in any
image processing applications and is used to extract
information from the gray level image and the information is
later used for feature extraction or object segmentation
processes. The edge detection algorithms were classically
implemented on software. The advancements in the VLSI
technology have made it an alternative.
The Canny edge detector is the most standard detector that is
used for edge detection because it provides sharp and thin
edges. The Canny edge detector is a multiple step detector and
the steps are
Filtering
Differentiation
Detection
A. Filter
The filtering stage is used to remove the noise in the
image. The noise is generated due to the undesirable effects of
sampling, quantization, blurring and defocusing of the camera
and the irregularities of the surface of the object. A linear filter
is used to remove the noise with the help of a Gaussian kernel.
The filter operation averages the noise in the neighborhood
and this operation is a convolution of the input image and the
filter. The equations representing the filter operation is given
below
In the equations mentioned above ‗h’ represents the
filtered image, ‗f‘ the input image and ‗g‘ the filter. The *
represent the convolution operation which is seen in the last
equation.
B. Differentiation
The general equation for differentiation is
The images are discrete and hence the minimum value of
∇x is 1 and hence the derivate becomes
The task that I had been working on involves gradient
operators for the above mentioned task. The differentiation
process can also be worked out using a mask, similar to the
one used for filtering. Some of the gradient operators are
Roberts, Prewitt, Sobel and these operators have been
extensively been used in the past. Roberts and Prewitt
operators have a equal weights which are assigned to the
pixels whereas Sobel uses different weights as shown below.
Prewitt Operator
Sobel Operator
Robert's Operator
As mentioned in the introductory chapter, the Prewitt
operator is used for the Co-Processor. Though Prewitt is not
the best operator that is available to solve the problem, we still
use it during this stage of the work due to its simplicity for the
work. More of the weights are equal and unity which makes it
easier to compute the values of the pixels. In the next section,
we will see how the Prewitt kernel for the canny filter is
implemented using Simulink with DSP builder toolbox.
IV. IMPLEMENTATION OF THE ALGORITHM
The implementation of the edge detection algorithm using
Prewitt Kernel involves both software and hardware tasks.
These tasks are further divided into three phases namely
The Prewitt Kernel Co-processor
The system built using NIOS II5 to work synchronous
with the co-processor
System model execution codes written in C for both the
hardware and the software
The edge detection design will finally will be in the format
as shown below. Except for the co-processor design all the
components that are included in the design will be part of
NIOS system and this will work along with the co-processor
to perform the task of edge detection. The SOPC builder will
be the tool which will be used to create the system, which is a
tool in Quartus II for rapid prototyping
Block diagram of edge detection system design
The input image will be a 640 x 480 image which will be
fed into the system through a compact flash, the data is passed
to the buffer which translated the pixels into arrays and fed to
the co-processor for processing and then again transmitted
back to the buffer which will be stored in the SRAM. The
edge detected image that is stored in the SRAM is then pushed
to the video lines for the display. Each of these components
will be explained in detail in the sections below.
A. Prewitt kernel co-processor
The Prewitt kernel as mentioned in the chapter on Edge
detection, is Implemented in Simulink with the help of DSP
builder6 toolbox. Since this is the prototype that is being
developed, the measures taken to clear the noise of the image
is removed and hence we will start with the matrix gradient
kernel for canny detector.
The kernel is a 3 X 3 matrix and this is the mask that will
be put on the image to detect an edge in the pixels. The
hardware implementation for doing this is shown in fig ()
which uses the line buffers to create a delay in the lines for the
incoming pixels. The array lines are fed with data
simultaneously with a delay and pixel values are multiplied
with the respective co-efficient at the node as seen the figure.
5
NIOS II is a softcore embedded processor from Altera, which
can be used to emulate a processor for a logical system based
on the requirements for an application.
6
DSP builder is a product from the Altera Corporation which
is used as a toolbox with Matlab for modeling systems meant
to be implemented on Altera FPGA development kits.
All the video lines are then fed to an adder which will produce
the middle point resultant pixel.
Implementation of 2D non symmetrical filter
This filter is then used for the implementation of the
vertical and horizontal filters. The filter designs for the two
are the same except for the data that flows into the array. The
block diagram in fig () shows how the vertical and horizontal
filters are merged together to obtain the combined threshold
values and this will produce the binary image with the edge
detection.
Block diagram for the implementation of canny filter
The interface between the compact flash and the co-
processor takes places through the Avalon7 interface
component which will be used in Simulink as a black box. The
Avalon interface component can be either added using the
DSP Builder toolbox or the HDL code can be imbibed using
the black box toolbox in Simulink. The model that is built in
Simulink is shown in fig ()
Prewitt Kernel Co-processor
The Prewitt block is the small block that is seen in the
figure above. It is a block which is used for modeling the
co-processor. The Prewitt edge detection model is as
shown in the figure below.
Prewitt edge detection
The input and output that is seen in this model are
reflected back to edge in and edge out scripts that run in
7
Avalon interface is a standard interface seen in SOPC builder
which is used as interfaces for high-speed streaming and
memory mapped applications.
MATLAB. The edge in script is used to convert a real image Exception controller
into an image in gray scale and convert the two dimensional
Interrupt controller
image into a linear array of data. The vice versa happens in the
edge out script. These two files were used during the Instruction bus
simulation stage. This helped to check if the algorithm was Data bus
working. But in the co-processor model, the data in is from the
compact flash which is buffered at the DDR2 SDRAM and Memory management unit (MMU)
similarly at the output it is again buffered and stored in the Memory protection unit (MPU)
SRAM. The data from the SRAM is sent out to the VGA
video channels. Instruction and data cache memories
Tightly-coupled memory interfaces for instructions and
B. Building the system data
Once the co-processor is built, it is the turn of the system JTAG debug module
to be built. The system is built with NIOS 2 a RISC core
processor. We used the economy8 version of the processor to
work as the CPU for the system. From the block diagram as
seen in fig (), we see that the DMA helps the system in
transferring the data from the flash memory and also store the
data to the SRAM. This is done to reduce the burden on the
processor itself, the definition of DMA. The step wise process
on how to build and configure a system is as shown below.
1) System
Any system will have the following basic components
 A microprocessor for a processor based system (NIOS II)
Memory to store data or instructions (SRAM or
SDRAM)
JTAG to transfer data from the host machine into the
target device (Development workstation to DE2
Development board)
Other devices based on the application (VGA)

a)NIOS 2 Processor: Block diagram of NIOS 2 processor core

Nios II processor is a 32 bit soft core processor provided
by the Altera. The Nios II processor has Reduced Instruction
Set Computer (RISC) architecture and the architecture uses
separate instruction and data buses, which is often referred to
as the Harvard architecture. The Nios II architecture supports
a flat register file, consisting of thirty two 32-bit general-
purpose integer registers and six 32-bit control registers. Nios
II processor cores can address a 31-bit address span. The user
must assign base address between 0x00000000 and
0x7FFFFFFF.
The figure below shows the block diagram of the NIOS II
processor and some of the functional unit seen in the block
diagram are
Register file
Arithmetic logic unit (ALU)
Interface to custom instruction logic
8 A SOPC file is built in a Quartus 2 project file. This opens
NIOS II processor comes in three different formats NIOS II/e
(economy), NIOS II/f (fast), NIOS II/s (standard)
Each of the functional units mentioned above can be used
based on the objective to be achieved. The NIOS II processor
has a custom set of instructions which can be used for specific
implementations and also supports user defined instruction
which will connect the NIOS II ALU to the user defined logic
to perform the implementation.
C. Instantiating a NIOS II processor and building a system
in SOPC builder
System On a Programmable Chip is the current technique
used for prototyping applications for every field of
engineering. This technique is faster, cheaper and is the most
efficient form of testing a system during the development
phase. This development tool is used to specify, configure and
generate the embedded system using graphical user interface.
The software development tasks can be accomplished using
the Eclipse based development suite called NIOS II embedded
development suite or NIOS II studio.
a window with all the components which can be included into
a system. The Nios 2 processor being the heart of the system
is considered the CPU for the system. Representative names
can be provided to the system based on the application that is
being developed.
Nios 2 processors have three possible standards based on
the kind of system it is used in and they are
NIOS II/s (standard)
NIOS II/e (economy)
NIOS II/f (fast)
Nios II/f Fast CPU (Adler) – Optimized for maximum
performance, the Nios II/f processor delivers up to 220
DMIPS performance in the Stratix II family of high-
performance FPGAs, placing it squarely in the ARM 9 class
of processor. While this core is 4 times faster than the original
Nios CPU, it is 40% smaller as well.
Nios II/e Economy CPU – Optimized for lowest cost, the
/e core achieves a smaller FPGA footprint (less than 600 LEs),
consuming as little as $.35 worth of logic in a Cyclone II
device. It is half the size of the smallest Nios core and 4 times
the performance.
Nios II/s Standard CPU – This core balances processing
performance and logic element usage. It is 0% faster than the
fastest Nios CPU and smaller than the smallest Nios CPU, and
it achieves over 120 DMIPS while consuming only 930 LEs
(Stratix II).
A detailed table with the possible features and the
difference among the processors can be seen in the appendix
[]. For the task that we are working on, it was suitable for us to
choose the standard processor. Once the main task of choosing
the data is done, it is now the turn to choose the memory. The
possible memories that are available are the on-chip memory
(RAM/ROM) (variable), SRAM (512KB) and an SDRAM (8
MB). The on-chip memory as mentioned in the chapter on
Hardware components are the flip flops which can be used as
memory units. These memory blocks are also called L2K
memory block. When defining these blocks, the size and
length of the data that can be stored into it can also be set,
which make it a flexible memory. The other memories are the
SRAM and SDRAM memory which are fixed to the memory
size, but can vary the length of the data that we store into it.
We will use the IP library files to evoke the memory.
The next component in list for the building is the system
timer or the clock to synchronize the movement of data. We
will use the component called ―internal timer‖ for this
purpose. Each of the components which are used for the
system can be renamed based on the specific function it
performs in a system. There is another clock which is used in
the system. This is the clock from an external source. The
board that we use has 50 MHz and 27 MHz oscillators which
can be used to generate the clocks. The other possibility is
generating a PLL and clock based on the user needs can be
generated with this.
The final component which will connect the host pc to the
target board has to be activated too. It is a serial
communication port using a JTAG. This component can be
seen under the Interfaces menu. The other components which
are the peripherals can called either by using PIO components
or if the university program DE2 board is installed, then we
have build in library for all the peripherals on the board. This
will include the drivers for VGA, LED, seven segment display
and Ethernet port.
Direct Memory Access (DMA) is used to give complete
control of the data bus to the DMA unit for bulk data transfer
from or to the memory. DMA can be activated by calling the
required functions or the component from the library.
D. Co-processor in the system
The prewitt edge detection design as shown in the section
above, is converted into a HDL code using the signal
converter which is a component of DSP builder. This tool will
convert the model in Simulink into a HDL for the target
device specified. The HDL code that is generated is called into
the system using SOPC builder as a component. Parameters in
general are not available for this component, but it depends on
the designer.
E. Nios 2 embedded development suite
The embedded development suite also called the Nios 2
Studio is used to write software codes for the hardware that is
generated. The development suite is built on eclipse and hence
a couple of parameters has to be set before we start working
with this tool. Some of the features that can be seen with Nios
2 EDS are
 2x faster compile/build times as compared to Nios II
IDE
 Simple file system based project management
o Supports relative paths for easy copy, move
of sources
o Version control compatible
o Easy to read/manage makefiles
 GUI based generation of Board Support Packages
(BSP) featuring
o A menu of HAL-based operating systems
o Software packages (network stack, files
systems, graphics libraries etc.)
o Support for custom device drivers
o Support for multiple versions of a device
driver
o Full control over build options and OS
settings
o Linker memory map management
  Full compatibility with command line tools for
scripting and automated regression testing
The .sopc or .ptf file that is generated from the SOPC
builder can be used here as the CPU. An alternative to this is
to call the Nios 2 EDS from the SOPC builder, this will use
the current sopc file as the cpu and the program that has to be
run on this cpu can be run. As mentioned above the script can
be run based on our requirement. It is also possible to test the
code by simulating it on the cpu on the host PC. This enables
the designer to debug the code even without porting it onto the
FPGA.
For programmers who prefer to use command prompt for
testing the code can use the Nios command shell on command
prompt and build and execute codes from the command
prompt. For beginners, it is advisable to use the GUI as it quite
user friendly and since it is opensource, the possibility of
obtaining help and solutions for problems regarding the
configuration of the software is better.
V. RESULTS
The simulation of the edge detect on Simulink provided
with good results before the code were used in the system. The
results show that, the edges are detected with the codes that
were created. For the edge detection, any real image is
converted into gray scale before it is used and is also brought
to the resolution 600*480 before fed to the co-processor for
further processing. This system was meant to be a prototype
and hence not much importance was given to the accuracy and
the optimization methods. When we use the image, we do care
about the noise in the image and hence the results were not
that good when compared to the software simulation results.
Due to lots of undetermined reasons, the hardware test
failed. A couple of problems that we faced during the course
of our work were solved but a few problems did not get apt
solutions to give it a final result. Some of the problems that we
faced are:
 The HDL codes that were converted from Simulink
with the help of DSP builder could not be analyzed
and hence the code had to be used with the system
just as it was.
 Intellectual Property (IP) library functions for all the
components were not available and hence some of
the components could not be used to build the
system.
 One of the major reasons was the lack of subscription
licenses for the software tools that we were working
on, which hindered the work to a large extent.
 Nios 2 Embedded development suite, working on
eclipse had issues of configuration, which seemed
very odd. Every CPU that was generated with the
custom co-processor could not be processed and
errors due to this made the system fail for
implementation.
 Lack of updated documentation for the work flow of
the software tools was indeed a drawback of its kind.
ACKNOWLEDGMENT (HEADING 5)
I would like to thank Univ.-Prof. Dr.-Ing. Kyandoghere
Kyamakya for furnishing us with all the resources that we
needed for the research task. I was very happy to be part of the
team working with a big goal in mind. I would like to thank
Mr. Koteswara Rao Anne for constant technical and moral
help throughout the various phases of my work. A great
amount of the stress passes away when we have a support of
this sort. I would like to thank Mr. Singanamallu Gupta and
Mrs. Deepthi Rao for their support and help during the course
of my work. I am extremely happy to be part of the Intelligent
Transportation system research group. My colleagues Mr.
Sreeram Kumar Bhagavatula, Mr. Vamsi Prakash Makkapati
and Mr. Venkata Sai Dattu Potapragada were a great support
and it was great fun to work along with, their innovative talks
and work fashion especially during late nights provided me
the energy to work a lot better. Last but not the least all my
family whose constant prayers and wishes helped me thorough
out my work.
REFERENCES
[1] Mubbarak Shah, Fundamentals of Computer vision, University of
Central Florida, 1997
[2] Mahmoodi A, Kraut J, Jahns M and Mahmood S, ―Hardware edge
detection using an Altera Stratix Nios 2 development kit‖, Digital ASIC
design, December 2005
[3] Forsyth, D., & Ponce, J. (2002). Computer Vision: A Modern Approach.
Alexandria, VA: Prentice Hall.
[4] Shapiro, L., Shapiro, L., Stockman, G., & Stockman, G.
(2001). Computer Vision. Alexandria, VA: Prentice Hall
[5] Altera Corporation, SOPC Builder datasheet, September 2002.
[6] M. Young, The Technical Writer’s Handbook. Mill Valley, CA:
University Science, 1989.