Real Time Image Warper using Digilent Spartan 3 FPGA

By Shane Agnew Supervisor: Dr. Fearghal Morgan Final Year Project

Oct 1, 2012

Contents
Initial aims / Specifications Introduction Camera Bitmaps VHDL Graphical User Interface Future Work Summary
Oct 1, 2012 2

Initial aims / Specifications

Generate image data Incorporate a communication protocol into the design Perform all required file formatting Implement digital signal processing on-board the FPGA Develop a real-time multi-functional video processing application

Oct 1, 2012

Introduction
8 - b it B M P

ra w 2b m p

R A W O u tp u t F ile

S p a rt a n 3 S ta r te r B o a r d

file R e a d

S p a rt a n 3 F P G A

U S B 2 .0 P e r ip h e r a B l C o m m u n ic a t io n s 1 M o d u le

D P IM R E F

3FFFFh E m p ty

file W r ite

3000h
D a t C t r lr IO C S R B lk

A p p lie d V H D L U S B S im p le V B G U I

R A W In p u t F ile

2000h

D S P B lk

M e m C t r lr

1000h

C a m e ra

2 4 - b it B M P

b m p C o n v e rt e r 2 4 to 8 B it

8 - b it B M P

b m p 2 ra w

0000h SRAM 0 SRAM 1

S R A M M e m o ry C o n te n t

Oct 1, 2012

Camera
Trust 120 Spacecam
Standard web camera 24-bit BMP Output Software provided

24-bit BMPs
Simple to use Uncompressed data Most web-cameras support them 1 byte for each of red, green and blue, hence RGB value

Oct 1, 2012

Bitmaps
8-bit BMPs
Limited SRAM memory available 3 times less communication overhead 3 times less file access Less awkward to process and store in SRAM Single byte represents a single pixel

Oct 1, 2012

Bitmaps Greyscale
Greyscale
256 colours available Black represents 0, white represented by 255 24-bit BMP converted to 8-bit by processing each pixels RGB value

Oct 1, 2012

Bitmaps Colours
Colours
256 Colours created using RGB values from 0% 0% 0% to 100% 100% 100%, e.g. 20% red 0% green 20% blue 24-bit pixels rounded to these boundaries Colours sorted by hue to give a natural colour progression
Oct 1, 2012 8

VHDL AppliedVHDLUSBSimple System Overview

ramDone txDat(7:0)
8

DPIMREF 1.0
Register1(7:0) mclk astb dstb pwr pdb(7:0) pwait Register0(7:0) rxDatValid txDatAck 8 8 8

mclk astb dstb pwr clk

pdb(7:0) pwait dspAddRange(15:0)

32

ramDone datFromRam(31:0) dspAddRange(15:0) 16

dspBlk 3.0
dspDone dspDat2Ram(31:0) 32

dspDone

dspFunc(7:0)

dspFunc(7:0) 8 dspActive

dspRamAdd(17:0) 18 dspRamRd dspRamWr dspDat2Ram(31:0) 32 18 dspRamAdd(17:0) dspRamRd dspRamWr dspActive

memCtrlr 5.0
ramDone datFromRam(31:0) ramAdd(17:0) ramIO(31:0) 32 18 32 CE1L UB1L LB1L WEL

rst

clk rst

ramAdd(17:0) ramIO(31:0) CE1L UB1L LB1L WEL OEL CE0L UB0L LB0L

8 32

IODat2Ram(31:0) IORamWr IORamRd datFromRam(31:0) 32 rxDat(7:0) 8 rxDatAck txDatValid ldByteCnt ldIODat2RamReg csrTask 8 csrOut(7:0) 8 IODat2Ram(31:0) 32 dPathDone txDat(7:0) 8

OEL CE0L UB0L LB0L

IOCSRBlk 4.0
rxDat(7:0) 8 rxDatValid txDatAck dspActive rxDatAck txDatValid ldByteCnt ldIODat2RamReg csrTask

datCtrlr 2.0

IORamAdd(17:0) 18

clk txDat(7:0)

dPathDone

dPathDone dspDone ramDone maxCount(15:0)

16

dPathDone dspDone ramDone maxCount(15:0)

csrOut(7:0) IORamWr IORamRd addrCnt(15:0) 16 CSR(7:0)

clk rst

clk rst

addrAndMaxCount 6.0
8 8 8 8 8 8

Oct 1, 2012

8 8

CSR7(7:0) : dspFunc(7:0) CSR6(7:0) : dspAddRange(7:0) CSR5(7:0) : dspAddRange(15:8) CSR4(7:0) : endIOAddr(7:0) CSR3(7:0) : endIOAddr(15:8) CSR2(7:0) : startIOAddr(7:0) CSR1(7:0) : startIOAddr(15:8) XXXXXX CSR0(1:0) : quadrant For Reference

16

& & &

8 16

dspFunc(7:0) dspAddRange(15:0)

maxCount(15:0)
16 16 2

&

18

VHDL DPIMREF
m em R dVa l 1.3
8 8 00000001

m em RdValid

m em W rVal 1.4

00000000

8 8

m em W rValid

m em RdValid m em W rValid N S & O utput D ecode

CS

d p im refF S M 1.1
txDatA ck rxD atValid

tx D atA ck rxD atVa lid pw ait

0 1

busEppO ut

pdb
8 8

m clk dstb astb pw r clk

clkM ain ctlEppDstb ctlEppAstb FS M C om b

NS ctlE ppW ait ctlE ppD ir FSM Synch ctlE pp Awr ctlE ppD wr

ctlE ppW r

d p im refR eg iste rR W 1.2

busEppIn 8 ctlEppAw r ctlEppD wr reg ister0 busEppD ata 8 regEppA dr 8

reg ister0
8

register1
8 ctlEppW r

register1

clk

Oct 1, 2012

10

VHDL DPIMREF
Sample provided by Digilent Provides VHDL side USB interface with USB 2.0 Peripheral Device Altered to generate rxDatValid and txDatAck signals These signals along with the rxDat and txDat input and output data lines allow communication with rest of the appliedVHDLUSBSimple system
Oct 1, 2012 11

VHDL IOCtrlr
allReadDone 4.1.3 maxCount
16 16

>

allRdDone

allWriteDone 4.1.4
+1 16 16 16

>

allWrDone

X0000" 16

addrCnt 4.1.2 Din Cnt(15:0) Ld CE

addrCnt(15:0)
16

ioCtrlrFSM 4.1.1
allRdDone allWrDone ldAddrCnt incAddrCnt ldIODat2RamReg

clk rst ldIODat2RamReg txDatValid rxDatAck ldByteCnt IORamWr IORamRd dspActive enCSRWr rxDat(6:4) rxDat(1)
3 1

txDatAck rxDatValid dPathDone dspDone ramDone rxDat(7:0)

8

txDatAck rxDatValid dPathDone dspDone FSM Comb ramDone

CS

txDatValid rxDatAck

NS & Output Decode

NS

ldByteCnt IORamWr IORamRd FSM Synch dspActive enCSRWr regCSRInfo clrCSRInfo

rxDat(3:0)
4

rxDat(3:0)

cmdReg 4.1.5
D0(2:0) D1 Ld clr Q0(2:0) Q1 3

clk rst
4

csrAdd(2:0) csrTask

clk rst

rxDat(1) & rxDat(6:4)

Oct 1, 2012

12

VHDL IOCtrlr
Responds to USB transfer signals, rxdatValid and txDatAck, generated by the DPIMREF Allows the system registers (CSRs) to be setup, both DSP and Input/Output Keeps track of the total number of bytes written or read from SRAM and increments the current SRAM address accordingly Controls DSP activation
Oct 1, 2012 13

VHDL DSPBlk
intDSPFunc (int)
A 32 B 32

B-A 32

dDatFromRam(31:0)

datFromRam(31:0)

32

32 0 32 32 32 32 32 1 2 3 4 32

This ANDing reverses the order of bytes -> backwards

dDatFromRam(31:24) dDatFromRam(23:16) dDatFromRam(15:8) dDatFromRam(7:0)

8 8 8 8

&

dspRamDat(31:0)

32

0 32 32 1 32

imageInvDSPWr

Conv_std_logic_vector
0 16 16 16 1 16

dspRamAdd(15:0)

cnt

(int)

%2
16

dspAddRange(15:0)
16

B-A

Oct 1, 2012

14

VHDL DSPBlk
Six DSP functions implemented:
Delta function Colour inversion Flip image over y-axis Distortion Morphing Colour change

Effective demonstrations of the Spartan 3s power but more complex DSP functions are possible
Oct 1, 2012 15

Graphical User Interface

Oct 1, 2012

16

Graphical User Interface

Allows a user to check the contents of any SRAM location Can activate the camera software to take pictures Image loading and conversion from 24-bit to 8-bit BMPs DSP options as mentioned previously Option to perform all operations in colour or greyscale

Oct 1, 2012

17

Future Work
Implement more complicated DSP, including:
Edge detection Object Extraction Object warping

Provide networking solution to allow Electronics department to host the project and make it accessible anywhere in the world

Oct 1, 2012

18

Summary
Initial aims / Specifications Introduction Camera Bitmaps VHDL Graphical User Interface Future Work Summary
Oct 1, 2012 19