# SIFT Forth

Views:

Category: Education

## Presentation Description

No description available.

## Presentation Transcript

### SIFT, FPGAs and Forth:

SIFT, FPGAs and Forth Dave Wyland Reasonable Machines

### Overview:

Overview SIFT algorithm for Object Recognition SIFT: FPGA vs DSP Forth Machine in FPGA SIFT on the R1A1 - The R1A1 Robot Project SIFT and R1A1 Status Implications for Robotics and HBRC

### SIFT – What Is It?:

SIFT – What Is It? An algorithm for object recognition Developed by David Lowe of University of BC Identifies objects using ordinary cameras (TV, webcam) At any distance For any orientation (pose) For (almost) any lighting With simple image distortions

### SIFT – What Good Is It?:

SIFT – What Good Is It? Lets us recognize objects automatically Gives bearing and approximate range with 1 camera Gives pose of object: normal, tilted, etc. Lets our robots see what we see Lets us build robots to go to named objects, pick them up

### SIFT – How does it work?:

SIFT – How does it work? Outline Find keypoints using Difference of Gaussians 2D Gaussian filters of image Look at 16x16 patch around keypoint Generate a 128 dimension feature vector per point Record new or search for match among recorded

### SIFT – References:

SIFT – References Lowe, David, “Distinctive Image Features from Scale Invariant Keypoints,” International Journal of Computer Vision, 2004. Google for SIFT, Scale Invariant Feature Transform, David Lowe

### SIFT – How Much Computation?:

SIFT – How Much Computation? Steps in Algorithm Keypoint finding: 49.6 x 10^6 pixel operations Keypoint descriptor generation: 1.4 x 10^6 pix ops Keypoint search: 100 ref objects: 32 x 10^6 compares

### SIFT – FPGA vs DSP:

SIFT – FPGA vs DSP 600 MHz ADI Blackfin DSP vs Xilinx Spartan 3 FPGA Time for DoG on 640x480 Pyramid, 3 planes/level Blackfin: 12 frames/second at 600 MIPs FPGA: 41 frames/second at 50 MHz

### FPGA Characteristics:

FPGA Characteristics

### Xilinx Spartan 3 Evaluation Board - \$99:

Xilinx Spartan 3 Evaluation Board - \$99

### Xilinx Spartan 3 Evaluation Board - \$99:

Xilinx Spartan 3 Evaluation Board - \$99

### FPGA: Housekeeping CPU:

FPGA: Housekeeping CPU Need a CPU function for housekeeping, supervisor Plenty of resources in FPGA for a CPU Proposal: 32-bit CPU running Forth => Forth machine 32-bits to address at least 1Meg of memory Added benefit: Use primitives for FPGA custom functions Launch & sequence keypoint processing steps Control complex I/O, etc.

### Forth Machine in FPGA:

Forth Machine in FPGA One 512x36 block RAM holds data and return stacks 512 words => 8 sets of stacks for multitasking Use internal block RAM and external SRAM Use 2+ block RAMs = 4 Kbytes for fast primitives Have >200 Kbytes of external SRAM available for code, variables Boot load from/save to 32 Kbyte serial EEPROM Ref: Stack Computers by Philip Koopman

### Forth Machine Support Idea:

Forth Machine Support Idea Use browser in a PC/Laptop for an IDE Provides a clean way for visualizing Forth Small TCP/IP stack in FPGA to communicate with Net, browser Use web page and HTML links for word display Show colon defs as a list of links to the words in the def Click on a link, and nest into sub word Primitives show the Forth machine instructions

### Forth: Some Controversial Ideas:

Forth: Some Controversial Ideas Use Data Stack for parameters, not calculation Data stack ops are arcane: RPN calculators have lost the war Use local variables + a “frame pointer” FP = SP saved at entry Input parameters have fixed locations relative to FP Have ops that reference stack values rel to FP + local vars Use fixed size return stack “frames” for local variables Push 8 words instead of 1 1 word for return addrs, 1 for loop count, 6 for local variables

### SIFT on the R1A1 :

SIFT on the R1A1 David Wyland Reasonable Machines

### The R1A1 Project:

The R1A1 Project A Prototype to Test a Robotic Architecture Animal Training as a Robot Design Model Platform: Zagros base, Mini-itx CPU, Lynxmotion arm, webcam Test Task: Go get a named object and bring it back David Wyland Reasonable Machines

R1A1

### R1A1 Block Diagram:

R1A1 Block Diagram David Wyland Reasonable Machines

### R1A1 Status:

R1A1 Status SIFT algorithm running on a PC Demo at HBRC meeting SIFT Gaussian filter running in FPGA R1A1 elements to be done Platform control: drive motors, arm servos, camera control Dialog mgr (alicebot), Task mgr (RAP), Behavior mgr

### Robot Design References:

Robot Design References Wyland, David, “Autonomy Without Independence: Animal Training as a Robot Design Model,” 2nd Workshop on Radical Agent Concepts (WRAC), NASA Goddard Spaceflight Center, September 2005. To be published by Springer-Verlag Wyland, David, “Reasonable Machines: Analogical Reasoning in Autonomous Agent Design,” 1st Workshop on Radical Agent Concepts, NASA Goddard Spaceflight Center, January 2002. Springer-Verlag

### SIFT: Implications for Robotics:

SIFT: Implications for Robotics Robots are currently blind and deaf Try walking around with your eyes closed + earplugs With SIFT, robots can move around and pick up objects Work in changing environments With SIFT, we can make robots just like in the movies Walk, talk and do things for us

### SIFT: Implications for HBRC:

SIFT: Implications for HBRC With a SIFT Camera, We can make robots that: Walk around without bumping into things Can go to named objects and pick them up With a SIFT Camera, We Can Make Real Robots Just like In the Movies (JLIM™) With SIFT, we can make robots just like in the movies

### That’s All, Folks:

That’s All, Folks David Wyland Reasonable Machines