A Mechatronic Marvel: The Digital Camera

Dr. Kevin Craig & Dr. Mark Nagurka Professors of Mechanical Engineering Marquette University
Digital Camera K. Craig & M. Nagurka 1

Design News Web Cast

Web Cast Introduction Mechatronics and the Digital Camera The Digital World & Digital Images Photography: Analog vs. Digital Key Features and Functions Difference between Film and Digital: Image Sensors Digital Camera and Mechatronics: How is Mechatronics Enabling the Digital Camera? Autofocus Image Stabilization
Digital Camera K. Craig & M. Nagurka 2

Plant Design

Plant Dynamics & Control Structure

K. Craig & M. Nagurka 3

Digital Camera

A Mechatronic Marvel

Digital Camera

K. Craig & M. Nagurka

Mechatronics
Zoom Lens Auto Focus Auto Exposure

Image Stabilization

Digital Camera

K. Craig & M. Nagurka

The Digital World & Digital Images

Before Digital There Was Analog Analog refers to phenomena that are characterized by fluctuating, evolving, or continually changing physical quantities, such as force or voltage.

Digital Camera

K. Craig & M. Nagurka

Discrete in Time Discrete in Amplitude Continuous in Amplitude

Continuous in Time

D-D D-C C-D C-C

Digital Camera

K. Craig & M. Nagurka

 Early electronic technology was built using an important analog device called the vacuum tube. Such tubes, used in nearly every piece of electronics, were used to control the electrical current and voltage in systems such as radios, radar, and very early computers. These vacuum tubes got very hot and burned out regularly. transistor A vacuum tube, a modern transistor, and an integrated circuit vacuum tube
Digital Camera

integrated circuit
K. Craig & M. Nagurka 8

 Birth of the Digital Age During the middle of the 20th century, mathematicians and engineers discovered a process for converting physical quantities found in the world (e.g., sound waves, light intensity, forces, voltages, current) to numbers or digits. This remarkable, yet simple discovery, was the mathematical foundation that gave birth to the digital age. The word digital refers to technology or phenomena that are characterized by numbers. The era was born with the creation of the transistor. The digital age is generally thought to have reached full maturity at the time that computers gained widespread use, during the mid-1980s.
Digital Camera K. Craig & M. Nagurka 9

 Some advantages to digitizing analog quantities: Real-time adaptability Less susceptible to noise or parameter variation in instrumentation Large amounts of data can be stored using compact, high-density data storage methods Fast data transmission is possible over long distances without introducing dynamic delays Digital processing uses low operational voltages Very high accuracy and speed are possible through digital processing Easier to change system through programming rather than by changing hardware
Digital Camera K. Craig & M. Nagurka 10

 In 1945, engineers produced the first digital computer using the readily-available vacuum tubes as the basic digital building blocks. It was called the ENIAC. ENIAC Built out of more than 17,000 vacuum tubes, weighing 30 tons, and filling a 30-by50-foot room. Imagine the heat produced!
Digital Camera K. Craig & M. Nagurka 11

 In 1947, engineers at Bell Labs developed a small, fast, inexpensive digital replacement for the vacuum tube that consumed much less power the transistor. A transistor is a switch either open or closed that regulates the voltage or current flow through electrical circuits. It is the basic building block for all digital electronic technology. The transistor might be the most important invention of the 20th century. The next critical step forward into the digital age was the ability to put many transistors onto a single small part that could be used for increasingly complex tasks. Jack Kilby invented the integrated circuit, or IC, in 1958. Nearly all the individual components on an IC are transistors.
Digital Camera K. Craig & M. Nagurka 12

The first IC produced by Jack Kilby

A modern IC

Digital Camera

K. Craig & M. Nagurka

13

 Moores Law says that the number of transistors on an IC will double every two years. Equivalently stated, the computing power of ICs doubles every two years. This law is used as one of the strongest principles for predicting the future of technology.

Digital Camera

K. Craig & M. Nagurka

14

 Digital Images Seeing is believing! A picture is worth a thousand words! About half of our brain activity is dedicated to gathering, processing, and storing visual information. Digital Imaging is the technique of representing images as a sequence of numbers. Of course, we also need a way to convert the numbers back into the original picture. The first step in converting a picture into a series of numbers is to create a grid with a uniform cell size and superimpose it on the picture. Each cell is now one element of a picture. A pixel (contraction of the words picture and element) corresponds to the smallest detail in a picture that one wants to preserve.
Digital Camera K. Craig & M. Nagurka 15

Digital Camera

K. Craig & M. Nagurka

16

The number of pixels in the image determines its pixel count. For example, a 640 x 480 image would have 307,200 pixels, or approximately 307 kilopixels; a 3872 x 2592 image would have 10,036,224 pixels, or approximately a 10 megapixels.

Digital Camera

K. Craig & M. Nagurka

17

Photography: Analog vs. Digital

A camera is basically a black box with: a lens (or lenses) to focus the image, an aperture that determines how bright the light is, a shutter that determines how long the light enters , and light sensitive film or an image sensor. To adjust exposure, two different settings can be changed: aperture and shutter speed.

Digital Camera

K. Craig & M. Nagurka

18

Shutter Curtain

Focal Plane

Lens

Optical Axis

Light from subject (A) reaches focal plane through the lens. The lens focuses the image.
Digital Camera K. Craig & M. Nagurka 19

Camera Focus

Digital Camera

K. Craig & M. Nagurka

20

Single Lens Reflex (SLR) Camera

lens

Digital Camera

K. Craig & M. Nagurka

21

Large Aperture Diaphragm

Small Aperture

Digital Camera

K. Craig & M. Nagurka

22

Depth of Field

Shallow depth of field

Digital Camera K. Craig & M. Nagurka 23

Shutter Speed

Digital Camera

K. Craig & M. Nagurka

24

Camera Tripod

Digital Camera

K. Craig & M. Nagurka

25

Film Sensitivity

ISO/ASA 100
Digital Camera

ISO/ASA 200

ISO/ASA 400
K. Craig & M. Nagurka 26

Film vs. Digital Photography

The major difference between traditional film and digital cameras is how they capture the image. Instead of film, digital cameras use a solid-state device (a silicon chip) called an image sensor. On the surface of the image sensor are millions of photosensitive diodes, each capturing a single pixel. Two types of image sensors are used: CCD or Charge-Coupled Device CMOS or Complementary metaloxide semiconductor
Digital Camera K. Craig & M. Nagurka 27

Light Path in a Digital Camera

image sensor

The heart of all digital cameras is the digital imaging sensor. It is the component that converts the light coming from the subject being photographed into an electronic signal.
Digital Camera K. Craig & M. Nagurka 28

 Although the principles of basic photography (aperture, shutter speed, ISO) are the same in a digital camera as in a film camera, the inner workings of a digital camera are quite different. Replacing emulsion-based light-sensitive chemical film, the imaging is performed by an image sensor. (CCD or CMOS sensors). Each sensor element converts light into a voltage proportional to the brightness which is passed into an analog-to-digital converter (ADC) which translates the sensor image signals into discrete or digital data.

Digital Camera

K. Craig & M. Nagurka

29

Sigma DP1 14MP Digital Camera

Image Sensor

Digital Camera

K. Craig & M. Nagurka

30

 The digital output of the ADC is sent to a digital signal processor (DSP) which adjusts contrast and detail, and compresses the image before sending it to the storage medium. The brighter the light, the higher the voltage and the brighter the resulting computer pixel. The more elements, the higher the resolution, and the greater the detail that can be captured.

Digital Camera

K. Craig & M. Nagurka

31

Digital Camera Operation

Digital Camera

K. Craig & M. Nagurka

32

Digital Camera

K. Craig & M. Nagurka

33

Nikon D3 12.1 Megapixel Full-Frame Optical Sensor

Digital Camera

K. Craig & M. Nagurka

34

Nikon D3 Cut in Half

Prism for viewfinder Mirror CMOS Sensor Lenses

Batteries
Digital Camera

Electronics
K. Craig & M. Nagurka 35

Inside the Back of a Digital Camera

Digital Camera

K. Craig & M. Nagurka

36

Image Sensors

Digital Camera

K. Craig & M. Nagurka

37

Digital camera light sensors rely on the ability of certain semiconductor materials to convert light into electrical charge.
Digital Camera K. Craig & M. Nagurka 38

Digital Camera

K. Craig & M. Nagurka

39

Digital Camera

K. Craig & M. Nagurka

40

Digital Camera

K. Craig & M. Nagurka

41

Digital Camera

K. Craig & M. Nagurka

42

Digital Camera

K. Craig & M. Nagurka

43

RGB Inside the Camera

Digital Camera

K. Craig & M. Nagurka

44

ISO 200; 100% Crop

Digital Noise
ISO 1600; 100% Crop

Digital Camera

K. Craig & M. Nagurka

45

Mechatronics & The Digital Camera

Today, Autofocus has all but eliminated fuzzy pictures and is a must for action shots. The implementations of autofocus are diverse and fall into active and passive categories. A sharp image is the photographers elusive goal, but even when focus and depth of field are just right, a photographers hands are still wiggling. Even on a tripod, a gust of wind or a truck rumbling by is enough to ruin a photo. Camera makers have come up with several ways to stifle vibrations in cameras. A Vibration Reduction System must counteract vertical vibration (pitch) and side-to-side vibration (yaw), and every direction in between, and it needs to make the correction within a few milliseconds!
Digital Camera K. Craig & M. Nagurka 46

Autofocus
There are approaches to autofocus in a digital camera: active and passive. In active autofocus, there are two types found on less expensive cameras. The first type is akin to the echo technology of radar and sonar. The other is based on the triangulation used in range finders. In passive autofocus, the light from the image is used to focus the camera. Both the active and passive types require a motor to make work and that motor is an ultrasonic motor.
Digital Camera K. Craig & M. Nagurka 47

 Echo Active Autofocus When the shutter button is depressed, an infrared light is emitted towards the subject of the photograph. The infrared light bounces off the subject and returns to a detector. The time it took for the light to make the round trip is determined usually in nanoseconds. A microprocessor controls a motor built into the lens housing. The motor rotates to a position thats been calibrated to focus on an object at the distance determined by the infrared bounce. This type of autofocus works with objects not more than 30 feet from the camera.
Digital Camera K. Craig & M. Nagurka 48

 Triangulation Active Autofocus Unlike timing autofocus, which uses the same transducer to generate and receive bursts of light, triangulation active autofocus works with one device to shine light at the photos subject and a second device to receive the light after it bounces off the subject. The separate devices allow the autofocus mechanism to make use of the different angles formed by the lights path to the subject and its path on the return trip.
Digital Camera K. Craig & M. Nagurka 49

 Passive Autofocus Here light passes through the lens of the camera and is diverted from the image sensor at the back of the camera by a mirror or prism. The light falls on a strip of 100-200 photocells, called linear sensors, similar to those that make up the imaging sensor. The strip is positioned so that the distance the light travels to it is equal to the distance to the imaging sensor. The cameras processor compares the intensity of the light falling on each photocell to the intensities of the adjacent cells. Adjacent pixels have similar intensities for an out-of-focus image.
Digital Camera K. Craig & M. Nagurka 50

 The Motor that Moves The Lens: Ultrasonic Motor Any autofocusing camera must have a motor to move the lens elements to bring the subject into focus. Great speed and precision is required and there is so little space! The Ultrasonic Motor meets those requirements! The Ultrasonic Motor is built on a phenomenon called the piezoelectric effect. The effect is shown in the diagram for the substance zirconium titanate (PZT), a ceramic lead.
Digital Camera K. Craig & M. Nagurka 51

 The next step in creating a piezo motor is to bond PZT to both sides of a flexible member (steel spring metal) and send opposite charges to alternating sections of the beam. The charge on one section makes the beam bow out at the same time the opposite charges going to the sections on either side make them curve inward.

Digital Camera

K. Craig & M. Nagurka

52

Digital Camera

K. Craig & M. Nagurka

53

Image Stabilization
Normally, light rays from a scene being photographed pass through the object lens element at the front of the lens and stay true to the lens optical axis.

To Subject

Digital Camera

K. Craig & M. Nagurka

54

 When the lens moves, the beams of light pass through elements of the lens at the wrong points. More importantly, the image sensor moves out of position and the light is smeared across the surface of the sensor, blurring the subject.

Digital Camera

K. Craig & M. Nagurka

55

 Preventing vibration and movement is virtually impossible. Any system that seeks to reduce blurs caused by camera movement must do something to put the light beams back on the right path before they hit the sensor. Electronic Image Stabilization is the one exception to this rule as it corrects the blur with software after it has occurred. How Does A Camera Detect Movement? When the system is active and something jostles the camera, the movement is detected by two gyro-sensors, which are gyroscopes attached to devices that measure motion. One is used to detect yaw and the other, mounted 90 away from the first, detects pitch.
Digital Camera K. Craig & M. Nagurka 56

Simplified Physical Explanation of Gyroscopic Precision

Fdt = dG Mdt = dH

Digital Camera

K. Craig & M. Nagurka

57

 Vari-Angle Prism System In a lens using a vari-angle prism (VAP) to correct movements, gyro-sensors are used to detect the slightest wiggle. The sensors send electronic information to a microprocessor built into the lens.

Digital Camera

K. Craig & M. Nagurka

58

 The microchip sends an electrical signal to voice coil motors, smaller versions of the motors used to vibrate the cones in speakers. The motors sit at 90 angles to each other on the rim of the vari-prism.

Digital Camera

K. Craig & M. Nagurka

59

 Piezoelectric Supersonic Linear Actuator Stabilization The VAP system just discussed attempts to bend the errant light rays as they pass through the lens so they strike the correct pixels on the image sensor. This is like trying to deflect an arrow on the way to its target. An opposite tact is to hold the target and move it wherever the arrow is coming in.

Digital Camera

K. Craig & M. Nagurka

60

Y-Axis Actuator

Vertical (Y-axis) Slider

Image Sensor

Microprocessor

Horizontal (X-axis) Slider

X-Axis Actuator

Digital Camera

K. Craig & M. Nagurka

61

Image Sensor stabilization

Digital Camera

K. Craig & M. Nagurka

62

 Shift Image Stabilizer This form of image stabilization (IS) is among the most common. It relies on an IS lens element mounted among other elements that make up the complete lens. This element is like the other lenses that make up the complete lens, except it can change position. Gyro-sensors detect vertical and horizontal movement. The lens microprocessor uses this information to send signals to the voice-coil motors mounted at 90 angles to one another along the edge of the IS lens element.

Digital Camera

K. Craig & M. Nagurka

63

Digital Camera

K. Craig & M. Nagurka

64

 The motors slide the element in the direction opposite from the direction the lens moved. If the lens dipped down, the motors push the IS element up. The correction takes 2 thousandths of a second to complete. Light now enters and exits the IS element at new angles that deflect the light so it regains its original path to the image sensor. Shift stabilization is most effective for lower frequency movements caused by platform vibration or wind.

Digital Camera

K. Craig & M. Nagurka

65

Lens-based stabilization

Digital Camera

K. Craig & M. Nagurka

66

Nikon Vibration Reduction System

Digital Camera

K. Craig & M. Nagurka

67

Survey Questions
The Digital Camera is one of the ultimate mechatronic systems ever developed. Do you work on systems of similar complexity? How is the integration required for a mechatronic system such as this implemented in your company? How is the control system design done? In an integrated way or as after-thought add-on? What skills do you feel you are lacking that you would urge young engineers to focus on?
Digital Camera K. Craig & M. Nagurka 68

THANK YOU!

Digital Camera

K. Craig & M. Nagurka

69