Sensors: Getting Engaged!

Event

I/O: The Museum Inside-Out/Outside-In 38th Annual MCN Conference Austin, Texas

Presented by

Abhijit Rao
1

Plan A Computing Associates

thepaca.net

Sensors: Getting Engaged!

thepaca.net

Sensors: What are they?

Miniature devices that capture sensory data Sensory data incl. heat, humidity, light.... Micro Electro-Mechanical Systems (MEMS) Data transmission is wireless, whereas the Sensing mechanism and power supply are not Sensor: Structure
Physical Abstract
Transducer + Control Logic (Analog + Digital) Part of a bigger network
3

React to external stimuli

thepaca.net

Sensors: What are they all about?

Captures sensory data from real physical world Detecting relevant quantities of the data Monitoring and collecting the data Assessing and evaluating collected data Formulating meaningful information

To be conveyed to the user To be stored for future

Performing decision making

4

thepaca.net

Sensors: Why Museums?

Traditional Uses User interaction Exhibit Design Ubiquitous Computing
Where technology recedes into the background of our lives. - Mark Weiser

Planned Activity

Wow! Sensing Capability Tetherless Experience

Sensors provide

Physically unobtrusive, low power, form factor, low maintenance New sensing paradigms

thepaca.net

Sensors: Outlining a Framework

Sensor Net Link / Agent

Exhibit

Back Office Observer

Remote Exhibit

Sensor Fusion Sensor Management db Management ...

Services

thepaca.net

Sensor: Ecosystem Players

Multiple devices per square meter Mixed data rate requirements Varying streaming requirements Real-time and tolerate latency Databases Static Database Content, Plan & Rules Dynamic Database User activity context, Sensor data Applications Visitors Database Applications Middleware Services Sensors

thepaca.net

Sensors: Sense Element

Physical property to be sensed

Transducer

Detectable signal

Transducer Input (Property of Interest Output (Property of Interest type for Sensor) for Actuator) Digital representation (along Sensor Acceleration measured the axes) Sensor Actuator Actuator Pressure measured Digital Signal Digital Signal Voltage output Voice Rotary Motion

Also known as Accelerometer Pressure sensor Loudspeaker Electrical Motor

thepaca.net

Source: Springer.com Author: J Fortin

Sensors: Typical Sensor

Physical Property such as heat, light

Sense Element Signal (Charge current)

thepaca.net

Source: Springer.com Author: J Fortin

Sensors: Transducer Details

Energy Domain Measurand Sense Principle Piezoresistive, Capacitive Thermistor, Thermo-mechanical Resistive, capacitive Electro-Magnetic, Contact Sensor Vibration (Proof-Mass), Capacitive Capacitive, Electro-Magentic Piezoresistive Piezoresistive, Sound Contact, capacitive Hall effect, Magnetic, RF Fingerprint, data stored, retinal scan Physical Properties Pressure Temperature Humidity Motion Properties Position Acceleration Gyroscope Contact Strain Vibration Presence Tactile/Contact Proximity Identification ID

Source: Silicondesigns.com
10

thepaca.net

Sensors: SMART Sensor

Provides extra functions beyond just the correct representation of the sensed physical quantity
Moving the intelligence sor closer to the point of Sen RT SM A measurement S.M.A.R.T. SelfMonitoring, Analysis, and Reporting Technology Physical
Input Storage (Threshold / Calibration)
Control Logic

Base Station

Host

Communication Interface

Application Algorithm

User Interface buttons/led

Virtual Sensor

Sense Element (MEMS)

Signal Conditioning

Analog-to-Digital Conversion

Digital Signal Processing

11

thepaca.net

Sensors: Virtual Sensor

Base Station

Source: sparkfun.com
Host
sor S en art Sm

Communication Interface Application Algorithm User Interface buttons/led

Virtual Sensor

Storage (Threshold / Calibration) Control Logic Signal Sensor Conditioni Element Virtual Sensorng

Analog-toDigital Conversion

Digital Signal Processing

12

thepaca.net

Sensors: Wireless Connectivity

13

thepaca.net

Source: laboratoryequipment.com

Sensors: Power & Networking

14

Source: techbriefs.com

thepaca.net Source: laboratoryequipment.com

Sensors: Key Drivers

Drivers Scale Today's sensor Micro future Nano energy harvesting

Power consumption long life (> 2 years)

Improving Cost & Yield Standardization efforts in Wireless Networking Cheaper Efficient Tinier Processing Logic Quick Integration Methods Development of low power radio technologies Advances in low-power embedded micro-controllers
15

thepaca.net

Sensors: System Design

Wireless Networking Middleware Services Data Management Sensory User Interaction (SUI) Approach
Identify the use case scenarios Calculate anticipated workload

List sense element and energy domain details Identify the sensor network algorithms

Perform the Sensor System Design in conjunction with the Exhibit Design
16

thepaca.net

Sensor: Identifying a Sensor

Identify the measurand Usage Pattern
Energy Domain Bandwidth and other resource requirements Power consumption Network support Form factor

Characteristics

Volume of sensors to deploy and use Vendor selection

Minimum Quantity requirements Lead time
17

thepaca.net

Sensors: Technology restrictions

Size and packaging Energy efficiency (Power constraints) Data rates for the wireless data transfer

Fine grained estimation and tracking of location and orientation of large number of objects that are being sensed Quality of Service needs

High data rates (Audio/Video) / Streaming Low data rates (spatial sensors, environment sensing)

18

thepaca.net

Sensors: System Example

Sensors
Location & Orientation Identification

- Lightweight

Network Agent Conduit for the sensors Master Act upon sensor data
Location & Orientation Identification Object Tracking

Spatial Configuration of multiple users and objects

19

thepaca.net

Sensors: Outlining a Framework

Sensors Sensor 1
Handheld

System Hardware

Software App stack Exploration & Learning Plan

Sensor 2

Agent

Host
Computer Web

Content Management

Sensor 3

Exploration & Learning Interaction

20

thepaca.net

Sensor: Getting Engaged!

Sensors are a mature market! Using those sensors in a large scale, in Museums, will have a revolutionary effect in user experience. Sensory User Interaction is here to stay!

If there is sufficient interest we can explore opening a SIG under MCN To receive these slides, please send an email to abhi@thepaca.net

21

thepaca.net

Thank you!

22

thepaca.net

Backup

23

thepaca.net

Sensors: Experience Realms

Sensors can enhance the all experience realms

24

thepaca.net

Sensor: Sample applications

Wireless network of toys Learning individualized to the participant / each child Hidden in the infrastructure (walls, chairs, ...) Music toys (MIT Media Lab's Toy Symphony)
Heavy on gestures and tactile in nature

Unobtrusive evaluation of the learning process

Embed sensors for touch, pressure .. in toys and exhibit objects and detect their manipulation
25

thepaca.net

Sensors: Compare to a typical Computing System

Ad-hoc and Distributed structure Likely to be large scale yet sparse in density Unreliability due to nature of the link Restricted system resources, Power management Overcome by huge amount data that gets collected (redundancy) Physical stimulus and reactionary in nature Predominantly unstructured and asynchronous Diversity of data types with large dynamic range of rate, latency, processing requirements
26

thepaca.net

Sensors: Transducer Properties

Physical Properties Used to measure Pressure, temperature, Humidity... Motion Properties Used to measure Acceleration, Position, Angular velocity, Direction Contact Properties Used to measure Vibration, Slip, Strain Presence Used to measure Tactile, Contact, Proximity, Motion, Light Identification Used to measure ID Data, Retinal, Voice, ...

27

thepaca.net

Sensors: Sense Element Basics

Capacitive Sensors as in Touch Sensors Piezoelectric effect as in balls that lit when bounced Eddy-current sensors allows bigger sensing range and bigger probe spot size RFID use power in the interrogating signals to send back small amounts of data Thermoresistive effect exploits resistance of a conductor/semiconductor changes with temperature Thermocouples sensor based on current flows between two materials are joined together and one is hotter than the other. It is less expensive and is very reliable Resonant Temperature sensors are expensive and very high accuracy. Photoelectric effect measures the flow of electrons based on incident light

Useful for detecting Electromagnetic proximity / Magnetic events

Thermal

Optical

Measure temperature or heat flux convert light to various quantities

28

thepaca.net

Source: eetimes.com

Source: lionprecision.com

Mechanical

Rely on direct Physical contact

Source: analog.com

Energy Domain

Characteristic

Transduction Principle

Sensors: Other Sensor Elements

Description / Uses Interact with matter of all types Used in sensing Chemical pollutions causing gases Biological Medical Diagnostic, detecting Pesticides Implemented with an array of chemical sensors, interpreted using a pattern-recognition algorithm to sense a variety of chemicals. Used in Contamination Chemiresistors assessment relies on adsoprtion of gases onto certain Metal-Oxide semiconductors changes their resistivity. Used in Gas sensors smoke detectors Sensing depends upon selectivity of biomolecular reactions such as molecular binding sites detect property changes mass, resistance, spectral reading Biosensors EM spectrum Radar / Lidar, GPS Sound as the sensing medium detecting fluid leakages and earthquakes Acoustic Employs two sets of fingers one to generate the surface Acoustic Wave and the other to detect the Acoustic Wave changes. Very versatile in its applications. Energy Domain
Source: sandia.gov thepaca.net Source: sensormag.com

29

Sensors: Mapping a Smart Sensor

Base Station Host

sor Se n rt Sm a
Storage (Threshold / Calibration) Control Logic

Communication Interface

Application Algorithm

User Interface buttons/led

Sensor Signal Element Conditioning Virtual Sensor

Analog-to-Digital Conversion

Digital Signal Processing

30

thepaca.net

Sensors: MEMS Sensor SoC

Base Station

Host

Source: invensense.com

sor Sen rt Sm a
Storage (Threshold / Calibration) Control Logic

Communication Interface

Application Algorithm

User Interface buttons/led

Sensor Signal Element Conditioning Virtual Sensor

Analog-to-Digital Conversion

Digital Signal Processing

31

thepaca.net

Sensors: Form factor

Source: monnit.com Source: kcftech.com Source: freescale.com
32

Standalone

Integrated onto a PCB

thepaca.net

Sensors: Adoption Today

Building industry, Green evolution, Transportation, Home automation, ... Importance indicated by IEEE driven standards in this space Driven by affordability and ease of use Efficient and low cost connectivity methods Classified solutions based on data rates

33

thepaca.net

Backup

34

thepaca.net

Sensor: Challenges
Power Lack of ruggedness Steep learning curve

35

thepaca.net

Actuator: Display Technology

Source: flexlighing.com

Source: holografika.com

Source: microvision.com

Source: mirasoldisplays.com

36

thepaca.net

Actuator: Tactile Feedback

Source: immersion.com

Source: densitron.com

Source: densitron.com

37

thepaca.net

Actuator and Sensor Combined: Touchable 3D Display

Source: docomo

38

thepaca.net

References
Weiser, Mark The Computer for the 21st Century Scientific American Vol 265 p 66-75 Smart Kindergarten: Sensor-based Wireless networks for Smart Developmental Problemsolving environments - ACM ISBN 1-58113-422-3/01/07
http://www.ubiq.com/hypertext/weiser/SciAmDraft3.html

39

thepaca.net