NE Handbook Sensor Networks NE Handbook Series
Sensor Networks
Transforming society through sensor network technologies
al Logistics Comm ustry Medic ercial on Ind BEMS ortati p s HEM n ra T S A e r u gric t c u r ultu t s a re Infr
e Mobil
Secur ity
Big Dat a
Sensing
netism Pressure Accele eomag ratio n G re G u t a yro per m e Tou T l ch l a H MCU Im ag UV e s s n u I i c m a tion MC om e C s s e l U s S ire W e n s s or C CU rM on tro we lle Po r w RF
s
Lo
d are nfr
Br igh tn
ble ara We
Health Care
tooth EnOcean 92 Blue 0M Hz Fi Wi
ROHM Sensing Solutions
ROHM's vision of a connected society that uses sensor networks for processing and transmitting information instantaneously over large distances is becoming a reality. By combining a wide range of compact, high-performance sensors with industry-leading microcontrollers and the latest wireless technologies, the ROHM Group is able to provide complete sensing solutions optimized to application requirements.
contents
2
［Overall trends ］
Sensors covering our lives and social
activity usher in a new era in information
［Applications / The human body ］
［Applications / Vehicles ］
［Applications / Civil engineering ］
［Applications / Architecture ］
［Applications / Agriculture ］
［New trends / Trillion Sensors ］
［New trends / IoT, IoE, and Big Data ］
［New trends / Energy harvesting ］
［New trends / Wireless networks ］
［New trends / Infrared sensors ］
［New trends / Smart Meters ］
［New trends / MEMS sensors ］
［New trends / CMOS sensors ］
10 16 20 26 30
34 44 46 50 52 54
56
Supporting ADAS with diverse sensors Sensor-driven infrastructure maintenance Information technology saves energy in even small buildings Visualizing agricultural product status
A trillion sensors create new businesses
“Things” generating massive amounts of data From low-power to no-power Diverse techniques for each application
Applications expanding from automotive sector Network control for household appliances, too Proving performance in consumer electronics The key to image data acquisition
NE Handbook
55
Medicine and healthcare beyond the five senses
［Overall trends］
Sensors covering our lives and social activity usher in a new era in information Sensors and sensor utilization technologies are becoming increasingly important in a range of fields where electronics hasn’t been fundamental, such as automotive, medicine, construction, energy, and agriculture. Sensors have mostly been used for autonomous control of independent pieces of equipment, or system operation, such as factory automation（FA ）systems automatically controlling machine tools and other manufacturing equipment, or microwave ovens that monitor food temperature to cook automatically. In general, the ways sensors are used are developing in two major directions（Fig.1 ）.
The first is working to create new applications and new value by using increasingly sophisticated technology to analyze and interpret the data collected by sensors. Sensors and （1）Improved technologies for analyzing and interpreting data Technologies combined with sensor technology Utilization technology for microprocessors, FPGAs, etc. Digital signal processing technology Application fields Self-driving cars, non-contact and non-invasive medical tests, etc. （2）Collecting and integrating large amounts of data to extract information Technologies combined with sensor technology Radio and other network technologies Information processing technology utilizing big data Application fields Monitoring in construction, agriculture and livestock, and energy
Fig.1 Sensor utilization evolving in two directions
2
NE H a n d b o o k
autonomous control systems are evolving for expanded use in fields demanding safety and reliability, such as vehicle automatic braking functions, and medical electronics to continually collect patient data. The second is collecting and integrating the data from a large number of sensors with network technology, and extracting information useful in improving the quality of life or social activity. The maturation of “big data” thanks to new developments and applications in the information and communications technology（ICT ）field is driving this trend. Sensors today are social devices, critical in the creation of sophisticated infrastructure. This report takes a closer look at sensor utilization trends as they develop in these two directions, and the evolving sensors that continue to make new applications possible.
New applications through developments in analysis and interpretation
The technology to analyze and interpret data collected via sensors is evolving rapidly. Sensors can directly input physical quantities such as temperature, pressure, velocity, or brightness. Even the image sensor, one of the most advanced sensors, cannot sense a two-dimensional（2D ）map of brightness distribution over an object. The only way to obtain information significant in controlling equipment, such as determining what an object is, or what state it is in, is to analyze and interpret collected data. Recognition technology, needed to extract information essential in equipment control from collected sensor data, has evolved enormously in recent years, thanks to rising performance and falling prices for the semiconductor devices used
3
Overall trends
for image processing, such as microprocessors, digital signal processors（DSP ）, and field-programmable gate arrays （FPGA）. Another major factor is the continuing evolution of the algorithms used to efficiently analyze and interpret data. Here are a few representative examples. Right now, perhaps the most exciting field of technology development for vehicles is autonomous driving（pp.16-19 ）. Practical self-driving cars will have to be equipped with a diverse variety of sensors, and engineers are already considering image sensors, infrared sensors, millimeter wave radar, laser radar, and ultrasound sensors, among others. The automatic braking function, positioned as a necessary preliminary to autonomous driving, is already in commercial use, and became mandatory on all commercial vehicles in the （a）Grasp position of vehicle two cars ahead using millimeter wave radar Alarm（audible, visual） and automatic braking
Sensed vehicle
Millimeter waves pass under first car
（b）Dual single-lens cameras Lane sensing camera
High-beam control camera millimeter wave radar
Fig.2 Sensor fusion application in driving
4
NE H a n d b o o k
European Union（EU ）in 2013. Analysis and interpretation of imagery from around the vehicle provides information on positions and distances of nearby vehicles, pedestrians, and lane lines. Before automatic braking can be upgraded to full-fledged automatic driving, however, recognition precision must be improved. Automobile manufacturers and sensor manufacturers both are working to improve sensor precision, and develop sensor fusion technologies to integrate multiple
sensors（Fig.2 ）. The key to success here will be how to best utilize different types of sensors to implement a safe, lowcost autonomous driving capability.
Constant health monitoring In medicine, technologies used to analyze and interpret sensor data are being utilized to acquire biological data noninvasively, and even without physical contact. Until recently, the biological information essential in medical diagnosis was usually obtained by restraining the patient, or attaching electrodes to the patient’s body, or by some other unpleasant method. No doubt many people have felt stress or nervousness when having their blood pressure or an electrocardiogram taken. With non-contact, non-invasive measurement, however, patients wouldn’t need to worry so much. The approach would combine accuracy with convenience, and has already been positioned as a key technology for constant health monitoring systems. Concretely, Fujitsu Laboratories Ltd. of Japan has developed a system to measure pulse rate and heart beat using an image sensor. Image processing technology analyzes facial colors from a video stream to detect the pulse.
5
Overall trends
Tapping into big data to expand application Another direction of growth is collecting and integrating data from sensors, and extracting information useful in improving the quality of life and social activity. Concepts like the Internet of Things（IoT ）, big data, and machine-to-machine（M2M ）are driving new information systems for the new era in the ICT field（pp.44-45 ）. By combining these information systems and sensor networks it is now possible to extract information from the massive volume of collected sensor data and illuminate previously invisible social activity. Google Corp. of the US created a bold new service with significant value by organizing the huge amount of information in virtual space. Sensors, big data, and IoT together will expand this revolution from virtual space into the real world. It is possible that godlike services could emerge, capable of answering instantly such questions as “How many
Fig.3 The TSensors Summit（October 2013 ）
6
NE H a n d b o o k
people in the world are awake right now ?” The organizations involved in developing sensor technology are steadily bringing this future closer. At the TSensors Summit in the US a project was launched to create the Trillion Sensors Universe, a company using sensor networks on
the order of a trillion sensors a year（Fig.3, pp.34-43 ）. It will work on technology development and application for the supply and use of more sensors than ever tried before: a trillion sensors is about one hundred times more than the total annual sensor demand at present. Trillion Sensors plans to put sensors on all sorts of equipment and tools that have never had them, bringing about a world enjoying all the benefits of ICT. In short, it is a key technology for the realization of the Internet of Things.
Sensor networks as infrastructure In Japan as well, a national project is under way to use sensors to support society as high-level infrastructure: the Project to Develop Sensor Systems for Responding to Social Issues, launched by the Ministry of Economy, Trade and Industry（METI ）. The project was consigned via the New Energy and Industrial Technology Development Organization（NEDO ）to NMEMS technology research association of Japan. Development is under way now on sensor basic and application technologies. There are a large number of issues in Japan which demand a major overhaul of the infrastructure. In construction the social and industrial infrastructure suffers from a variety of problems due to natural disaster and age（pp.20-25, pp.26-29 ）.Agriculture needs to improve safety and reliability of agricultural and livestock products, thereby improving international competitiveness（pp.30-33 ）. In medicine, soci-
7
Overall trends
ety faces skyrocketing medical costs brought on by a society of fewer children and an aging population. Social issues such as these involve huge scales, and complex systems of nature, plants and animals, and people, making it difficult to adequately grasp the causal phenomena. In order to fully comprehend a phenomenon, it is essential to collect information on diverse aspects of society. NMEMS has proposed a constant monitoring system for social issues, which would run continuously to help grasp involved phenomena and provide the needed information for decisionmaking. It is developing the necessary constituent technologies now.
The impact of sensor evolution Much of the new sensor development in progress now is actually just new application technology for existing sensors. If new sensors are developed for new applications, of course, adoption will accelerate. A number of key areas have been identified as promising for new sensor development: finding ways to capture data currently unavailable, integrating information processing and communication functions, multi-module designs incorporating functions from different types of sensors, arrays of sensors of a given type, and improved reliability and environmental resistance to make it possible to utilize them in as many situations as possible. An example of a sensor to capture information that was previously difficult to get would be the near infrared image sensor from Rohm Co., Ltd. of Japan, which is being developed collaboratively with the National Institute of Advanced Industrial Science and Technology（AIST ）, and will provide visualizations of the human body from the skin to a depth of
8
NE H a n d b o o k
a few cm（pp.10-15 ）. The sensor will be able to make cancerous cells visible from outside the patient’s body. Blood flow can also be visualized, suggesting that it might be possible some day to “read” emotions based on differences in blood flow caused by differing areas of activity in the brain. Technologies to integrate multiple sensors and peripheral circuits, to miniaturize sensors, and to improve reliability are also crucial. Emerging cloud technologies mean that data no longer has to be stored or processed where it is acquired. Instead, it can be processed by a central facility in a site designed for smooth operation of electronic systems. The information input functions of the sensors, on the other hand, must be located where the data is born, which means they must be environmentally tough, compact, power efficient, and low cost. Electronic systems consist of input, output, processing, storage, and communication. Of these, there has been major improvement in performance and function for output （display technology ）, processing（microprocessors and memory ）, storage（memory, hard disk drives ）, and communication（network technology ）, but input continues to lag far behind. On the other hand, that same fact also means that the appearance of new sensor technologies could trigger enormous development in the utilization of information throughout society.
9
［Applications / The human body］
Medicine and healthcare beyond the five senses Visualizing the formerly invisible portions of the human body, and slashing the cost of short- and long-term health care to monitoring lower the barriers for start-ups. A number of exciting new proposals are being made in the medical and health care fields, a change from the semiconductor and electronic component fields that have been popular of late. Semiconductor and electronic component manufacturers until recently have concentrated their efforts in consumer electronics, such as smartphones and personal computers, but now they are combining the high-reliability, low-cost technologies gained with newly developed technologies to expand into non-consumer fields. The primary business of ROHM Co., Ltd. of Japan is semiconductors, but the firm began expanding its scope of application beyond consumer electronics quite some time ago. Rohm Managing Director Hidemi Takasu, in charge of the R&D Headquarters, explains that development adopted the
Fig.1 Vascular imaging with near infrared light
The image on the left was taken with a CIGS image sensor, using 880nm near infrared illumination. Shots taken with 780 and 950 nm illumination were added to the image on the right, and images processed to show vessels distinctly.（Source: Rohm ）
10
NE Handbook
Visible Light Image
NIR Image
Blood Vessel
Fig.2 Applications in biometric verification
The visible light image is shown on the left, and the near infrared image at right.（Source: Rohm ）
motto “More than Moore” twenty years ago. Until recently the firm has primarily worked on expanding the range of product application while ignoring the restraints imposed by Moore’s Law on performance improvements in PC and server semiconductors.
Visualizing the human body and measuring stress with new materials
For example, the firm is working on a semiconductor that offers optical and chemical properties superior to those found in silicon, a commonly used semiconductor material. This new material will enable the development of new medical and biological sensors, and low-power monitoring elements capable of operating for extremely long periods. The new material will be used in an X-ray imaging sensor with 10µm resolution, an imaging sensor capable of showing non-visible spectra from infrared to ultraviolet, and more. One image sensor can visualize the human body（Fig.1 ）,
11
Applications / The human body
using near infrared light with a wavelength longer than that of visible light. Near infrared can penetrate about 3cm into flesh, and in this system is emitted by a light-emitting diode （LED ）and reflected light detected by a custom imager using a compound CIGS（copper indium gallium（di ）selenide ） membrane. By sensing different infrared frequencies separately for image processing, blood vessels and other features can be made to stand out（Fig.2 ）.
If vessels can be visualized clearly, then it can be utilized in high-precision biometric verification. An application of Raman scattering with 1.2µm near infrared light can also identify cancerous cells clearly. In X-ray imaging technology, the new sensors will deliver scans that are clearer and more precise than existing X-
ray systems（Fig.3 ）. An X-ray of a fish, for example, shows SOI Pixel（INTPIX4 ） Pixel Size : 17um×17um No. of Pixel : 512×832 （= 425,984 ） Chip Size : 10.3mm×15.5mm Vsensor=200V, 250us Int.×500 （=1/8s ） X-ray Tube : Mo, 20kV, 5mA
High resolution image can be captured!
Fig.3 Tiny fish bones captured by X-ray image sensor
An X-ray photograph of a fish made by the X-ray-sensitive image sensor developed by Rohm.（Source: Rohm ）
12
NE Handbook
Fig.4 Blood tests with minute samples
The blood sample is fed into the chip, and the chip placed into the equipment. Results are ready in about six minutes.（Source: Rohm ）
bones as small as 10µm in size. The firm is currently developing a system capable of detecting mammary cancer. Another development is a single-chip sensor that can digitize a variety of wavelengths. The chip incorporates sensors sensitive to light frequencies in the ultraviolet, visible, and infrared spectra. This spectrometer is implemented as a single semiconductor chip, and as a result will be small, low dissipation, and（in volume production ）low cost. Researchers have also developed a transceiver module for terahertz （THz ）radiation, which can pass thorough a variety of materials with unique reflection characteristics. One type of low-dissipation semiconductors is the nonvolatile device, which is off unless needed. In addition to medical devices, Rohm is also developing
and selling medical equipment, including Banalyst（Fig.4 ）. A minute amount of blood is drawn from the subject, and in five or six minutes Banalyst outputs results for a number of key factors including HbA1c, CRP, and hsCRP. The device can also be used with saliva, and offers an objective, quantitative measurement of stress level, which until now has been measured through verbal interaction.
13
Applications / The human body
A method already exists to quantify stress and tension by measuring the amount of green light reflected by fingertip blood flow（Fig.5 ） . Green light is absorbed by blood hemoglobin, which means that reflection intensity varies with blood flow. Results can be used to measure pulse rate, stress, and tension, as well as assist in diagnosis of vessel aging and atherosclerosis.
New industries through expanded human senses “We are looking into ways of using these devices to expand human senses, and create new industries,” continues Rohm’s Takasu. He suggests that it would be possible to sense information that human beings cannot, and feed that information back in a human-sensible form（Fig.6 ）. For example, new sensors can pick up light that human eyes cannot see, or sounds that human ears cannot hear, and make them sensible to human beings by directly stimulating the brain, or muscles. If user intentions or emotions can be input as data, then a variety of new applications become possible. “It might be 1 pulse
Variation of blood stream
Hemoglobin absorbs the light Green LED
Photo Detector
What we can determine from the pulse wave
Light Attenuation （absorption magnitude）
Heart expansion/ contraction
Hemoglobin Blood vessel
Attenuation by blood in artery
Pulse wave
Variation
Attenuation by blood in vein Constant Attenuation by human tissue Time
Fig.5 Measuring stress via fingertip blood flow Pulse rate is measured using the reflected green light.（Source: Rohm ）
14
NE Handbook
Act
Enhanced human function
uat
ion
Five senses
Sensing
Expanded sensing function
Fig.6 Expanding human senses
Sensors are used to expand the range of human senses by feeding information back in a way that can be sensed by human beings. This information could be used to actuate human function, such as by direct muscle stimulation.（Source: Rohm ）
possible to make a game, for example, that reads user intentions and develops a strategy to counter them,” he adds. Rohm is now developing customers for the new devices, preparing to launch business operations. In the past, Rohm’s customers have almost always been electronic equipment manufacturers, but this time they have expanded their scope to cover manufacturers of medical electronics, and health service providers. Instead of offering independent units, they are selling solutions. One proposal is a complete system, now being offered to long-term health care providers. The R&D Headquarters has its own sales force to introduce new solutions to service providers.
15
［Applications / Vehicles］
Supporting ADAS with diverse sensors The acronym ADAS（Advanced Driving Assistance System ）refers to the entire array of systems such as automatic braking used in a “collision-free” car, and increasingly stringent regulations are accelerating ADAS adaption. The European safety standard for new cars, EuroNCAP, plans to include automatic braking and lane veer warning systems as evaluation items from 2014, and an automatic braking system with a pedestrian sensor from 2016. Automatic braking functions will become mandatory in Japan in large buses and trucks first, and the scope of coverage expanded gradually into other classes, from November 2014. There is a variety of vehicular obstacle sensors, including cameras, infrared sensors, millimeter wave radar, laser radar, and ultrasound（Table1 ）. At present, the most common design combines a single-lens camera with millimeter wave radar, complementing the low-cost camera with radar Table1 Major sensors used in autonomous driving（by NikPerformance Range
Infrared sensor
To about 40 m
○
○
To about 150 m
○
○
Distance resolution（Under 30 cm）
×
×
Angle detection
○
○
Relative speed detection
×
×
Environmental resistance
Rain, snow, fog
×
○
High temperature
○
×
Low light, no light
×
Remarks
16
Camera
Inexpensive singlelens cameras; high-performance stereocameras
○
Detects heat of people and animals even at night
NE Handbook
to provide good recognition precision even in rain, fog or other inclement weather, and at night. Low-cost cars such as compacts are more commonly being equipped with infrared laser radar. Manufacturers are extremely interested in sensor fusion, integrated control of multiple recognition sensors, as a necessary tool for realizing self-driving vehicles. Cameras are generally expected to assume the primary role. “Nissan is really forging ahead,” became a common comment among ADAS engineers after September 2013, which was the date that Nissan Motor Co., Ltd. of Japan announced the new X-TRAIL at the 65th Frankfurt Motor Show（Internationale Automobil-Ausstellung, or IAA ）. The vehicle implements automatic braking with one single-lens camera, utilizing it to measure distance to obstacles in front of the vehicle. Distance is usually measured with millimeter wave radar or stereocameras, and a number of people in the industry have expressed apprehension concerning the precision available from a single-lens camera.
kei Electronics based on material courtesy Panasonic Corp. ） Millimeter wave radar （76GHz）
Millimeter wave radar （79GHz）
Millimeter wave radar （24GHz）
Laser radar
Ultrasound sensor
○
○
△
○
×
○
△
×
○
×
×
○
○
○
×
×
○
○
×
△
○
○
○
×
×
○
○
○
×
×
○
○
○
○
○
○
○
○
○
○
Good for 3D environmental sensing at short and medium ranges
Low-cost solution already used in many vehicles
Cost is dropping, but usable waveband is narrow
Wider waveband In limited use in Europe from 2015 will improve dete c- and Japan tion precision
17
Applications / Vehicles
Automatic driving with a single-lens camera The single-lens camera making it possible was developed by Mobileye of Israel. Other functions provided by Mobileye include warnings of other vehicles and pedestrians, and warnings of lane veering based on lane line sensing. In the new X-TRAIL, the firm has demonstrated that a single-lens camera can be used to implement automatic braking. The company has already prototyped a test car with automatic driving based on one single-lens camera, in a move calculated to demonstrate the performance of the approach. The test car has been trialed on public roads in Israel extensively since 2012, and Mobileye is now developing a system utilizing multiple single-lens cameras, slated for completion in about summer 2014. There is no denying that some automobile manufacturers are uneasy about using only one single-lens camera. In response, the company explains it is working on “using multiple single-lens cameras to improve recognition precision and provide redundancy.” Concretely, the already-developed single-lens camera with a 52°angle of vision will be used for intermediate distances, and an additional camera with a 26°angle of vision will handle distances over 200m.
Succession of stereocameras
Stereocameras deliver better precision than single-lens cameras for obstacle recognition and distance estimation, but are more difficult to get full performance from. The leader in the field at present is Hitachi Automotive Systems, Ltd. of Japan, which makes the EyeSight system used by Fuji Heavy Industries Ltd. of Japan. The competition in stereocameras is set to intensify rapidly from 2014 through 2015,
18
NE Handbook
as companies like Robert Bosch GmbH of Germany and Continental AG of Germany release new stereocamera products. Bosch has developed a design with only 12cm between the two cameras, trying to facilitate mounting on smaller frames. The product from Hitachi Automotive Systems measures 35cm between cameras. Prices are dropping rapidly for the millimeter wave radar used in conjunction with cameras, too. A millimeter wave radar system could cost more than 400,000 yen in 2000, but the price is likely to drop under 10,000 yen in 2015. The biggest change is the material used in high-frequency circuits: while GaAs semiconductors were most common in the past, SiGe is becoming increasingly popular. In 2015, even less expensive Si will enter widespread use, and because the major foundries handle volume production under consignment, prices will plunge. In addition to the material, the wider millimeter wave bandwidth also plays a part. Millimeter wave radar utilizes the 76 to 77GHz waveband worldwide, but it has proven difficult to achieve high resolution in pedestrian sensing with a bandwidth of only 0.5 to 1.0GHz. The International Telecommunication Union Radiocommunications Sector（ITU-R ）is expected to allow use of the band from 76 to 81GHz by vehicular radar in 2015. It should be possible to effectively detect pedestrians with millimeter wave radar with a bandwidth of more than 1GHz. 3D laser radar is also attracting considerable attention in the industry. While primarily utilized in military applications, development for vehicular use is accelerating. Bosch, for example, is said to be developing a “third sensor needed for automatic driving,” which probably refers to 3D laser radar.
19
［Applications / Civil engineering］
Sensor-driven infrastructure maintenance There are innumerable bridges and roads in Japan built thirty to fifty years ago that are rapidly approaching the end of their service lives. Simultaneously, though, the engineers and technicians responsible for maintaining and repairing the infrastructure are also aging, and with the drop in the population overall, maintaining the infrastructure is becoming a major social issue. One possible solution being considered is to automatically measure physical change with sensor networks, thereby monitoring trends. By monitoring the strain on a bridge during an earthquake, or the displacement of its various components, it is possible to check how much has changed from normal state. Continuous monitoring of loading, and calculation of cumulative load, is handy for estimating when maintenance and repair will be needed, and invaluable in rapidly aging bridges. It is impossible to automate maintenance, but monitoring improves human work efficiency and speeds up decision making. Given the increasing scarcity of people to actually do the work, it has become indispensible in infrastructure maintenance. Examples of sensors being used for monitoring are becoming common, with the Tokyo Gate Bridge one of the most well known in Japan. Sensors have been installed into a few other sites as well, such as the Shuto Expressway in Tokyo, and its Yokohane Line branch. The BRIMOS bridge monitoring solution from NTT Data Corp. of Japan was designed for this type of application, and was recently installed on the longest bridge in Southeast Asia, the Cân Thó Bridge
20
NE Handbook
in Vietnam that collapsed once in 2007. But how is sensor-based infrastructure monitoring actually implemented, and what can the acquired data actually be used for? While the specific monitoring sites and items will vary with the type of infrastructure, structure, and materials, the Tokyo Gate Bridge provides a good example. It is a steel bridge, and so strain effects are quite from those in concrete bridges, and the methods of measurement differ as well. Sensors can be installed for continuous data logging, or sited specifically to acquire periodically, such as at weekly intervals. In both cases data is collected, analyzed later, and used to make informed decisions on maintenance and repair.
Status monitoring for main bridge section The Tokyo Gate Bridge measures 2.6km in total length, and sensors have been installed on the 800-meter long main
Weight of passing traffic
Temperature change and bridge elongation/shrink
Bridge safety assessment based on displacement and acceleration sensors
Monitoring range is center section（about 800m）
Fig.1 The Tokyo Gate Bridge is monitored for three major objectives
The bridge was designed to meet height restrictions imposed by Haneda Airport, and width specifications imposed by the needs of passing ships, and as a result the center section is a single steel sheet. Displacement is monitored primarily at the junctures between the center section and bridge supports.（Source: Bureau of Port and Harbor, Tokyo Metropolitan Government ）
21
Applications / Civil engineering
bridge section in the center to monitor strain, elongation/ shrinkage, motion of joints linking the main section to the adjoining sections, and other items（Fig.1 ）. Total weight of passing traffic is measured using the weigh-in-motion technique, and data used to calculate bridge roadway loading. And while not monitoring the bridge, meteorological sensors have also been installed for wind speed and other items. There are three primary objectives in monitoring, namely rapid determination of whether the bridge is safe or not after an earthquake, daily status monitoring, and collection of data for maintenance and repair planning（Table1 ）. The main bridge section was selected for monitoring because of the construction parameters for the bridge overall, and the structure of that section. The Tokyo Gate Bridge is a part of the Tokyo Bay Waterfront Expressway, which connects Wakasu in Koto Ward with the manmade land outside the Chuo breakwater. It had to be built wide enough and Table1 Outline of Tokyo Gate Bridge monitoring Monitoring objective
Monitored items
Description
Safety after Displacement in non-visi- Monitor force exerted on dampers earthquake ble locations, damage on top of the bridge supports, and the stays connecting the bridge members to the supports to determine if the bridge has suffered any displacement or damage. Bridge safety assessment can be comple t e d thir t y minu t e s a f t e r the earthquake.
22
Daily management
Temperature change inside bridge members and resulting bridge member change（elongat i o n d e f o r m a t i o n）f o r comparison
Bridge members elongate/shrink in synch with temperature change. Memb er displa c emen t is me asured and compared to temperature, and if they are out of synch it indicates unexpected strain.
Preventive maintenance
Weighing passing traffic with weigh-in-motion technique, by measuring deformation.
Bridge members degrade under cumulative loading (weight), but the concrete correlation is unknown. As a result, maintenance priority is assigned to sites with higher loading.
NE Handbook
Accelerometer（elongation/ shrinkage collision）
Accelerometer （support damage）
Accelerometer（stay damage）
Accelerometer（regulator damage）
Fig.2 Sensors, including accelerometers using optical fiber sensors, detect bridge component displacement
high enough that it would not interfere with ships using Tokyo Port. In addition, because it is quite close to Haneda Airport, it had a maximum height restriction. After considering design, strength, and cost, the center section was constructed as a single steel sheet. Displacement of the center section, therefore, was monitored at points where it connected with other structures such as bridge supports or adjacent bridge sections.
Detecting positional displacement with fiber sensors Concretely, monitoring for assessing safety after earthquakes consists of measuring change in components susceptible to earthquake damage, such as the dampers on top of the bridge supports, and the stays connecting the bridge to the supports（Fig.2 ）. Accelerometers using fiber sensors, and other sensors, measure displacement from the original position, and applied force, providing data useful in assessing component structural safety. It is possible to make the final decision on bridge safety within thirty minutes after an
23
Applications / Civil engineering
earthquake. Fiber sensors detect external change through the effect of vibration on light passing through the fiber. While electrically operated sensors only last for a few years, fiber sensors are said to have service lives of almost twenty. And since it is sufficient to merely pass light through the fiber, the power sources can be located at the ends of the fiber, eliminating the need for a power source in the middle of a bridge span, for example. For regular bridge management, deformation and temperature are measured in the center section, and the correlation checked. The bridge elongates and shrinks slightly, in close synchronization with temperature change（Photo1 ）, and if they are out of synch it is clear that there is a problem somewhere, such as unexpected strain. Monitoring makes it possible to identify and fix these problems early. For maintenance, sensors measure the weight on the steel roadway plates, which transfer traffic weight to the bridge members and supports — in other words, the weight of passing traffic. Weight is measured using the weigh-in-mo-
Photo1 Graph of deformation and temperature in center bridge section
If they are out of synch it suggests unexpected loading.
24
NE Handbook
Passing lane
400
800
July 20（Fri.）
July 21（Sat.）
July 19（Thur.）
July 17（Tue.）
400
July 21（Sat.）
0
July 20（Fri.）
200
July 19（Thur.）
July 21（Sat.）
July 20（Fri.）
July 19（Thur.）
July 17（Tue.）
July 18（Wed.）
July 15（Sun.）
0
July 16（Mon.）
200
600
July 17（Tue.）
400
■ 50∼59t vehicles ■ 40∼49t vehicles ■ 30∼39t vehicles ■ 20∼29t vehicles
July 18（Wed.）
600
Number of vehicles of 20 tons or more
1000
800
July 18（Wed.）
0
July 15（Sun.）
200
July 16（Mon.）
July 20（Fri.）
July 21（Sat.）
July 19（Thur.）
July 17（Tue.）
July 18（Wed.）
0
July 15（Sun.）
200
600
July 15（Sun.）
400
800
July 16（Mon.）
600
Number of vehicles of 20 tons or more
800
July 16（Mon.）
Number of vehicles of 20 tons or more
1000
1000
Number of vehicles of 20 tons or more
Harbor outside（from Wakasu）
Harbor inside（toward Wakasu）
Travel lane 1000
Fig.3 Measured weight of passing traffic
The graph shows the number of vehicles of 20 tons or more each day, for the week starting on July 15, 2012. Traffic on July 16（Monday ）is as low as on the weekend because it was a national holiday. Note that measurements will count multiple vehicles in close proximity to each other as a single vehicle.
tion technique（Fig.3 ）, which determines weight from measured deformation. The bridge suffers more damage under heavier loading, so cumulative loading is tracked as another parameter in maintenance. There are no established guides for how much damage is caused by how much loading, however, and for the time being the system is merely determining which sites experience the heaviest loading, flagging them for maintenance priority. Since the Tokyo Gate Bridge feeds into Tokyo, it is heavily traveled by large trucks, and so heavy loads are less frequent in the passing lane.
25
［Applications / Architecture ］
Information technology saves energy in even small buildings Building Energy Management Systems（BEMS ）easily adaptable to even small-scale buildings are attracting considerable attention in North America. These systems deliver the biggest improvements in large-scale structures, by controlling building power to boost energy efficiency, and that’s where they are being installed. It is more difficult to justify the investment into the required information technology（IT） in smaller buildings, however, and as a result the installation ratio is much lower. The BEMS offered by REGEN Energy Inc. of Canada, on the other hand, can be easily added to existing buildings by merely installing controllers, and it is rapidly gaining popularity in the small-scale building sector.
Fig.1 The controller installed on individual pieces of equipment （Source: REGEN Energy ）
26
NE Handbook
Installation complete in a day or two REGEN Energy’s BEMS is of extremely simple design, consisting of only the controller（Fig.1 ）. Each piece of equipment that needs power control, such as air conditioner or pump, is equipped with a controller. The controllers exchange data and control the equipment to optimize overall building energy efficiency. The system does not require concentrators or other equipment to centrally manage information. Each controller works in unison with all the others, sharing information almost like a swarm of bees working together. And that’s what the company calls it: swarm logic, or swarm energy management （Fig.2 ）. The controllers have internal ZigBee wireless transceiver
Fig.2 Swarm logic, where controllers work cooperatively to boost energy efficiency （Source: REGEN Energy ）
27
Applications / Architecture
modules, capable of communicating with each other over a range of 800m. A single controller can be installed in 20 to 40minutes, which means a whole building can be done in a day or two, unless it’s unduly large. Given that simplicity, 95% of customers are retrofitting existing buildings. There has been no simple solution for existing structures, and many owners are buying now to fill the need. The current controller has a response time of two minutes, which is more than adequate for demand response（DR; the system providing incentives to reduce consumption, or imposing penalties on excess consumption, during peak loads ） . The newer model released in September 2013 cut this to only one second. It handles high-speed DR and makes it possible to respond to price fluctuations in the electricity market in realtime.
Investment recovery in only 1〜2 years with DR participation
Judging from installations in the US, REGEN Energy’s BEMS cuts peak power by 15% to 30% on average. Peak cut directly translates into a lower base rate from the electric power utility, and if customers participate in DR they may also receive incentives. The controllers support AutoDR, so users don’t have to take any other action at all: when a controller receives a DR signal it simply implements control as programed. Because the system can be set up just by installing the controllers, says president and CEO Tim Angus, “The installation cost is about a third that of conventional BEMS.” In most cases, that means the investment can be recovered in three or four years, and if cash incentives from the electric utility for demand response are included, it can pay off in
28
NE Handbook
only a year or two.
California launches AutoDR
The three major electric utilities in California, US — Southern California Edison Corp.（SCE ）, Pacific Gas and Electric Co.（PG&E）, and San Diego Gas and Electric Co.（SDG &E ）— have already begun AutoDR service. In 2012, they announced that they would comply with the Open ADR 2.0 standard, and REGEN Energy BEMS sales are booming as a result. Some 80% of the firm’s customers are in California. REGEN Energy was established in 2005, with the first controller prototype completed in 2007, and trial installations operating from 2008 to 2010. They began full scale sales in 2010.
Business opportunities in Japanese small-scale buildings
“We have great hopes for the Japan market, where BEMS are not very common in small-scale buildings,” comments Angus. Electric power supplies are tight and there is strong demand for building energy management. Japan is especially lagging when it comes to retrofitting older buildings, and there would seem to be plenty of opportunity for REGEN Energy’s BEMS.
29
［Applications / Agriculture］
Visualizing agricultural product status Both farmers and the Information and Communications Technology（ICT ）industry are eagerly promoting the adoption of ICT in agriculture. Farmers want to boost their international competitiveness now that Japan is involved in Trans-Pacific Strategic Economic Partnership Agreement
（TPP ）negotiations, and the ICT industry is looking for new markets. The government is pushing too, as part of its “Abenomics” growth strategy. Smart agriculture is attracting interest from all over. It boosts productivity by utilizing ICT in the form of big data, and sensor networks to monitor agricultural growth processes（Fig.1 ）.The scope of application is no longer limited to closed, managed plant factories, but spreads to include open fields, greenhouses, and more. Wireless sensor networks offer low prices and high reliability, and now that ICT is being tapped for uses in sale and procurement as well, it offers a way for agriculture to improve revenue and profitability and establish itself as a new industry stretching from agricultural production to the consumer. Major Japanese ICT companies such as Fujitsu Ltd. and NEC Corp. are already active in the smart agro field, and multiple device manufacturers are working on sensors to measure field nutrients, for example. For the ICT and electronics industries, this is one of the few markets they are still not dominant in.
Support from agricultural and communications ministries Smart agro is not being left to private enterprise, how-
30
NE Handbook
ever. The government has developed a number of strategies designed to promote the growth of farmers and the ICT industry. The Abe government has positioned the national agriculture, forestry, and fisheries industries as growth industries. For example, agricultural and fisheries production value has been trending downwards from the 1990s, but the government wants to boost it toward 2020. Under the plan announced by the Ministry of Agriculture, Forestry and Fisheries（MAFF ） , the 11trillion yen production value for agriculture and fisheries will be increased to 14trillion by 2020. Exports will also be increased, with rice and its processed products, for example, to jump from 13billion yen in 2012 to 60billion in 2020.
Fig.1 Example of agricultural sensor network product（Manufactured by Fujitsu. ）
31
Applications / Agriculture
The Ministry of Internal Affairs and Communications （MIC ）,meanwhile, has positioned agricultural, forestry, and fisheries products（foodstuffs ）as “resources indispensible for living,” along with ores, energy, water, and the infrastructure, and intends to apply ICT to ensure reliable, continuing supplies. The use of sensor networks and big data will first turn agriculture into a knowledge industry, and eventually support the creation of an information network capable of supplying value-added foodstuffs. Agricultural, forestry, and fisheries data will be used to boost productivity and establish a sales network to achieve higher sales prices, making it easier to turn a profit. MIC is also eager to create new demand for ICT products and services.
Just collecting data has no value Smart agro offers considerable merit for a number of parties, but there are several problems that will have to be resolved first. Director, vice president, and founder Hiroshi Shimamura of eLAB experience company of Japan, a consultant in sensor network application systems, presented a talk on connecting and visualizing agricultural crops at a nextgen sensor symposium on the potentials and problems of smart agro on September 25, 2013, organized by the Japan Society of Next Generation Sensor Technology. This firm markets agricultural sensor network systems. Farmers can install smart agro systems fairly easily, but growing quality crops demands more than merely collecting data and leveraging big data: knowledge is needed to interpret the data, and algorithms to feed the data results back into growth processes. Shimamura points out this is not a trivial process. He cited an example of a farmer whose lettuce produc-
32
NE Handbook
tion was below expectation. An analysis of the cultivation history showed that insufficient water had been supplied immediately after planting. This type of analysis can be done by an expert in plant physiology, but is unlikely to be found in farmers even if they are skilled at growing lettuce. Experience at lettuce cultivation is not always useful when it comes to analyzing cultivation histories.
Noticing problems from raw data He cited another example as well, this time of a farm with a reputation for tasty melons. The farmer would always put on swimwear when turning on the sprinklers in the greenhouse, so that he could feel the humidity on his skin. Sunlight and other factors can result in delicate humidity variations even within a single greenhouse, and he modified sprinkling appropriately to match. His decisions were based on intuition, and it was difficult to capture this intuition in growth algorithms. A variety of external support is essential. These problems represent business opportunities. In the former example, farmers utilizing plant physiology expertise can expect to produce lettuce superior to farmers relying on intuition and experience. If the abnormalities concealed in sensor data can be noticed and acted on by even “average” farmers lacking knowledge on plant physiology, it would have value. Shimamura explains that instead of merely helping farmers notice the problems, the service would gain value by offering teaching value as well. Many parties are interested in producing value from raw sensor data: the computer companies selling big data, the device companies handling the raw sensor data, the experienced farmers, and the corporations that understand all of their needs.
33
［New trends / Trillion Sensors］
A trillion sensors create new businesses An American entrepreneur has called for the establishment of the Trillion Sensors Universe, utilizing a trillion or more sensors a year, and his vision is developing into an international initiative that is attracting industry and research institutions from Japan, America, and Europe. Today about ten billion sensors are used around the world every year, and a trillion would be one hundred times
greater（Fig.1 ）. The proposal covers such detailed applications as putting sensors on each every pill, and verifying that the patient takes it as prescribed. If realized, the Trillion Sensors Universe would produce data on everything, make it possible to analyze that data, and create a safer, more efficient society than ever before.
From Silicon Valley to the World The proposal was made by American entrepreneur Janusz Bryzek, who resides in Silicon Valley. He has launched seven start-ups already, and raised them successfully to commercial concerns. They grew sufficiently to evolve into business divisions of companies like General Electric Co., Intel Corp., and Maxim Integrated Products Inc. He began conceptualizing the Trillion Sensor Universe idea in about 2012, gradually developing it as he received feedback on his ideas. He reached the conclusion that sensor networks of huge scale throughout society could resolve a number of global-scale social problems including energy, medicine, and famine. In preparation, he has begun working with leading high-
34
NE Handbook
100trillion
Annual sensor shipments
10trillion 1 trillion mark
1trillion 100billion
2013: Sensors in pills for new growth
10billion 1billion
100million
2007
Until 2013: Sharp growth for smartphones 2012
2017
2022
2027
2032
2037
Year
Fig.1 The roadmap is only the first step
Activities toward the realization of the Trillion Sensors Universe, primarily by TSensors Summit. The target date has been set for 2023.
tech corporations and universities to make it possible to obtain diverse and inexpensive sensors that anyone can use easily. The roadmap has set the year 2023 as the target for realizing his vision. He began drawing up the roadmap in February 2014. TSensors Summit of the US, a tech start-up launched to support his activities, is working with interested parties, namely several dozen engineers, managers, and researchers from corporations, universities, and research institutions in Japan, Europe, and the US. The roadmap classifies sensors for the proposed trillion into about ten categories, and will standardize the manufacturing platform for each. In addition to sensors, the concept also encompasses peripheral technology such as wireless networks, energy harvesting devices, the Internet of Things （IoT ）, and the Internet of Everything（IoE ）. According to
35
New trends / Trillion Sensors
Bryzek, standardization will reduce the time needed to establish volume production technologies and significantly cut manufacturing cost.
Still only a concept, but a fascinating one “It’s just a concept with no means of implementation,” says one manager in the high-tech sector. “It’s just not possible without the full array of constituent technologies, including peripheral devices and systems, in addition to sensors,” comments management at an American tech start-up. There are many people in the industry who feel the Trillion Sensor Universe initiative is hopeless. At the same time, though, there are also many engineers and managers who actively support Bryzek’s work, because without it the era of a trillion sensors will never come to pass. Apparently, they are not moved so deeply by the boldness of the concept itself. In the information and communication technology（ICT ） world, in fact, it is not terribly unusual to find people who believe that sensor demand will surpass a trillion sensors. Instrument and device manufacturers like Hewlett-Packard Co., Intel Corp, Robert Bosch GmbH of Germany, and Texas Instruments Inc., among others. A number of people join Bryzek in believing that a wide range of sensor types can serve as “social devices” to help resolve social problems. Libelium Comunicaciones Distribuidas SL of Spain, involved in infrastructure sensor systems, cites a number of examples that could reach a trillion sensors. Bryzek is merely trying to minimize the time it takes society to reach the trillion sensor universe. Based on his experiences launching firms in the sensor field, he believes
36
NE Handbook
that it takes twenty years to create a new sensor business, and a fundamental problem is the “chicken or the egg” issue: which comes first, widespread adoption or low price? The roadmap was drawn up to accelerate the realization of the trillion sensor universe, and the idea is attracting more than just sensor manufacturers and users. The information technology industry is also intrigued, hoping to leverage the massive flows of information from those sensors in new businesses.
99% from brand-new sensors The idea of spreading massive quantities of sensors everywhere to collect data seems to be much the same as the proposal for Smart Dust made in the 1990s by the University of California, Berkley. And the Smart Dust concept is already partially implemented in the form of sensor networks for monitoring in farms, factories, and similar applications. The difference between the trillion sensor universe and existing sensor networks is the sheer quantity of sensor types, and of sensors. In a society using one hundred times more sensors than today, less than 1% of them would be existing types. The remaining 99% would be new applications that do not yet exist, or which exist but are impractical for reasons such as high cost. Many of the presentations at the TSensors Summit held in the US in October 2013, and the Trillion Sensors Summit Japan 2014 held in Tokyo in February 2014 discussed applications which have yet to achieve widespread adoption.
Pill sensors and throwaway image sensors The pill sensors mentioned above, used to detect whether a pill had been taken as per prescription, would react with
37
New trends / Trillion Sensors
the patient’s stomach acid and use chemical energy to emit an electrical signal, which would be picked up by a skin patch. The sensor itself would be the size of a grain of sand, and made of silicon. Mark Zdeblick, co-founder and chief
（a ）
（b ）
（c ）
38
NE Handbook
technical officer（CTO ）of developer Proteus Digital Health, Inc. of the US, says it would just be excreted once swallowed. One proposal suggests that existing image sensors could be made much cheaper by eliminating the optical lens and
（d ）
（e ）
Fig.2 Sensors on pills and underwear （a ）The sensor-equipped pills（inside the case to right ）and the skin patch.（b ）The Rambus image sensor can be used to make inexpensive, throwaway cameras that do not need lenses.（c ）A wireless sen-
sor that can be pasted on like a piece of tape, developed by the University of California, San Diego.（d ）A bra equipped with a pulse rate sensor, which automatically opens the front hooks only when the pulse rate exceeds a target value. Developed by Verygry Co., Ltd. of Japan. （e ）A helmet that senses when an athlete has received a dangerous impact, developed by MC10 Inc. of the US.（Source: Photos b, c, d, and e from corporate and university data ）
39
New trends / Trillion Sensors
making them disposable. A diffraction lattice would be formed on the surface of the complementary metal-oxide semiconductor（CMOS ）sensor so that it only detects light from one direction. Distortion would be handled in signal processing. It would find use, for example, in smart card facial recognition. There is also a proposal for biosensors created in sheets that could just be taped to the body. Once prices drop sufficiently for throw-away use, applications would spread rapidly.
Opportunities for new information businesses “One possible business model would be to give away shoes fitted with sensors, and make a profit off of the sensor information,” explains Bryzek. The appearance of the trillion sensor universe would accelerate this type of business in an era where massive amounts of sensor information are collected and widely utilized. In the trillion sensor universe, sensor data will mean added value for service industries, because as it becomes easier to utilize this data, it will also be easier to combine it with other data and produce information that consumers want. For example, the interpretation of the output from the temperature sensors used as air conditioning thermostats could vary depending on whether they were installed near the window or in the hall, and whether they were exposed to sunlight or not. If sunlight is coming in the window, that sensor would be warmer than the one in the hall. By combining output from multiple sensors, and other related data, it would be possible to generate information that has value to the user. Sensor information can also be stored, and user trends
40
NE Handbook
analyzed from the log to produce useful information in a variety of promising businesses. There is no value in the sensor-generated location data for a user, for example, but if information analysis indicates that the user likes coffee from a particular coffee shop chain, it becomes useful marketing information. In other words, the combination of large sets of data produces value. The medical field as well needs sensor data. Young, healthy individuals will eventually grow old and become patients. If biomedical data is sensed and stored from youth, it becomes possible to analyze trends that might lead to illness, and make reliable predictions. This would be very valuable information, and “could be used very effectively by the government and medical institutions,” explains Junichi Maekawa, of InfoCom Research, Inc. of Japan. A number of major electronics manufacturers are already researching possible medical and healthcare information services. They all use a wide range of sensors and analyze big data, as well as assuming the realization of IoT and IoE.
Accumulating annual demand for a billion sensors, for a decade
The TSensors Roadmap, which spells out the strategy for the implementation of the trillion sensor universe, is aimed at sensors which can be expected to show demand of a billion sensors a year, for ten years. Janusz Bryzek explains that the total will exceed a trillion by 2023. The roadmap does not include sensors which are likely to reach that target on their own, such as the microphones and acceleration sensors in smartphones. First Bryzek’s organization will group multiple types of sensors into specific TApp platforms broken down by sensed
41
New trends / Trillion Sensors
data type, defining parameters for each. He selected chairs by February 2014 for about ten TApps from a list of applicants, and the project descriptions have been updated often since. The TApps include non-invasive health monitoring, artificial senses, environmental sensing, infrastructure sensing, and sensing for the food industry. The plan calls for individual chairs to document their roadmaps before the end of 2014, with the final specification spelling out the entire project to be issued in June 2015. The next step will be for TSensors Summit to provide services designed to accelerate industry related to the trillion sensors, such as assisting new sensor-related tech startups. It also plans to begin lobbying the US government for financial assistance, claiming that the manufacturing industry will mean new jobs.
Sensors for under US$0.13 The key items that have to be defined for each TApp are the target price and the optimal manufacturing method, the objective being to slash manufacturing cost through standardization. Given current trends in sensor market scale, Bryzek expects that the target price will not exceed “0.1% of gross domestic product.” World GDP in 2023 is estimated to be 130trillion US dollars, which means, he explains, that the limit on sensor unit cost is US$0.13. Similar economic forecasts indicate a per-unit upper limit on sensor note systems, including networks and control circuits, of under US$1.00. Applications are not specified in either estimate. A look back at the way sensors have spread in smartphones makes the relationship between unit price reduction and production volume increase very clear. Bryzek believes
42
NE Handbook
that shipping such a large quantity of devices will significantly slash unit cost, and has reflected that belief into the roadmap.
43
［New trends / IoT, IoE, and Big Data］
“Things” generating massive amounts of data The Internet of Things（IoT ）is the network created by all sorts of things connected via the Internet, specifically everything other than personal computers, smartphones, and such information technology（IT ）products. Many of the televisions and gaming systems sold recently already communicate via the Internet, and now it is becoming possible for white goods, sensors, and all kinds of equipment and components to do the same. Internet connection makes it possible for computers and other equipment to check on status or the environment, or even operate other things or affect the environment remotely. Many sensor networks are based on the Internet, and are therefore included in the IoT.
IoE and M2M are in the Internet of Things, too The term “Internet of Everything”（IoE ）is used in the same context as the IoT, while machine-to-machine（M2M ） is a communication system linking two machines to each other, or a machine to a computer. Communication that is not implemented through the Internet, such as 1:1 communication that does not use an Internet Protocol（IP ）address is also called M2M.
More precise analysis of big data IoT is assuming increasing importance now that a variety of data related to things is appearing on the Internet, leading to increased demand for communications and data-crunching power, the expectation that the number of
44
NE Handbook
Estimated Internet-connected devices worldwide 20000 Devices worldwide（unit: 1000）
18000
BI INTELLIGENCE
16000 14000
Wearables Smart TVs
12000 10000
Internet of things
8000 6000 4000 2000 0
Tablets Smartphones Personal Computers （Desktop And Notebook）
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013E 2014E 2015E 2016E 2017E 2018E
Fig.1 The Internet of Things continues to grow （Source: IDC data modified by Intel ）
networked things will surpass the number of smartphones in the near future, the potential to increase the precision of big data analysis, and the birth of new services based on IoT data or analyzed data（Fig.1 ）. This development could bring a host of changes in efficiency, safety, and security to diverse industries including medicine, the environment, transportation and distribution, industry, architecture, primary industry, and disaster response. The impact that IoT might have is actually far greater than merely an Internet connected to a variety of things. Intel Corp. of the US, for example, has defined it as a revolution on a global scale, with innumerable devices seamlessly connected and managed intelligently and safely on the network, providing valuable services to people, devices, and systems by instantly converting data into useful information.
45
［New trends / Energy harvesting］
From low-power to no-power
Energy harvesting（also known as energy scavenging ） offers electricity generation in a variety of places. The problem is that the available power is very small. The majority of the systems on the market only generate on the μW order per-unit, which is far from enough to run, for example, a smartphone. But even though the generated energy is small, the market is being eyed eagerly by industry because of the convenience it offers. The biggest sales point of energy harvesting is that a system utilizing it won’t require battery replacements, wiring, or maintenance… in other words, it represents a switch from low-power design to no-power design（Fig.1 ）. The technology is beginning to turn up here and there as engineers search for ways to tap into this convenience. The Hoki Museum in the city of Chiba, Japan, for example, has installed an audio guidance system with switches powered by energy harvesting. Control signals are handled via wireless, eliminating the need for new switch wiring, too. For art museums, where exhibits are changed frequently, power cables and other wiring have been a longrunning problem. Energy harvesting is cable-free.
Changing peripheral components The key elements of a representative energy harvesting system are（1 ）detect an energy source and generate electricity,（2 ）convert the harvested energy with a power supply circuit, and store it to capacitors or rechargeable batteries（3 ）use the stored energy to drive a microcontroller,
46
NE Handbook
Fig.1 Examples of battery-free switches
sensors, etc., and（4 ）transmit the sensed information via wireless to an external system. The idea of energy harvesting is not new, and it has been a research theme for many years. It is being implemented more of late because of evolution in the peripheral components needed to take full advantage of the generating devices （functions 2 through 4 above ） . In addition, more and more applications are being developed which can utilize them. These evolved peripherals include power circuits capable of efficiently utilizing the generated power, wireless integrated circuit（IC ）transceivers, microcontrollers, sensors, and other components operating with extremely low power consumption. Until now, even though the energy could be harvested, it was consumed driving the peripherals with none left to implement the target function. Now that ICs are available offering high efficiency and low dissipation, energy harvesting has finally entered the stage of practical utilization.
Low-power wireless transceiver ICs Higher performance of the power circuit has an especially
47
New trends / Energy harvesting
great impact, because the power circuit determines defines how much of the generated power can actually be used, and how much is wasted on loss. Power circuits now can efficiently recover even extremely low voltages. Low-power wireless transceiver ICs have an equal effect, and one of the most important firms in the business when it comes to energy harvesting is tech start-up EnOcean GmbH of Germany. The EnOcean wireless standard designed for communication between devices runs on no more than a tenth the power of competing systems. As Frank Schmidt, chief technical officer（CTO ）of the firm explains, “Control is exceedingly simple.” The wireless IC for the above-mentioned switch and similar applications implements on/off control by sending three signals, each about 1ms in length, over a 30-ms period. The sensors used to measure temperature, humidity, and the like also operate on lower power. The brightness sensors used for lighting control, thanks to development for mobile phones, are also running on less and less power, as are the microcontrollers managing the energy harvesting circuit, sensor drive, and other operations. For microcontrollers, the key point is to keep dissipation low in standby, but with a sharp rise for operation. This type of intermittent operation is extremely common in wireless sensor network applications, so standby current drain is crucial. R&D on generators is under way as well. The Georgia Institute of Technology of the US and others developed a technology for generating power from vibration, with a high output density of 31.3mW/cm2.
Wireless sensor networks in action Progress in low-power, high-efficiency peripheral circuits
48
NE Handbook
and generators has finally made energy harvesting a practical technology, and applications are waiting. There is plenty of room in addition to light switches, especially for wireless sensor networks. Sensors only need to send data a few times an hour, so power consumption is low, and systems can operate on very low levels of generated power. The scope of application for battery-free, wireless systems is growing rapidly. One example is a temperature sensor network in remote forests. It is capable of detecting forest fires, and would help minimize damage. Given the potential damage a fire could cause, and the money spent on people and materials to control a fire, a wireless sensor network would more than pay for its own installation and maintenance. Health monitoring of infrastructures is another application rising rapidly in prominence. Structures such as buildings or bridges are equipped with sensors, and changes in status monitored, making it possible to detect sudden changes or estimate service life. The collected data is invaluable in maintenance and parts replacement. Demand is also high in motors and engines, for example. Automobiles are webbed with wire harnesses needed to drive the sensors that monitor component status, and if the thermal or vibrational energy of the engine or motor could be used to power cable-free sensors, the number of required harnesses would drop dramatically. Life recorders would also benefit from the technology. Mounting sensors on livestock or even wild animals would not only provide location information, but also temperature, pulse rate, and other data. And if the power source were the heat energy of the animal, even battery replacement could be eliminated.
49
［New trends / Wireless networks］
Diverse techniques for each application A brief outline of major standards for short-haul wireless communication technologies. Different standards cover different wavebands, communication speeds, and ranges, as well as a variety of topologies for terminal-hub connection, for example. Standards such as Wi-Fi, Bluetooth, and ZigBee are well-known in Japan, but there are many more, such as the ANT/ANT+ standard being utilized in sporting applications primarily in North America, and the EnOcean standard used in wireless switches for lighting and air conditioning.
Table 1 Network topology （ table by Nikkei Electronics based on material courtesy LAPIS Name
Standard
Communi- Frequency Speed （MHz） （bit/s） cation protocol
Wi-Fi
IEEE 802.11
Wi-Fi Alliance
5600、 5200、 2400
100 300M、 54M、11M
Bluetooth
IEEE 802.15.1
Bluetooth SIG
2400
24 M 、3 M 、20 1M
Bluetooth Low Energy
IEEE 802.15.1
Bluetooth SIG
2400
1M
20
None
ANT+
2400
1M
20
IEEE 802.15.4
ZigBee Alliance
2400、 250k 902 ∼ 928、 868 ∼ 870
50
ZigBee IEEE Green Power 802.15.4
ZigBee Alliance
2400
250k
50
Special low-power wireless
IEEE 802.15.4
None
150 ∼ 950
100k
700
None
Z-Wave
779 ∼ 956
100k、40k、30 9.6k
Wireless HART
IEEE 802.15.4
HART Alliance
2400
250k
EnOcean
ISO/IEC 14543-3-10
EnOcean Alliance
315、868、 125k 902、928.35
ANT/ANT+ ZigBee
Z-Wave
50
（exclusive） Alliance
（exclusive） Alliance
Range （m）
50 100
NE Handbook
star
P2P
device
device
device
hub device
device device
tree
mesh hub
hub
device
device
hub
device
device
device
device
device
device
device
device
device
device
device
Fig.1 Network topology
Semiconductor ） （? ）indicates uncertainty Transmission/ Transmission power reception current（mA） （mW）
Topology
Characteristics
300
30
P2P, star
General applicability, only streaming protocol
35
2.5、1、100 P2P, star
Affinity with mobile phones, can transmit music
15
1（10）
P2P, star
Low-power version of Bluetooth
15
1
P2P, star, tree, mesh
For fitness and sports, primarily in North America
20
1
P2P, star, tree, mesh
Sensor networks
20（？）
1
P2P, star, tree（?）, mesh（?）
Low-power version of ZigBee, for harvest communication
25
20、1
P2P, star, tree, mesh
General applicability, individual national standards, increasing deregulation
30
1
mesh
Home networks
20
1
mesh, star, mesh+star
Specialized for industrial use
25
1
star
Specialized for harvest communication
51
［New trends / Infrared sensors］
Applications expanding from automotive sector The range of applications for infrared sensing is growing rapidly from the original thermography uses. It is emerging as a key technology in safety, with applications in identifying biologically contaminated meat cuts, identifying pills of the same color and shape, diagnosing cancerous cells and vein positions from outside the body, detecting people in healthcare and security uses, sensing alcohol in vehicle operator exhalations, as night vision systems for driving, detecting gas leaks, locating loose tiles on structure walls, and more. Automotive night vision systems, once limited to only high-end automobiles, are now spreading to a wide variety of vehicles, and special legal measures promoting their use are expected to come into effect in Europe in 2016-18. Parts manufacturers in the lens and semiconductor industries are getting involved, and the prices of cameras are ready to drop. Infrared light has a wavelength longer than that of visible light, but shorter than sub-millimeter wave radio. Characteristics vary with wavelength, and infrared is generally broken down into far, middle, and near infrared ranges. In sensing applications, the specific characteristics of each waveband are utilized. Applications for near and far infrared commonly detect infrared radiation that is reflected or emitted by the target, while middle infrared can detect which infrared wavelengths are absorbed. Near infrared has the shortest wavelength, and is utilized in household appliance remotes, and IrDA and other infrared communication, as well as in sensing applications including night vision systems and infrared cameras. Near infrared light has reflection charac-
52
NE Handbook
teristics similar to those of visible light, and can reach farther, while remaining invisible to the human eye and therefore unlikely to be noticed. The waveband is
Fig.1 Imagery from a far infrared image sensor（Source: Autoliv ）
from 0.7 to 2.5µm. The infrared light with the longest wavelength is far infrared, with a waveband of 4µm to 1mm. It is often used in thermography（temperature measurement ）. Objects generally emit electromagnetic radiation if they are above absolute zero, and the emitted wavelength will vary with the temperature of the object. The most commonly measured objects have temperatures between minus tens of degree to plus hundreds of degrees（Celsius）,representing an electromagnetic radiation wavelength peak from 3 to 20µm. Infrared wavelengths have a frequency of 300GHz at the lowest, which means they pass through cloth, paper and similar materials: a characteristic useful in determining the content of an envelope without opening it, for example. Automotive night vision systems also use infrared sensors. Autoliv Inc. of Sweden, which holds the overwhelming share of the market, uses a combination of near and far infrared sensors. Middle infrared has a wavelength between the other two, ranging from 2.5 to 4µm. It makes use of the fact that objects irradiated by electromagnetic radiation will absorb a specific wavelength, a phenomenon frequently used to identify materials. The absorption spectrum covered by middle infrared includes bases found in organic compounds（O-H, etc. ）, and absorption spectrum characteristics can be used to identify chemical reactions.
53
［New trends / Smart Meters］
Network control for household appliances, too Smart meters are systems incorporating multi-function watt-hour meters with equipment management functions, communication functions and more. By equipping watt-hour meters with short-haul wireless communication functions operable over ranges of a couple dozen to about one hundred meters, it becomes possible to connect them to facilities and equipment in the home or office such as air conditioners, lighting, thermostats and security systems. The operational status of this end-user equipment is managed by the electric power utility via the network and the smart meter. In other words, the smart meter plays a key role in the smart grid, helping balance demandside and supply-side system operational states to match the changing electric power supply-and-demand situation. The reason behind connecting the watt-hour meter to equipment in the home is to manage energy use by these home systems. The first step is to make the home energy use status visible, showing the user the current situation and promoting an interest in saving energy. In the future, the electric power utility may be able to control air conditioner thermostats via the network for fine-grain energy saving measures on a society-wide scale.
Solar panel Electric Route A power utility
Smart meter
Route B
Storage battery HEMS
Household appliances Electric vehicle
Fig.1 The smart meter network
54
NE Handbook
［New trends / MEMS sensors］
Proving performance in consumer electronics Many conventional sensors made use of material properties, such as the piezoelectric devices converting deformation into voltage. Microelectromechanical systems（MEMS ） sensors, however, are tiny machines made with semiconductor microfabrication technology. Tiny mechanisms, such as pendulums for example, react to external change, and that reaction is converted into a change in voltage or current. Compared to conventional sensors, MEMS sensors have few drawbacks, are easy to use, and boast excellent possibilities when it comes to future capabilities. MEMS sensors are made utilizing the precision film growth, photolith and etching processes developed for semiconductor device manufacture, which enable the construction of mechanisms on the micrometer order. And in general the smaller the geometry, the better the sensitivity and precision. It is also fairly simple to single-chip the sensor device with the electric circuitry needed to process the signals, shortening the wiring between the two to minimize external noise effects. It is also simple to combine multiple sensor devices with differing characteristics to detect a wider range of targets, or to make a sensor array to acquire distributed information. The use of MEMS sensors in consumer equipment is accelerating sharply. Acceleration sensors, for example, have shown up in the controller for the Wii game system from Nintendo, while gyro sensors help correct hand wobble in digital cameras and silicon microphones are common in notebook PCs. Next-generation applications such as gas sensors and biological sensors are in the laboratory now.
55
［New trends / CMOS sensors］
The key to image data acquisition Complementary metal-oxide semiconductor（CMOS ）sensors are imaging devices that convert the charges stored in photodiodes into voltage, on a per-pixel basis, amplify the voltages and read them out. Together with charge-coupled devices（CCD ）, CMOS sensors are the most commonly used types of solid-type imagers. Originally application was limited due to excessive noise, but the development of CMOS sensors accelerated abruptly for use in mobile phones because they can be designed for low power operation and miniaturized relatively easily. Advances in digital signal processing have led to improvements in noise compensation, eliminating that initial drawback. The difference between CMOS and CCD imagers in terms of structure is basically how the image signal is read. In both designs incident light is first converted to electricity by the photodiode, and the resulting stored charge is read as voltage, then amplified for output. The CMOS sensor converts and amplifies these charges on a per-pixel basis, while the CCD sensor transfers charges through a “bucket relay,” and processes them all in the output stage. The major problem with CMOS sensors is that because amplification is implemented on per-pixel basis, the amplifier threshold voltage varies, which can cause noise. Formerly CMOS sensors also exhibited high dark current, caused by lattice defects, but this has been improved significantly of late. New back-side illumination sensors have also been developed, moving the metallization from above the diodes to below them, boosting sensitivity.
56
NE Handbook series Sensor Networks Publisher
Published Editor Editorial assistance
Design Printing Sponsored by
Nikkei Business Publications, Inc. 1-17-3 Shirokane, Minato-ku, Tokyo 108+8646 Japan June, 2014 Nikkei Electronics Nikkei Technology Online, Nikkei BP Cleantech Institute Motoaki Ito（Enlight, Inc. ） Nikkei BP Consulting Dai Nippon Printing ROHM ©Nikkei Business Publications, Inc. 2014 All rights reserved. Printed in Japan
