Sensors are great things—essential technology—but they can only go so far on their own. It’s when they get paired with data collection and machine learning (and even AI) that powerful applications are born.
In that spirit of connection and invention, Berkeley Lab scientists have developed an intelligent sensor in spectroscopy that sniffs out and seeks target objects and thereby drastically increases the speed of digital processing, up to 100 times.
The achievement has advanced to the point that the Department of Energy’s Lawrence Berkeley National Lab is offering information about licensing the patent pending technology, as explained on a Berkeley website. The technology is deemed useful in applications for waste recycling, agriculture, quality control in food manufacturing, pharma, wafer development, smartphones, robotics and more.
As a refresher, spectral machine vision is performed by collecting spectral and spatial dependence of light data to find the chemical composition or microscopic structure of a material. It’s the analyzing of this dense data that can cause a data bottleneck, which can lead to tradeoffs in spatial/spectral information and frame rate and drain power efficiency. As such, Berkeley said it has developed a novel photodetector sensor capable of learning to perform machine learning analysis and readout by automatically learning from example objects to identify new samples.
The team is building devices in visible and mid-infrared bands to perform intelligent tasks. They said they offer 1,000 times lower power consumption and 100 times higher speed than existing approaches. In other words, a single photodetector would surpass the complex spectral recognition of recent neuromorphic photodetectors. The Berkeley sensor is used as a machine learning computer and performs machine learning computations on the incoming light, Berkeley scientists told HPC Wire.
Berkeley did what many industries have done with machine learning by training the machine vision model with many examples of spectral signatures, such as infrared patterns that come from an actual leaf versus an artificial one. Or a bird’s feathers as compared to a tree’s similarly colored bark.
In training, the researchers actually showed the sensor dozens of images of colorful birds in wooded settings, and the sensors sniffed out a random sample of pixels, either those labeled as “bird” or “background.” Then, an external computer sent an electrical signal to the sensor commanding it to “identify bird” or “identify background” and recorded the sensor’s response. Software then picked the best command combination for teaching the sensors to highlight the bird while suppressing other responses.
They tested the training by showing the sensor a new image and told it to find a bird using the commands developed in training. The sensor then gave positive output only for pixels that belonged to the bird. In essence, the sensors had learned from examples to identify target objects even when never they had never seen the objects before.
“The most exciting part is the concept of giving intelligence to sensors,” said Dehui Zhang, lead author on the study, in comments to HPC Wire. Conventional sensors only collect raw environmental data, and then leave the intelligent recognition work to digital processors. The team co-designed semiconductor materials, devices and algorithms and enabled the sensors to learn and compute without the need for digital post-processing of data.
Using the tech, the team also successfully identified oxide layer thicknesses in semiconductor samples to determine uniformity and was able to determine hydration levels in plant leaves and transparent chemicals in a petri dish.
The work was funded by the US Department of Energy’s Office of Basic Energy Sciences. While companies such as John Deere have developed object recognition technologies with machine learning to detect weeds in grain crops and have created spraying technologies that target only the weeds, the Berkeley work promises to open the intelligent sensor to a wide range of industries.