There is no denying that self-driving vehicles have grabbed their fair share of headlines in recent years. This in turn has thrown what is undoubtedly one of the most important technologies behind them, lidar, into the spotlight.
Most of the reports surrounding autonomous vehicles have included the caveat that they are still a long way into the future. It was widely understood that there were still some huge hurdles to jump in terms of technology and market requirements in order to reach level four or five autonomy, where minimal intervention from the driver is required.
The past few years have seen heavy investment in making that happen, with more than $1bn in corporate and private investment between 2017 and 2019 across 50 lidar start-ups. This includes a whopping $420m in 2018, according to a Reuters analysis of publicly available investment data. It has allowed the technology to move forward significantly, and it is now beginning to reach the required performance, size and price-point that the autonomous vehicle sector demands.
A more recent report from Forbes cited lidar’s enabling potential for autonomous vehicles as attracting upwards of $1.5bn from venture investments, with similar levels within corporations like OEMs, Tier 1 suppliers, and technology companies such as General Motors, Toyota, Bosch, Continental, Waymo, Argo and Aurora.
Little wonder
In the last couple of months, a number of announcements have further demonstrated that lidar has made significant progress in being compact enough and produced economically enough to make it a viable option to compete with cheaper (although arguably less effective) alternative technologies.
A team of researchers at Stamford University has been striving towards making lidar smaller and more cost effective. The team, led by electrical engineer Jelena Vuckovic, is working on shrinking the mechanical and electronic components in a rooftop lidar down to a single silicon chip that could potentially be produced at volume for just a few hundred dollars. The study, which was published in Nature Photonics, saw researchers structure the silicon in a way that used its infrared transparency to control, focus and harness the power of the photon. The process used was called ‘inverse design’ – something pioneered by Vuckovic’s lab over the past decade .
This relies on a powerful algorithm, which drafts a blueprint for the actual photonic circuits that perform specific functions. In this case, shooting a laser beam ahead of a car to locate objects in the road and routing the reflected light back to a detector. Based on the delay between when the light pulse is sent forward and when the beam reflects back to the detector, lidars measure the distance between car and objects.
It has taken Vuckovic’s team two years to reach the circuit layout milestone for their prototype, and building this range-finding mechanism on a chip is just the first step toward creating inexpensive lidars. The team is now working on the next milestone, ensuring that the laser beam can sweep in a circle without using expensive mechanical parts. A ready-for-road-testing prototype is expected to be ready within the next three years.
Back to business
There are also significant moves forward in business case scenarios, as evidenced by autonomous vehicle sensor and software firm, Luminar. The company’s lidar technology was selected to be included in Volvo’s next generation of cars, which will be manufactured to be hardware-ready for autonomous driving as soon as 2022. The partnership will deliver Volvo’s first fully self-driving technology for highways with the SPA 2 vehicle. The lidar technology is designed and sized for seamless integration into the roof of the vehicle’s modular architecture. The companies believe that this could pave the way for future active safety developments.
Henrik Green, chief technology officer at Volvo Cars, explained: ‘Autonomous drive has the potential to be one of the most life-saving technologies in history, if introduced responsibly and safely. Providing our future cars with the vision they require to make safe decisions is an important step in that direction.’
The technology can be scaled to integrate seamlessly into the roof of a vehicle's modular architecture. (Image: Luminar)
The lidar sensors from Luminar operate by emitting millions of laser pulses to scan and accurately detect objects in the vicinity, creating a temporary, real-time 3D map without requiring internet connectivity. The technology operates via the pulses of a single 1.5µm fibre laser being steered by two-axis scanning mirrors. It offers a 250-metre range and high resolution, which is made possible because of the longer wavelength of light (many other lidar technologies operate at 905nm). Operating at 1.5µm is also deemed less damaging to the eyes and means higher-power lasers can be used.
The lidar sensors from Luminar operate by emitting millions of laser pulses to scan and accurately detect objects in the vicinity. (Image: Luminar)
‘Volvo is recognised as the pioneer of automotive safety, having driven standardisation across the industry for the most advanced life-saving technologies,’ said Austin Russell, founder and CEO of Luminar. ‘The next era of safety lies within autonomous driving and once again, Volvo has taken the lead with a major industry milestone. We’ve solved the key cost, performance, and auto-grade challenges to make series production possible, and alongside Volvo are making the technology available to the world.’
Partnership production
From a manufacturing standpoint, lidar expert LeddarTech recently announced that it will begin volume production of its Pixell Cocoon lidar module, following a partnership with manufacturing firm, Clarion Malaysia.
The module is a 3D solid-state flash lidar with a 180-degree field of view. It has been designed and tested for off-road, shuttles, robotaxis, delivery, commercial and heavy industry vehicles in addition to other robotic and automated applications. It ensures safety thanks to its ability to reliably detect pedestrians, cyclists and other obstacles in the vehicle’s vicinity. It has already been adopted by a number of autonomous vehicle providers in North America, Asia and Europe.
Leddar Pixell integrates the company’s patented technologies embedded in the LCA2 LeddarEngine. This uses signal processing software to produce raw lidar data such as distance, position, and intensity of echoes. LCA2 is suitable for a wide range of applications and can be produced in high volumes for rapid mass-market deployment.
Charles Boulanger, CEO of LeddarTech, explained: ‘The large-scale production of the Leddar Pixell will enable us to quickly support customers around the world with this highly in-demand cocoon lidar solution.’
Leading the way
This kind of investment and product development in making lidar more accessible and cost-effective for the autonomous driving sector is showing the additional benefit of paving the way for other applications such as environmental monitoring. As an example, GreenValley International, as part of its lidar in forestry applications series, investigated the accuracy of diameter at breast height (DBH) measurements – this is the measurement of a tree trunk at a height of 1.2m – using handheld, mobile lidar.
The researchers selected an approximately 8,100m2 site in Berkeley, California as a study area. This area was surveyed using the firm’s LiBackpack handheld mobile lidar system. This integrates laser scanning with simultaneous localisation and mapping technologies to rapidly generate high accuracy geospatial products with minimal data processing.
A field technician measures each tree at 1.37m ( ~4.5ft) above the soil's surface on the uphill side of the stem base. (Image: Green Valley International)
To assess the accuracy of the fitted DBH measurements generated, a field technician took in-situ stem diameter measurements of all trees located within the study area using a traditional diameter tape placed at a height of 1.37m above the soil’s surface on the uphill side of the stem. Measurements were taken to the nearest centimetre and did not include any adjustment for stem slope. The species type was also determined and recorded by the technician for all trees included in the field inventory. In total, 87 individual trees were measured for DBH.
This analysis of DBH measurements – derived from the backpack’s data using the firm’s LiDAR360 software solution’s forest module – demonstrated that a high level of accuracy (root mean square error to 4.26cm) can be achieved. Accurate individual tree metrics scale up to become reliable estimates of volume, cost, and value that are of great interest to natural resource managers and forest landowners.
Thin down
Likewise, lidar is being used by the U.S. Forest Service to help with landscape-scale forest restoration planning under the Lake Tahoe West Restoration Partnership. As part of this, a team from the University of Nevada, Reno, led by Adrian Harpold, has developed a modelling tool to look at water quantity in order to help predict how different approaches to thinning the forest impact snowpack accumulation in Lake Tahoe, which controls how much water is available for downstream communities such as Reno.
The tool, developed with funding from the College’s Experiment Station and the U.S. Forest Service, was built to specifically model the west shore of Lake Tahoe, which the team saw as a good sample of the Sierra Nevada forest. They initially created a small-scale high-resolution model using data collected with 3D lidar.
Harpold explained: ‘The lidar data lets us see individual trees, which we use to “virtually thin” the forest by taking trees out of the model. As such, it lets us create a thinning experiment that’s realistic. We can then represent different management actions, such as removing trees below certain heights.’
His team tested the model’s accuracy by conducting thinning experiments and comparing the predictions to measurements in the real forest. Results were discussed in a recently published article in the Ecohydrologyon journal, for which Harpold was the lead author.
Once the team determined the model was working correctly, they increased the size to represent Lake Tahoe’s western shore. Results are discussed in another recently published article in the journal Frontiers in Forests and Global Change. Their experiments showed that, overall, more trees removed means more snow maintained. However, there are beneficial ways and detrimental ways to remove trees. Harpold will continue to expand the model, testing to see if it will work for Lake Tahoe’s eastern shore and in the American River Basin, with the ultimate goal of providing a tool for Forest Service decision-makers to inform their forest-thinning plans.
Beaming bugs
As well as forestry, lidar has been used to monitor mosquito activity in a bid to help advance malaria prevention. The project was set up by a global team under Lund University’s Mikkel Brydegaard. The research team set up their lidar next to a village in Tanzania. The device scanned 3 to 5m above the ground over a wedge of land at distances of up to 596m.
The lidar used the Scheimpflug configuration, in which the entire laser beam is transmitted into the atmosphere and can be clearly focused over the kilometre range on a tilted image sensor, if the optical configuration satisfies the Scheimpflug principle. The principle is the geometric rule that describes the orientation of the plane of focus, the lens plane, and the image plane of an optical system.
(A) The pan-African annular eclispse on 1 September 2016. The location of the lidar is indicated with an arrow. (B) A close-up aerial view of the agricultural patches with the lidar transect indicated by the red line. (Image: Lund University Science Advances)
The university team’s lidar has infinite focal depth to enable very high sample rates for modulation analysis of remote insect targets. Monitoring was carried out using an invisible near-infrared beam, transmitted from a 3W laser diode, emitting at a wavelength of 808nm with vertical polarisation.
The beam is expanded by a f600mm, 120mm-diameter refractor and focused into a 2.5 x 23.3cm line at the distant termination target. Total emitted intensity is estimated as 400mW continuous wave, or 457µJ pulse energy. The intensity at the aperture is 3.5mW/cm2 CW or pulse energies of 4µJ/cm2. Intensity at the termination is 17mW/cm2 continuous wave or pulse energies of 19µJ/cm2. The MPE (maximum permitted energy) is 4µJ/cm2 at 808nm and 300µs for pulsed consideration and 20mW/cm2 at 808nm and 300µs for continuous wave consideration.
A good idea
The backscattered light was collected by a f800mm, 200mm-diameter Newtonian reflector telescope. The baseline transmitter-receiver separation was 814mm and positioned vertically. After being transmitted through an 808nm, 10nm FWHM band-pass filter, light is detected with a linear, 2,048-pixel complementary metal-oxide semiconductor detector in Scheimpflug configuration at a 45-degree tilt. The sensor was operated at a 3.5kHz line rate, and the laser was modulated on even and odd exposures for sunlight subtraction throughout the day.
The team observed more than 300,000 insects over five days, which allowed them to gain a better idea of how mosquito activity varied throughout the day. To determine whether these behaviour patterns were caused by light levels or the circadian rhythms of mosquitos, the researchers also made measurements during a solar eclipse in September 2016.
To quantify activity, Brydegaard and colleagues used previously measured values for the wing-beat frequencies of mosquitos, which vary between males and females and between different species. This is important because only the females bite. By searching for these wing-beat frequencies in their lidar data, they found that mosquitos were most active during two ‘rush hour’ periods in the morning and evening. They also became far more active during the eclipse, which suggested that light levels, not circadian rhythms, are responsible for these bursts in activity.
Brydegaard says these results prove lidar could be groundbreaking in the study of insect ecologies over large geographical areas. This in-turn could help inform measures to reduce the risk of malaria infection. The next step in the research will be to improve the techniques used to incorporate rapid advances in lidar technology. The researchers ultimately hope that their work could help bring about techniques that will help to ease the suffering of millions, caused by malaria.
Dr Simon Rankel and John McCauley, of Ophir, discuss the importance of precisely measuring lidar parameters and its impact on the performance and construction of lidar systems
The Ophir IS1.5 integrating sphere includes a fast photodiode (FPD) to accurately measure pulse-to-pulse energy. Shown here with a Centauri energy meter
Many experts believe lidar is one of the pivotal sensing technologies required to make partial to full autonomous vehicles possible. The most promising way seems to be combining lidar with other detection/ranging systems like cameras and radar products into a suite of detection and ranging systems allowing autonomous vehicles to use at least two independent redundant subsystems to provide high-resolution, 3D information about the surrounding environment with very low error rates. Lidar is not only a key driver in the development of autonomous vehicles, it also prevails in numerous other application sectors from all kinds of mapping, industrial automation/robotics and smart city to aerospace, security and defence sectors. Just recently, the discovery of hidden archeological sites in the middle of overgrown jungle or monitoring of air pollution in cities were hot topics in the media.
While the benefits of lidar are indisputable, and the applications are wide, the development of these systems are still improving with respect to areas such as eye safety, power consumption, and overall system reliability. Knowing how the light source in these systems is behaving is critical to the success of the overall system. Measuring and understanding key lidar parameters such as average power or pulse energies, wavelengths, pulse duration, repetition rates, and beam divergence are critical to successful development of lidar systems.
To choose the right measurement solution which will provide reliable results, it is important to understand the characteristics of the lidar system used. Most lidar systems are based on pulsed time of flight (ToF) distance measurement method. In creating reliable and adequate flash ToF lidar with no mechanical moving parts, the intensity, phase and frequency of the transmitted signal needs to be precisely modulated to generate the nanosecond pulses with high peak power. The pulse energy then needs to be precisely measured for such lidar to reach long distances while complying with permissible exposure limits, thus a complex function of wavelength, repetition rate and energy per pulse.
As eyes of humans and animals are very sensitive to laser light the question arises: What can be referred to as being eye safe? Many automotive lidar systems are using 905nm or 940nm laser wavelengths. At those wavelengths light is easily transmitted to the retina. At 1,550nm wavelength in contrast, is the threshold for retinal damage much higher than at lower wavelengths, so higher powers and consequently longer ranges can be achieved. But even at the 1,550nm wavelength, lidar corneal damages could occur if the laser power is too high. When developing a lidar system, the complete picture needs to be considered. Therefore, measuring and precise determination of pulsed energy and power is crucial for knowing how to stay within eye safety limits of laser illumination. And lidar detectors themselves can suffer from overexposed laser illumination.
Typical flash lidar pulse durations are from few ns to few 100ns with pulse repetition rates (PRR) in kHz to MHz range. High PRRs are crucial to provide fast scanning and reliable 3D feedback on scanned environments. As here a combination of a high peak power with a low average power is used a special measurement approach is required, for example using power sensors with special frequency modes. Ophir integrating spheres in power ranging from 1µW to 30W can measure highly divergent beams as well. As pulse characterisation is essential, Ophir´s IS1.5 integrating spheres (image above) are equipped with fast photodiodes (FPDs) to accurately measure pulse-to-pulse energy. Additionally, a spectrometer can be connected to the sphere and all measurements can be performed in parallel. To overcome pulse stretching issues, small spheres need to be used.
Another option to measure the pulse energy are pyroelectric and photodiode sensors. Those sensors can measure pulse rates up to 25kHz and pulse energies down to 10pJ at 900nm and 30pJ at 1,550nm.
To ensure that the light source is operating at peak efficiency, measuring average power is essential to overcome issues in power consumption, which is critical to the performance of products such as battery-powered flying vehicles, like drones.
Another important aspect in terms of the accuracy of lidar systems is the quality of the evaluation of the returned signal in relation to the quality of the beam being sent out. If the performance of the light source is consistent and of high-quality, the signal received from the target is evaluated more accurately. Ophir´s camera-based beam profilers analyse beam quality, size, shape and divergence precisely, both in far-field or near-field set-ups.
Most will agree that providing robust and accurate lidar systems to the automotive arena is crucial to the evolution of self-driving vehicles. The measurement, analysis, and full characterisation of light sources in these lidar systems is a small part of achieving this goal, but it is vital to the success of these programs.
