Given the value of the resources that can be found underground, how much risk from the thousands of tonnes of rocks above is acceptable? Opening up new areas to mining involves controlled blasting, creating rock fragments that must be removed, and potentially disturbing unsupported ground that might fall away. This creates significant risk to mine workers, as well as the extremely expensive remotely-operated bulldozers that modern mines extract rock with.
‘Sudden rock fall generates such an air pressure blast that it can kill people,’ said Stefan Hrabar, chief executive officer at Emesent. The firm’s Hovermap helps minimise that risk by automating drones to independently scan mines with lidar. ‘We’ve heard surveyors and mining engineers saying, “If only we’d had this system for 10 minutes we could have saved lives or millions of dollars.”’
Mines had already been using lidar to map their excavations, or stopes, mounted at the end of long booms over the precipice of underground cliffs, attended by surveyors. But Hrabar warned that not only is it dangerous to be in these underground areas, but it’s hard to obtain accurate volume estimates or get a full perspective. ‘Sending in a drone equipped with Hovermap means that they can launch it from a safe distance,’ added Hrabar. ‘The drone can fill in all the gaps, so there’s no shadowing or missing data.’ The resulting data also has even and high point density, which enables geotechnologists to resolve fine features, such as natural cracks and crevices in the rocks.
Mining is just one dramatic example of how optical sensing equipment is going into dangerous environments and providing information to help make people safer. There are many of other types of hazardous places, such as volcanoes and remote natural environments, where mounted drones are used when the risk is too high for humans. And, while lidar is the most common tool such drones exploit, there are also opportunities for spectroscopy and visual-wavelength imaging.
Brisbane-based Emesent’s Hovermap lidar integrates an off-the-shelf VLP-16 Puck lidar from Velodyne, using Emesent’s simultaneous localisation and mapping (SLAM) expertise to construct maps while tracking the drone’s location. The system allows drones to fly autonomously and can easily be mounted and unmounted, according to Hrabar. ‘You can actually just walk around with it, or mount it on a ground vehicle,’ he explained. While Hrabar couldn’t reveal the differentiators of his company’s approach in detail, they do include the choice of environmental features that the system recognises.
An important consideration for mapping is the lidar’s beam divergence, Hrabar observed. More divergent beams risk striking multiple objects and generating multiple return signals, which is a problem for producing detailed maps. ‘You can’t be totally sure exactly what measurement you should use to produce the map,’ Hrabar explained. ‘But then, from the safety perspective, if you’re trying to pick up structures like small wires, you might want a more divergent beam, so that you’re guaranteed to see these things.’ The Puck lidar provides a compromise between these two extremes, he added.
Lighter detection
Autonomous drone company Exyn Technologies also targets ‘infrastructure-free’ environments like mines that do not have access to GPS signals. ‘That means no persistent communications with the platform, and in many cases no prior information,’ said Jason Derenick, Exyn’s vice president for technology. Exyn’s Advanced-Autonomy Aerial Robots (A3Rs) are designed to fly autonomously in such environments, while generating real-time maps and executing high-level missions. If a mission cannot be executed or something unexpected happens, the drones are smart enough to make decisions and handle contingencies. ‘That delineates us from other drone companies,’ Derenick said. ‘All the sensing, all the computation, everything has to be done on-board.’
Exyn's A3R conducting a fully autonomous flight at a gold mine in Chelopech, Bulgaria. Credit: Exyn Technologies
Carrying out processing on-board means there needs to be excellent mission planning and state estimation functions, Derenick said, where the drone calculates its position and orientation with relation to its real-time 3D map, generated by lidar-powered SLAM. ‘We command the vehicle using missions – it could be that you ask it to visit some series of relative locations while avoiding obstacles,’ Derenick explained. ‘Or it could be that you ask it to execute exploration-type behaviour.’ Supporting those high-level goals ultimately involves determining what commands the drone should deliver to its low-level flight control unit, all while perceiving its environment in real time. The drone sees the world as a collection of cubes, Derenick said, some of which are occupied. Based upon which cubes it determines as free, it computes safe flight corridors.
While lidar is important, Exyn would like to find more lightweight ways to exploit it. ‘The smallest unit that we can reliably use in a mining environment is the Velodyne VLP-16 LW,’ Derenick said. This weighs 590g, while its robots weigh a maximum of 1.5kg. The Exyn executive also noted that commercial 3D lidar systems have very narrow fields of view (FoVs), just +/- 15 degrees from horizontal. Exyn therefore mounts the Velodyne lidar on a gimbal and rotates it to cover ‘almost the full volume surrounding the vehicle’. However, Derenick would prefer lidars with fewer moving parts, which would be more robust.
It’s harder for Exyn’s flying drones to deal with limited fields of view than autonomous road vehicles, Derenick added. ‘Autonomous cars know that they live on the ground,’ he said. ‘Our drone operates in a full volume, so its needs to see that volume in order to make a decision. Also, a lot of autonomous car startups do their testing in milder climates, where conditions are much more favourable for sensing at longer ranges. In a mine environment, you have high humidity and high density of dust, to the point where it can drastically reduce the effective range of the sensing to 50 per cent or less, depending upon the conditions.’
Velodyne’s unmanned aerial vehicle (UAV) and robotics business manager Frank Bertini added that road-based autonomous vehicles want lidar with high vertical resolution, with smaller vertical distances between measured points. ‘In many cases, UAV customers will sacrifice vertical resolution for a larger vertical FoV,’ Bertini said. He also stressed that its Puck Lite model is seeking to reduce the weight burden of lidar even further. Other potential improvements come from how Velodyne has designed its Puck sensor. Bertini called it ‘an elegantly engineered combination of 905nm wavelength laser pulsing diodes, optical lenses, and detectors’.
‘While Velodyne has used commercially off-the-shelf components in the past, we are currently implementing our custom-designed application-specific integrated circuits (ASICs), which will increase performance, reduce cost, and reduce the physical size of the sensor,’ Bertini added. ‘With the introduction of Velodyne’s patented ASICs, we will be able to build a multitude of smaller form factors. Many of the components within the Puck will be moving from a printed circuit board, down to the size of an ASIC, or small microchip – similar to how cell phones now have the equivalent computing power of a desktop PC from five to 10 years ago. Lidar electronics roughly follow Moore’s Law and we see a runway for many decades of improvement, since we are at such an early stage.’
Sensing limitations
Lidar can also help in hazardous and hostile external environments, such as sites with inaccessible steep slopes and dense vegetation, said Emma Thomas, director of Edinburgh-headquartered Mapix technologies. Its Routescene brand produces a drone payload called LidarPod that has been used in surveying an archaeologically important site in Sand Canyon, Colorado, US. ‘There were some very steep slopes on this site,’ Thomas said. ‘Using drones removes the hazards.’ It also minimises survey time and cost compared to manual walking or helicopter and aeroplane surveys, and offers much higher data resolution. ‘We’ve opened up the ability to survey to places where, historically, it just wouldn’t have happened because of price, budget constraints and danger,’ Thomas asserted.
Routescene's pointcloud of the access point in Sand Canyon, Colorado, US. Credit: Routescene
Routescene uses Velodyne’s HDL-32 because it has been in the market since 2010 and has demonstrated a very long mean time to failure. This is ‘exactly what you want in the middle of nowhere – you do not want your product failing’, stressed Thomas. ‘It’s got 32 lasers and thanks to the accuracy of those lasers, we’re able to do what we need to, in terms of vegetation penetration that you can’t do with others that don’t have the same power, accuracy and resolution.’
However, the HDL-32 weighs 1kg, so Routescene would welcome the same capabilities in a lighter-weight package. The company would also like an upgrade to its 100m maximum range. ‘The higher the resolution and accuracy you need for your final output, then the lower and slower you need to fly your drone,’ Thomas explained. ‘If there are things in the way, such as tall trees, you need to maybe fly anything up to 80m. As soon as you start flying high, you lose accuracy and resolution.’
The LidarPod can also be used together with a separate camera, connected together to synchronise data collection. This is only effective for collecting terrain colour information, Thomas warned, rather than for making photogrammetric measurements. ‘If you’re trying to do photogrammetry and lidar, and then merge the two, your best practice would be to fly the site twice, once with lidar at low altitude and then a second time with the camera at a higher altitude, so that you get optimum results,’ she said.
Further expanding drone-based optical sensing, a European industry-academia collaboration called FLying ultrA-broadband single-shot Infra-Red sensor (FLAIR) is using spectroscopy to analyse emissions from volcanoes or chemical fires, for example. Currently, the team is targeting visible plumes a few kilometres in length, explained Frans Harren, from Radboud University in Nijmegen, the Netherlands. ‘We aim for environments at which the concentrations will be very high and dangerous for human health,’ said Harren, who is the coordinator for FLAIR. The project seeks to use a large AR5 drone produced by Lisbon, Portugal-based Tekever, which has an 80km range, but needs a runway to take off. ‘It has the proper weight and can generate the proper power for the sensing system,’ he added.
The system uses a supercontinuum fibre-based diode-pumped mid-infrared source from NKT Photonics. ‘It’s very broad band, but has the directionality of a laser beam,’ Harren explained. That enables the FLAIR team to do spectroscopic measurements of atmospheric gas across all wavelengths from 3-4µm. But they don’t aim the laser at the atmosphere. Instead, it’s focussed into a folded ‘multipass cell’ developed by SensAir, where the laser shines through 10 metres of gas. The system will then measure light absorption by gas molecules using either mid-infrared cameras from New Infrared Technologies, or grating detectors. The drone will then send data wirelessly to scientists on the ground.
Many groups developing drones for hazardous environments use Velodyne's Puck VLP-16
Having demonstrated the principles in the lab, the Swiss Centre for Electronics and Microtechnology (CSEM) in Neuchatel, Switzerland, intends to build a system that fits in a drone by this summer. ‘It will hopefully fly by the end of the year,’ Harren said. Then, the Swiss Federal Laboratories for Materials Science and Technology (EMPA) in Zurich, will compare the drone to existing carbon dioxide and methane sensing systems. The EMPA researchers have these systems on a measuring tower, and the drone will fly around it to validate its measurements.
While spectroscopic drones enabling measurements in hazardous environments are undoubtedly important, their success will likely be limited, Harren warned. That’s because the market for such products is small. Consequently, his team is also exploring using such drones to ensure fruit doesn’t spoil. ‘If you can save food loss, for example, during storage that helps the farmer,’ Harren said. ‘If you can save them €60,000 per year, then they’ll do it.' While only five drones might be purchased by fire brigades in the Netherlands, there could be hundreds of farmers in the country who would use the system for monitoring food, according to Harren.
Such economic equations will presumably always be the driving force for deploying sensing drones into hazardous situations. And even if the prospects for spectroscopic drones seem less certain, today, lidar is useful in mining and densely-vegetated, remote outdoor environments – and is becoming more so.
