This Homemade AI Drone Software Finds People When Search and Rescue Teams Can’t

When Charlie Kelly first messaged saying he wouldn’t make it home that night, his partner wasn’t happy. It was September 6, 2023, a Wednesday, and the 56-year-old, a keen hillwalker, had left the house that he shared with Emer Kennedy in Tillicoultry, near the Scottish city of Stirling, before she went to work. His plan was to climb Creise, a 1,100-meter-high peak overlooking Glen Etive, the remote Highland valley made famous by the James Bond film Skyfall.

The weather was unusually mild for the season, and Kelly thought he might even have time to “bag” a second Munro, as the Scottish mountains above 3,000 feet are known. In his time off work as a forensic psychologist for the Scottish Prisons Service, he had been ticking off the peaks steadily. “He had this book he would mark them in,” Kennedy remembers. “But we were due to go on holiday in two and a half weeks, so this was the last Munro he was going to do before the winter set in.”

Hiking wasn’t something that Kennedy was particularly keen on herself. When the pair had first met four and a half years previously, they’d bonded over a shared love of Celtic Football Club, and their “extremely quirky” sense of humor. She’d fallen in love with Kelly’s brain—his encyclopedic knowledge of all things football, Robert the Bruce, and Doctor Who. He loved the fact that she laughed at “his terrible jokes,” she says. But he also appreciated the fact that she encouraged him in passions they didn’t share. “One of the last things he said to me the night before was, ‘You let me be me,’” she says.

So when Kelly told her he wouldn’t make it off the hill before nightfall, Kennedy was worried, but she trusted that he knew what he was doing. “Charlie was a very resourceful person,” she says. “At work, he was a trained negotiator, for when prisoners took hostages or went up on the roof. He generally didn’t take risks.” Kelly reassured her that there wasn’t any need to call for help. He had packed extra food, had plenty of water and enough warm clothes. He’d just wait for it to get light and walk down.

At work on the Thursday, Kennedy checked her phone whenever she had a break. Kelly had checked in before dawn and sent further cheery messages whenever he had reception. At around 8 pm, with the sun starting to set, he wrote to say his battery was running low, but she needn’t worry: He could see the lights of the Glencoe Ski Center, where he’d parked his car. There was still plenty of daylight left to reach it, he said. “It’ll take me about half an hour.” That was the last anyone heard from Charlie Kelly alive.

The breakthrough, when it came, was more than six weeks later. Dan Roach and David Binks, two MR team members from the Lake District, in northern England, had been following the news of the ongoing search online. In their spare time, they’d been developing a new piece of piloting and image analysis software, designed to help drones find missing people more effectively. After two years of work, involving what Binks calls “some hideous maths,” they had what they believed was a working prototype.

“I’d heard this search was going on,” Binks says, “and I kept thinking this would be quite a good place to try and test the system out. Then Dan phoned me while I was on my way home from holiday. He’d had the same idea.” The pair contacted Glencoe MR and asked if they could help. “I don’t think they thought anything was going to come of it,” says Binks. But by that point, they’d tried every other tool at their disposal. “So they were happy for us to try.”

On October 24, Binks, Roach, and Dan Parsons, a friend and fellow MR team member of Roach’s, who had helped with early testing, traveled to Glencoe. They met up with their local MR counterparts, launched two drones, and found Charlie Kelly’s body within the first hour.

Image may contain Rock Nature Outdoors Scenery Wilderness Landscape Adventure Hiking Leisure Activities and Person

Mountain rescue volunteers are often called out to find missing hikers, and drones have become an essential part of search and rescue plans

Photograph: Tom McNally

Mountain Rescue in the UK is often referred to as the country’s fourth emergency service. But unlike the police, fire brigade, or ambulance services, it is staffed entirely by volunteers. The country’s upland areas are covered by a patchwork of teams made up of locals from all walks of life. Each team operates as a separate registered charity, responsible for its own fundraising, training, and equipment.

This decentralized structure has its advantages, according to Mike Park, the elected CEO of MR England & Wales, an umbrella body that helps improve cooperation between teams. “But the negative side is it can bring a lot of individualistic, ‘we know best’ type attitudes,” he says. Park is the former leader of Cockermouth MR in the Lake District—Roach and Parsons’ team—and a volunteer himself for “coming up to 42 years.” He explains that MR members—the majority of whom are men—sometimes get a bit too tribal about their areas of expertise, and new ideas take longer to spread than he would like. “There’s this old culture where it’s like you’ve got to do 20 years before you’re allowed to be listened to,” he says.

When consumer camera drones first became widely available in the early 2010s, their usefulness in search and rescue situations seemed self-evident. They would allow teams to cover vast areas, for a fraction of the cost of a helicopter. But Roach, an early enthusiast who’d used drones for photography work, wasn’t convinced. A keen climber, with long, curly hair and a Pedro Pascal mustache, he joined Cockermouth MR a decade ago and is still one of the team’s younger members at 34. “Everyone thought, ‘This is going to be brilliant because I can see things.’ But then they realized you have to be very close to the thing that you want to see, and you have to be able to see it on a very small controller.”

If you’re trying to find a needle in a haystack, having an aerial view of the haystack doesn’t necessarily help. Instead, Roach argued that they should be taking advantage of drones’ ability to fly preprogrammed flight paths and trying to automate the search process as far as possible. Among MR’s old guard, however, he faced an uphill battle to get his ideas heard. “I remember one MR conference at Leeds University, where Dan shows up with his long hair and his skateboard,” Mike Park says, “and someone asked if he’d walked into the wrong place—they thought he was a student who’d come back a week early.”

Frustrated by people’s refusal to listen, Roach pressed on regardless, with the help of his friend and fellow volunteer Dan Parsons. As well as being a similar age, Parsons, 33, had a background in forestry and had previously used drones for surveying purposes. The pair of them set about testing off-the-peg piloting and photography software. They also found a program that could pick out pixels of particular colors, like the red of a hiker’s jacket. “In the end we had this extremely tech-heavy, very janky system,” Roach remembers. “It worked, but it wasn’t pretty. You had three different bits of software that you had to engage to search.” It wasn’t until Roach met David Binks, a retired software developer who volunteers with Duddon & Furness MR, which covers an area about 30 miles south of Cockermouth, that the idea really began to take shape. Roach spoke about his attempts to automate drone searches at a meeting of Lake District pilots and remembers Binks collaring him afterward. “He said, ‘I hadn’t thought about it like that. Let’s chat.’”

They make for an unusual pairing. Roach, “the ideas guy,” exudes a Tiggerish enthusiasm, whereas Binks is a far quieter presence, with an engineer’s analytical brain. At 57, Binks is more than 20 years older, but crucially, he had experience not just of writing software, but software that had been widely adopted by MR. Starting in 2007, he’d built a program called MR Maps, which allowed teams to track their members in near real time, based on regular pings from their radios. In an era before GPS-enabled smartphones or handheld devices were standard, it proved a game-changer. After initial meetings in late 2021 and early 2022, Binks decided the best way to build Roach’s idea was as an add-on to MR Maps.

Image may contain Photography Camera Electronics Video Camera Adult and Person

Thanks to new image-analysis software, humans can be more easily located among even dense foliage and rocky ground.

Photograph: Tom McNally

“I knew how the maths would work, from my background working in simulation software for offshore oil platforms and wind farms,” Binks says. “So I knew it was possible. But I did a lot of nights where, well, I’d start at 9 in the morning, and before I realized, it would be 3 in the morning, and I’d just been at it all day.” Perhaps betraying its homemade origins, and the fact that it was written using C++, the resulting software has what Roach describes as “a Windows 95 aesthetic.” But the basic interface belies an impressive level of sophistication.

Their main challenge, Binks explains, was threefold. “Part one is flying the drone in a way that photographs the ground in the most effective way.” The camera has to point straight down, and every inch of ground must feature in multiple frames, so that objects can’t be obscured by the angle of a wall or rock. “We have a 55 percent overlap, so one object should be in nine frames, on average,” Binks says. The preprogrammed flight paths are plotted automatically, at whatever height the pilot chooses, using the 2-meter by 2-meter lidar data that underpins the contours of the UK’s Ordnance Survey (OS) maps, the official maps of the UK government (which OS supplies to MR for free). The software also tells the pilot where to stand to avoid losing the line of sight with the drone as it flies—a legal requirement in the UK.

“The second aspect of the software is calculating the grid reference of any particular pixel on an image,” Binks explains, “which you can do using rotational vectors, working out where they intersect with the ground.” The third element, Binks says, “is actually analyzing the images to work out what’s interesting.”

Roach and Parsons’ “janky” off-the-peg version involved inputting the specific colors that you wanted to search for, but Binks, crucially, realized it would be easier to flip that on its head. “So I tell it to count the number of pixels of any particular color, and then highlight clusters of colors that are unusual.” The red of a hiker’s jacket would still stand out on a grassy background, Binks explains, “but if the image is all rocks, then green would be a more unusual color.”

The system is designed to operate entirely offline, allowing it to be used in remote locations. This means the images have to be downloaded from the drone after each flight, but the analysis can be done in the field, with clusters of unusual-colored pixels flagged for human review while the drone moves on to the next search zone. “There are a lot of false positives, but it doesn’t take long for a human to go through them,” Binks says. “Usually it’s funny-colored moss, or rocks, or a sheep,” Roach says. “But every now and again, it’s a person.”

Image may contain Person Teen Computer Electronics Laptop Pc Diagram and CAD Diagram

Operators can plot out areas for the drones to focus on and can create automated flight paths to search.

Photograph: Tom McNally

The wild, boulder-strewn hinterland behind the Glencoe Ski Center provided a near-perfect proving ground for MR Maps’ new capabilities. Between the top of the lifts and the single-track road that snakes its way down to a dead end in Glen Etive, there’s very little except rocks, dun-colored tussocks, and the occasional deer. “The ground here is quite complex,” explains Brian Brathurst, one of Glencoe MR’s deputy team leaders, with an accent and a level of understatement that betrays his Zimbabwean origins. “There’s a lot of boulder fields, there’s a lot of gully systems, and you could walk within five or 10 meters of somebody and not see them.”

As the search reached the end of its second week, the chances of finding Kelly alive dwindled, and the intensity of the efforts was inevitably dialed down. “Sadly, people have their lives to lead,” Brathurst says. “But on our patch, we don’t tend to give up on these things. We were out every free day we had. For those six weeks, there was always somebody out—people finishing work in the afternoon and going and exploring areas that hadn’t been searched. It never really stopped.”

When Binks, Roach, and Parsons arrived, they were well briefed. “Without the work Glencoe had done, there’s no way we’d be able to do what we did,” Binks says. With the two Dans operating one drone, and Binks handling another, they divided up the most promising-sounding search areas between them and headed out to different starting points. Despite all the work they’d put into the software, they were nervous. “We’d found our coats in fields and simulated it,” remembers Parsons, “but we’d never found anything that we didn’t know we’d already put there.”

David started flying his first search zone, “Area A.” But about 10 minutes into the flight, the drone started to disappear out of sight. Annoyed that it had already gone wrong, he called it back and made a mental note to tweak the software. “I thought, OK, fine, I’ll do Area B now. But while it’s doing that, I’ll check what I’ve got from Area A.” He started cycling through the flagged photos, and suddenly, there it was. “Two of the images had Mr. Kelly in them,” remembers Binks. “I was stunned.”

When Glencoe’s recovery team reached the spot, they found Kelly had fallen a distance of several meters, suffering fatal head and chest injuries. There is no way of knowing precisely when he’d died, according to Brathurst. But there were combine harvesters working further up the valley on the evening of September 7. Because of Kelly’s last message, Binks says, we think he maybe made a beeline straight to those lights thinking it was the Ski Center, and fell in the process. But we’ll never know exactly.”

What was clear was that Kelly was nowhere near where he thought he was. Despite this, the area where he was eventually found had been searched extensively. But Kelly had fallen into a gully, obscuring the line of sight of rescuers on the ground. The muted colors of his clothing, meanwhile, had made him all but impossible to see from the air. “He was wearing a lightish brown top and dark blue trousers,” Binks explains. “To the human eye that’s really hard to pick out. But to the computer, it’s just totally different to the surroundings—and so it stands out like a sore thumb.”

Brathurst himself was one of the volunteers who had searched that area on foot. “We literally walked within probably 20 or 30 meters of him,” he says. And yet they’d not seen a thing.

Image may contain Rock Slope Nature Outdoors Wilderness Adventure Hiking Leisure Activities Person and Path

Even for experienced hikers, the outdoors can be a dangerous place.

Photograph: Tom McNally

If finding Charlie Kelly in the first area they searched involved a good deal of luck, it also showed the powerful potential of MR Maps. The software has proved its usefulness in multiple other searches in the months since, mostly by helping eliminate areas of interest. “Dan likes to boast that we have a 100 percent hit rate,” Parsons says. “Although we’ve only found one person, we’ve never flown over an area where someone was and not found them yet.”

So far, the software has been used mostly in the Lake District. But with Mike Park’s encouragement, Roach, Parsons, and Binks have been assisting other teams across the country too, when requested. When the celebrated British TV doctor Michael Mosley went missing on the island of Symi in Greece, Park even had conversations with his Greek counterparts about the trio flying out to help. “It actually looked like perfect terrain for this system, because it was rocky, and this can see around all the rocks,” Binks says. In the end, Mosley was found before they could deploy, but it’s easy to see how their software could reduce the agonizing wait for families like Mosley’s—as well as the burden for those with boots on the ground.

In the UK, MR is facing mounting pressure. “The number of callouts has been increasing for years,” says Ian Bunting, operations director for the umbrella body MR England & Wales, “but we’ve seen a massive leap since Covid.” Hiking and other outdoor activities became hugely popular when team sports were banned, he says, and with foreign travel off limits, the country’s national parks saw a surge in visitor numbers. “On the back of that, you’ve got the rise of social media,” Bunting says, which has helped publicize particular spots.

Cockermouth, Dan Roach and Parsons’ team in the Lake District, attended a record number of callouts last year. In North Wales, the Llanberis MR team, which covers Yr Wyddfa, the area’s highest peak, warned publicly that their team members were risking “burnout” after attending more than 300 incidents in 2023—up from just 100 in 2008. As a volunteer for 30 years with Edale MR in the Peak District, Bunting says the burden can be significant. “There are people in my team who attended over 100 incidents on their own,” he says. “With training, equipment checks, and everything, I was up to 600 hours last year.”

“The bottom line is that people are volunteers,” Bunting acknowledges, “and the beauty about being a volunteer is you can say no. But the other thing about volunteering is generally the kind of people who volunteer don’t say no.” Against this backdrop, any tool that helps reduce the time spent searching is a welcome addition to the MR toolkit, he says. Binks and Roach’s program wouldn’t necessarily be needed in the majority of callouts, where the location of the person is known. But its potential to cut the hours spent on resource-sapping “formal searches” (which make up between 10 and 15 percent of annual callouts according to figures shared with WIRED by MR England & Wales) is huge.

Globally, the market for search and rescue drones is worth around $4 billion, according to Lucintel, a market research firm. But despite their product’s myriad possible uses in mountainous regions around the world, Binks and Roach have zero interest in commercializing it. In the UK, Binks has always made MR Maps free for anyone involved in search and rescue. This includes the police, and several forces have started experimenting with the drone add-on. “It’s going to be used for search, so I’m quite happy with that,” Binks says.

“I’ve never really been interested in monetizing it,” he says. Instead, his motivation is simple. “I just quite enjoy the problem-solving aspects.” And in the end, he points out, the only real development cost was his, Roach’s, and Parsons’ time—which is something they, like thousands of other MR volunteers across the UK, are happy to give to help those in need.

Image may contain Clothing Coat Adult Person Bag Logo and Backpack

David Binks’ MR Maps software originally let rescue team members track “pings” from their radios.

Photograph: Tom McNally

When two uniformed police officers knocked on Emer Kennedy’s door in late October 2023, they simply said, “You know why we’re here,’” she remembers. She’d been hoping against hope that the outcome would be different. But after six weeks, she says, he “main panic was are they going to find him before winter comes in?” And if not, would they ever find him?

People traveled from far and wide for Charlie Kelly’s funeral. Professional associates from across the country. University friends from around the world. The Pogues played as his coffin was carried out, and a collection was taken for charity—with the proceeds going to the Glencoe MR team.

“Even though they do it on a voluntary basis, they’re incredible professionals,” says Kennedy. “If that Mountain Rescue team hadn’t come up with that software and hadn’t been willing to test it out there … ” She leaves the thought hanging. In the end, she says, “I’m just glad we have closure. It means his kids can have a life. And you know, we got to say goodbye—his friends and family got to say goodbye.”

This article appears in the November/December 2024 issue of WIRED UK magazine.

Source : Wired