Autonomous drones next step in shark safety

Bruce Mackenzie
Swellnet Dispatch

Surfers know they share the water with sharks, but technology may soon offer some added protection from a possible encounter.

According to Southern Cross University researcher Andrew Colefax, the day is nearing that autonomous drones — which do not require a line-of-sight operator — will be able to offer shark detection at any point along the coastline.

"I feel like that's around the corner," Dr Colefax said.

He has spent four years of research and development in the field of drones and shark detection, and said Artificial Intelligence (AI) and machine learning will be a game changer on beaches in the near future.

"There is continual research in this method to make it more reliable and provide a better level of safety," Dr Colefax said.

Drones are operating on many popular swimming beaches across the state under a Surf Life Saving NSW and State Government program, however they require a pilot to operate them within the line of sight.

"It does limit them a little bit to the area that they can cover and having to rely on a pilot means it can be considered … a bit resource intensive," he said.

Dr Colefax said another limitation was piloted drones under use were subject to a surf club's hours of operation and tend to operate only the vicinity of surf clubs, which leaves surfers vulnerable.

"The majority of shark encounters tend to involve board riders rather than swimmers, so it's important that we develop technology with the intention to provide surveillance beyond the red and yellow flags and hours of beach patrols," he said.

Autonomous shark detecting drones could patrol for longer periods of time and over longer stretches of coastline without requiring a line-of-sight operator, but roadblocks to the technology remain.

"We need that AI to be able to pick up and reliably distinguish one animal from another, or one shark that might be a threat to public safety versus another that is no cause for alarm," Dr Colefax said.

Air safety regulations are another issue.

"Drones share the airspace with manned aircraft and so when you talk complete automation it needs to be reliable," he said.

"I think what's missing is a bit of traffic management development. Aircraft [need to be] able to monitor what drones are in the air, where [they are] and vice versa."

Drone systems are not new to the Australian aviation landscape. Semi-autonomous drone home deliveries of food and other goods have operated in Canberra and parts of Brisbane for many years.

Spokesperson for the Civil Aviation Safety Authority (CASA), Peter Gibson, said thousands of hours of stringent tests would be required to permit a more autonomous drone system to patrol the coastline.

"We are certainly not keen seeing autonomous drones flying directly over people at beaches," Mr Gibson said.

He said the risk of a drone falling out of the sky and injuring a swimmer in the water or on the sand was the biggest risk.

But Mr Gibson said CASA had an "open mind" and will continue to work with universities like SCU to ensure public safety as drone technology advances.

"There's lots of things that need to be done in terms of meeting the safety standards, but it certainly is possible having autonomous drones flying," Mr Gibson said.

"We expect it will happen more into the future."

// BRUCE MACKENZIE, MELISSA MARTIN, and CLAUDIA JAMBOR
© Australian Broadcasting Corporation. All rights reserved.

Comments

t-diddy's picture
t-diddy's picture
t-diddy commented Tuesday, 8 Dec 2020 at 5:04pm

How was that second encounter - nearly run over by another surfer and then a shark after him!

donweather's picture
donweather's picture
donweather commented Tuesday, 8 Dec 2020 at 6:18pm

Interesting that that shark followed directly the wake of the previous surfer (perhaps using it as some cover). Then def got spooked by something around the second surfer. Wonder if that surfer had eyes on the underside of his board?

Blowin's picture
Blowin's picture
Blowin commented Tuesday, 8 Dec 2020 at 6:10pm

I’d rather roll the dice on the one in a million chance of shark attack than live under a perpetual drone surveillance state.

Fck your invasive technology.

99 percent of encounters will be post hoc click bait for the internet and nothing more. How does a remote drone save a life ? Oh wait .....you also need a smart device with an alert function strapped to your wrist 24/7 or implanted in your skin .

I appreciate the effort and sentiment* but spare me the death of a million cuts which results from the modern tendency to overreach with safety concerns. I go surfing to escape that shit.

* Is the sentiment even altruistic or is it opportunistic ?

Solitude's picture
Solitude's picture
Solitude commented Tuesday, 8 Dec 2020 at 7:32pm

Drones........naaaaaaaaahhhhhhh

Cannot stand the annoying whir when someone decides to invade my surf with one.

As Blowin said, what are they even going to do? Tell you what’s already there? Tell us what we already know?

As with surf cams, I reckon this constant need to surveil everything for a $ or otherwise has gone too far.

tony.dwyer's picture
tony.dwyer's picture
tony.dwyer commented Saturday, 12 Dec 2020 at 9:26am

Hey Blowin, it is another cash grab by what will be 'sponsored' by some random big end of town company who's CEO surfs! I agree with you I surfed when living right on East Ballina most days and never saw a shark then all these attacks. Surfed the ledge at Shelly's and all the pussies were further up the beach, again no tigers? We can get attacked from research a metre from the shoreline. Put it in the back of your mind and enjoy the sea or stay home and play video games.

Rabbits68's picture
Rabbits68's picture
Rabbits68 commented Tuesday, 8 Dec 2020 at 5:59pm

"We are certainly not keen seeing autonomous drones flying directly over people at beaches," Mr Gibson said.

He said the risk of a drone falling out of the sky and injuring a swimmer in the water or on the sand was the biggest risk."

Survive the shark encounter only to be taken out by the drone......

Pristine

gsco's picture
gsco's picture
gsco commented Tuesday, 8 Dec 2020 at 7:11pm

With a PhD in an area related to machine learning and having worked for the better part of a decade in applying it in hedge funds to algorithmic, high frequency financial market trading, I can say with certainty that training a machine learning model (say a deep convolutional neural network) to recognise a medium sized or large fish in a drone image is the easy part - even to distinguish them from other objects in the picture like surfers, swimmers, boats, sup paddlers, etc. There is already "off-the-shelf" models, say in keras/tensorflow python libraries, that could be used to do this immediately to an extremely high level of accuracy upon training on a large enough sample of drone images.

However, the researcher says:

"We need that AI to be able to pick up and reliably distinguish one animal from another, or one shark that might be a threat to public safety versus another that is no cause for alarm"

I believe it would be very challenging for even the best image recognition deep learning algorithms to do this from a drone image of the ocean - even distinguish a shark from say a dolphin, let along distinguish between shark species - because the image quality would likely not be good enough, particularly due to the sharks being under water with ripples on the surface...

I can also see the practical, ethical and privacy issues surrounding manning an Australia-wide drone fleet for shark detection... I don't really see a fleet of drones as a desirable option here. Better that people take personal responsibility for their actions and decision to enter a natural environment that poses an (albeit very low) safety risk.

Blowin's picture
Blowin's picture
Blowin commented Tuesday, 8 Dec 2020 at 7:19pm

Gsco....If a human can differentiate between the shark and dolphin or between species of sharks and even any behaviour of a shark which may be indicative of predatory intent , can’t a machine learn to do the same ?

Could a machine do so already ?

Surely a machine registers water clarity and sunlight reflection impediments to visibility to the same degree as a human eye ? If that’s the case then isn’t a drone only limited by the same conditions which would hamper human observations ?

gsco's picture
gsco's picture
gsco commented Tuesday, 8 Dec 2020 at 7:36pm

Yes it's entirely possible.

In the area of image recognition, deep learning algorithms have displayed quite unbelievably and astonishingly impressive accuracy, and in some (but not all) applications better than the human eye (eg cancer detection such as melanoma).

I'd be very interested in seeing the capability of the most contemporary, state-of-the-art image recognition algorithms in this application of shark detection from drone ocean images.

I would mention that as a human I'd have a hard time distinguishing shark species from a drone image that was taken fairly far away and in less than ideal (light, sea-state) conditions etc. I think these algorithms would also struggle a lot of the time to realise high/satisfactory levels of accuracy.

Blowin's picture
Blowin's picture
Blowin commented Tuesday, 8 Dec 2020 at 7:39pm

Interesting.

Cheers for the reply.

freeride76's picture
freeride76's picture
freeride76 commented Tuesday, 8 Dec 2020 at 8:00pm

I guess the thing with the drone is, then what?

Remigogo's picture
Remigogo's picture
Remigogo commented Tuesday, 8 Dec 2020 at 9:26pm

When ever I got to watching special features of 'finding nemo' dvd, specifically, how the animators added lighting features. I found difficult to believe and totally revolutionary. Still to this day...ah great flick.

Only a matter of time if not already these algorithms would give perfect clarity through the oceans surface. To believe otherwise is too simplistic (To a certain depth of course, for the time being).

But I agree with the noise and privacy intrusion crossing a line.

Daltz007's picture
Daltz007's picture
Daltz007 commented Wednesday, 9 Dec 2020 at 2:42pm

I would like to suggest that most of the sharks roaming around the beaches should be treated cautiously, with the likes of a nurse or a wobby generally slugging around not doing much off a reef or headlands.
Therefore, if one was spotted when people were around (beach), send a siren off, say if shape over 2m in length (and didnt have a bottlenose, wasnt a sunfish/seal/etc) and move off to the next beach, job done.
Tough titties if your offshore or on a slab/pointbreak :-)
Though the thing cant be in the air all of the time, monitor every beach/person or be expected to be infalible, so what changes ? a pooftienth of a chance less than the already tiny chance of attack ?

There's a market here, hire drone guy's to monitor your surf. "Shark Sheriff's"...Whitey no bitey>

On tech related, i had a mate tell me the other day of some software/startup hes invested in, that tags videos based on its content not what it is told/tagged by the uploader........hmmmmmm. E.g. I would like to see all the Alfie Langer tries in the 90"s or professor numpty's lectures on darwinism in Tasmania last term of 2010.

Thats some AI for you.

Ape Anonymous's picture
Ape Anonymous's picture
Ape Anonymous commented Wednesday, 9 Dec 2020 at 4:01pm

What a crack show.
Who pays for this shit!?
Soon we'll be getting pre-surf colonoscopies to make sure we are healthy-enough to surf.

Oh Swell's picture
Oh Swell's picture
Oh Swell commented Wednesday, 9 Dec 2020 at 4:33pm

Not a bad idea, would probably save a lot more lives. Death by ca colon isn't all that quick either.

"Sorry sir, no scope, no surf. No, a rectal examination is not sufficient."

Shaun Hanson's picture
Shaun Hanson's picture
Shaun Hanson commented Wednesday, 9 Dec 2020 at 5:52pm

Drones ...how is a drone going to stop a shark attack ? feel good bullshit i say and very annoying

Pastmypeak

truebluebasher's picture
truebluebasher's picture
truebluebasher commented Wednesday, 9 Dec 2020 at 6:41pm

Salute Bruce, Andrew, gsco & crew...for a great wrap.

(Part 1)
Late 1800's Quality accessible Surf beaches attracted surfers.
These Rail Port / beach resorts were patrolled by Saltwater > Pilots > SLS.
Bathing Reserves became WSR's now all lined with Hi-Rise Resorts.
Best Surf / Patrols / Shark Mitigation is the preserve of #1 Seachange Real Estate.
WSR = #1 Oz Beach Real Estate...from 1870's > 2020 # 1 Drone allocation.

Shark Mitigation lies at the heart of Oz Seachange City Plans.
Take away Shark Mitigation > No safe Beaches > No visitors > No High Rise Bank
No Developers = No Pollies.

Same & only ever one rule applies...Never ever dare say S_ _ _k.

(Part 2)
Iconic S_ _ _k Towers were a seachange developer's worst nightmare..."Wot" Tower?
1922 SLSA mandated 6m -14m Patrol Towers + Alarm Bells + Shark Spears / Guns?
Redhead WH listed complete with A Frame.
https://i.pinimg.com/originals/c7/77/01/c77701c4d33f0baa5866f97ab9fd90e5...
https://www.alamy.com/coogee-bay-beach-with-shark-tower-for-observation-...
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRLni2MdcdYbBNyYbXg...
1960's GCWSR
https://www.ar15.com/forums/general/Anyone_build_a_lookout_tower_in_thei...

(Part 3)
Land Sharks can sell more when the Sharks are fed 500m from the Sales Office.
Drone Halo ring of confidence journeys offshore to shore up real estate & votes.
Developers wipe sweat from their brows...no trace it was here at all...just calm down.

NSW 'Shark Smart' Drones are redeployed to "Provide real time crowd monitoring"
"In addition to real time council / police > Drones do Marine creatures, rescue, rips." (?)
2020 - Shark Smart mitigation now faces the wrong way all day in real time?
NSW fake Surf Drone Trials now pile higher than a Shark Tower...(Tell the Truth!)

Drone data is fed to Operations Centre to avoid Patrols yelling M _ r _ n e Creature.
SLS NSW Commands 350 assets / 129 clubs - 40 cameras.(Controls Council cams)
This includes private Cams & would most likely oversee this or any surfer's proposal.
In fact they do already coordinate Surfer's Rescues + as envisaged runs 24 hrs / day.
To do all NSW beaches you'd need a City size Command centre of 1000's staff.
Note current Council / SLS Drones report in different sequencing to operations.
It's doubtful that ocean users will escape, in a game of seconds...paddle much faster!
As crew said, gotta push yer ankle band deeper with wrist band held higher!
Now simply raise yer spare hand for Help! Easy!

(Part 4)
Shark Smart Drones wake at 9am > Smart Buoys / Drones knock off at 4pm.
NSW Drones are currently operating w/e & p/h only!
https://www.surflifesaving.com.au/news/drones-keep-sydney%E2%80%99s-beac...

SLS NSW Shark Smart : "Avoid swimming at Dusk & Dawn" (Feature Article)
Extending Drones into Dusk/Dawn service will spot 2 metre+ Shark danger daily.
Under Current rules > Ongoing 2m+ daily danger [ BEACH CLOSED ] > 365 days.
Resulting in a City Loss of Tourism / Real Estate far far greater than covid could do!
No Oz city or surfer could survive continuous Drone surveillance shutdowns.
In fact! Surfers would take the drone down by end of week 1 Trial. (Without a Doubt!)
The better Drone Tech becomes the less Surfers will surf. (We know that as fact)

(Part 5)
Constant Coastline Sand mitigation puts more lineups out of reach from Drone Base.
Surfers once hugged rock breaks now hitch Desert Trains to desolate offshore banks.
National / Marine Parks + Whale Migration all ban Drones.
Airports ban SLS drone operations -eg: GC WSR is way outta bounds.
UAV Drones 450m program run, 150m height + 30m bans over crowds / Lineups.
350m height is optimum Shark Spotting range to detect, warn & clear...(Less = lunch)

Headlands / Points / Bommies / Reefs fall well outside 400m Casa offshore range
Drones are banned from Surfer's fav wind generated swells or crisp offshore winds.
a/h Drone glare..filters & shields limit view range to seconds response time...Useless!
UAV program > (eg; East Coast)
Blinding Clockwise am outgoing run fails lineup vision
Midday South Chop Shadows + Overheated Midday Batteries.
Blinding Anti Clockwise pm inshore run fails lineup vision.
Depending on Headland North or South voids chosen flight rotation by half again.
Anti Clockwise pm Drone faces North Sun glare with South Headland / chop shadow.
UAV would respond by narrowing visual to min shark respond time.(Way too late!)

Whereas a [457] headland spotter points at the Shark chomping the SLS Drone.
Onshore manual spotting can best offset the offshore Drone's blinding Sun angle.
Where offshore drone fails, spotters can step up!
Surf Crews know this to be true & won't be convinced otherwise!
Yet! Where do we read of such obvious vital Spotter combos by techies!
(Part 6)
Aerial surveillance is on the march.
But a drone still hit tbb in the head & destroys the harmony of a surfing reserve
AI UAV needs to represent all beachgoers to avoid being labelled selfish surfie debris!
Surfers keep thinking they represent a majority of beach & ocean goers...they don't!
In summer...boardriders get crowded out of the ocean...go ask them.

AI UAV minimum to do list...
Aircraft, Sun Glare, Rising Seas, Winds, Cloud Cover, Storms, Bush fires, Whitewater, Baitfish, Algal Blooms, Foam, Flood Debris-Pathogens, Seaweed, Whales, Sharks, Dolphin, Birds, Blue Bottles, Crowds in surf & on the beach + drowning swimmer...most would argue to include rock platform Fishing / Jetski Patrol / Dogs on beaches.

Failing minimum list, it's not seaworthy or remotely useful it's just more marine debris.
None doubt that Drone tech is improving & prices are affordable.
As said, NSW Govt have trouble pointing Shark Smart Drones in the right direction.
Do any here think NSW Govt will allow Surfers to boss #1 Seachange Real Estate.
M.m.mm _ r_ ne Creature! (Shh!) Nothing to see...everyone back to the Sales Office.