Ellie Mackay piloting the drone during a survey of Soar Mill Cove, Dorset. (Credit: Theplastictide.com)
The drone images gathered by The Plastic Tide are uploaded to Zooniverse, a citizen science website where thousands of citizen science volunteers created hundreds of thousands of tags of what is and is not plastic litter, Dr Stefan Leutenegger (Lecturer in Robotics at Imperial College London, scientific advisor to The Plastic Tide) explains in an article on Theplastictide.com. Ben used this unprecedented dataset to train the machine learning algorithm, specifically a type called a Convolutional Neural Network (CNN). The network uses these thousands of training examples of labelled plastic pieces – such that it will finally be able to tell what is a plastic piece and what not in an unseen image all by itself.
We have furthermore developed a second algorithm, it is implemented in a piece of software that refines the position (roughly known from GPS) and orientation from which the drone images were taken. This is achieved by exploiting the fact that characteristic features of the environment are observed across several images of the same beach. This image registration technique allows us to know where the drone was and what exact direction it was facing, allowing us to precisely map the detected pieces of plastic in geographic coordinates. This means that not only can we use the CCN to identify plastics; we can accurately locate the items on beaches.
Below, we show some examples of the detection that the CCN outputs – remember, it has not seen these images before, but relies on the “memory” of similar-looking tagged training images. In other words it has learned, from the citizen science inputs, where to find the plastics!
The algorithm detecting plastics in images it has not 'seen' before, relying on what it has 'learnt' from previous images. (Credit: Theplastictide.com)
Above, a version of the network is used, that simply detects plastic pieces, without any discrimination of the exact type. Note that the results are not perfect: the network detects the same piece twice at different sizes in the left two images. It also detects something very small in the left image – which by closer inspection might not be a mistake, but an actual small piece of green-ish plastic that’s quite hard to spot even for a trained human!
Help them tag more plastics and litter CLICK HERE!!
To read more about this fascinating project, CLICK HERE!
