Researcher MSc. Anupama Rajkumar recently investigated whether VHR satellite imagery could be combined with advanced deep learning techniques to automate the detection of waste landfills. Prior to the use of remote sensing and advanced deep learning techniques, methods to detect landfills were based on visual identification, followed by classification methods to identify and map illegal landfills.

Data forms an integral part of all machine learning pipelines. For any machine learning algorithm to extract features or learn mapping from input variables, it must first be trained with high-quality data. The volume of data required depends on the feature to be extracted, the complexity of the model being trained and the number of parameters that need fine-tuning. The collection of the data used by a machine learning model is called the dataset.

To create the dataset that was used for the deep learning models for semantic segmentation to detect landfills, a series of landfill sites across Europe captured with GeoEye-1, WorldView-2 and WorldView-3 were selected—obtained through ESA’s Third Party Mission (TPM) Programme.

Born in 1990, in India, Anupama Rajkumar completed her Bachelor of Engineering in Electronics and Telecommunications from Pune University, India. She completed her MSc. in Autonomous Systems from Technische Universitaet, Berlin and Eotvos Lorand University, Budapest. Before pursuing research in Machine Learning (ML) and Computer Vision (CV), she worked as an embedded software engineer for several years at Bosch and John Deere in India.

Her research interest lies in the application of CV and ML in order to solve real world challenges, be it in autonomous driving or Earth observation. Currently, she is working as sensor fusion engineer for the AI Fusion team at the Continental's Artificial Intelligence Competence Center, in Budapest.

The research in this project was carried out in collaboration with Machine Perception Research Laboratory (MPLab), headed by Dr Tamas Sziranyi, Institute for Computer Science and Control (SZTAKI), Eötvös Loránd Research Network (ELKH), Budapest.

ESA: How and when did you decide you wanted to focus on Earth Observation?

Anupama: During the first year of MSc at TU, Berlin, I worked on a project where I used a vanilla autoencoder to extract features from a fully Polarimetric SAR (PolSAR) image of Oberpfaffenhofen, Wessling, Germany. It was while working on this project that I got interested in Earth Observation (EO) and became aware of the possibility of applying Deep Learning and Computer Vision to various types of satellite data that are available for use. The amount of useful information that can be extracted from satellite imagery fascinated me.

The same year I had an opportunity to volunteer at the International Geoscience and Remote Sensing Symposium (IGARSS), where I listened to various talks about the application to latest technologies to EO. That is when I decided that I should focus on this field for my thesis research.

ESA: How did this project come to life?

Anupama: In the past MPLab had collaborated with ESA, who supported projects based on EO. One of the latest projects being OWETIS, which dealt with the observation of small wetlands that deliver essential ecosystem and biodiversity. While discussing about my possible thesis research topic, my supervisor, Dr Andras Majdik at MPLab, identified that I had previous experience and interest in EO. Therefore, both our fields of interests matched. This led to him describing a real-life challenge where there was waste being dumped illegally by the banks of the river Danube, and it was getting difficult for the government to track the locations of these illegal dumps by visual supervision. On reading further about this menace, I realised that it was not only in Hungary, but governments worldwide have been struggling to track illegal waste by manual inspection. Since Deep Learning or Computer Vision was not used before this for detection of illegal landfills, we thought there could not be a better opportunity to explore in this direction. This is how the idea of this project came to life. We wanted to automate the detection of landfills with sub-metre accuracy, so that the governments and municipalities don’t have to do it manually.

ESA: How did your studies help you to approach this project?

Anupama: My courses at TU, Berlin and ELTE, Budapest, were focused on Autonomous Systems. I had a range of courses like Robotics, Computer Vision -both in 2D and 3D, Sensor Fusion, Image and Video Processing and Machine and Deep Learning. Hence, I think this course structure allowed me to be able to make a system autonomous irrespective of the application or domain. The architectures that I used to solve the problem in my research have been extensively used in self-driving cars, but it worked equally well when applied to EO.

Therefore, I think the thorough focus on concepts in my studies helped me in applying what I learnt to different application areas. Also, since CV and ML is a very dynamic field of study, with new research emerging every day, we were encouraged to read and study the latest research papers. In my opinion, this really helped me in my analysis, as I could read about the latest research in EO and implement my project accordingly.

ESA: Any last thought on your overall experience?

Anupama: For me, working on this project was a very fulfilling experience as in the end I could successfully implement a proof of concept for automating the detection of illegal landfills. This could end up having deep environmental impact where humans won’t have to enter the hazardous or unsafe landfill areas, to manually monitor them. The possible practical applications of this project make me very happy.

However, being an engineer, it was the whole experience of approaching and solving the problem technically that made the experience worthwhile. For any machine learning algorithm, data forms a very crucial part of the process. With the wrong kind of data, even the best of the ML architectures would not give the right result. With so many machine learning frameworks available for new engineers to learn from, I feel the focus on data has been diminished. Students and researchers are usually encouraged to use some existing dataset and apply an existing ML algorithm on them.

I feel this is not how an ML problem should be approached, as it robs the new machine learning students and researchers from understanding the essence of the whole process. Data should be given the most importance. Through this project, I was a part of the entire process of choosing the data and creating the ground truth labels on my own. In the end, I had created my own dataset. This helped me when I was debugging and testing my algorithms, and I was aware of the ins and outs of the data as well as the algorithm. Hence, I feel this project was a holistic learning experience for me as a relatively new ML researcher.

Lastly, I would like to thank ESA, who with their large repository of satellite data of different nature and resolution have been playing a crucial role in driving the research in EO. This has led to new milestones being set every day and resulted in some cutting-edge research. The availability of such a huge volume of data at the disposal of students and researchers allows for endless opportunities in the field of EO.

This project would not have come to fruition had ESA-TPM not provided me with very high-resolution images from WorldView and GeoEye satellite programs, with resolution as high as 30 cm to create my dataset.

Read more about Anupama's project