publications by categories in reversed chronological order. generated by jekyll-scholar.
2024
ECCVW’24
Underwater Uncertainty: A Multi-Annotator Image Dataset for Benthic Habitat Classification
Galadrielle Humblot-Renaux, Anders Skaarup Johansen, Jonathan Eichild Schmidt, Amanda Frederikke Irlind, Niels Madsen, Thomas B. Moeslund, and Malte Pedersen
Continuous inspection and mapping of the seabed allows for monitoring the impact of anthropogenic activities on benthic ecosystems. Compared to traditional manual assessment methods which are impractical at scale, computer vision holds great potential for widespread and long-term monitoring. We deploy an underwater remotely operated vehicle (ROV) in Jammer Bay, a heavily fished area in the Greater North Sea, and capture videos of the seabed for habitat classification. The collected JAMBO dataset is inherently ambiguous: water in the bay is typically turbid which degrades visibility and makes habitats more difficult to identify. To capture the uncertainties involved in manual visual inspection, we employ multiple annotators to classify the same set of images and analyze time spent per annotation, the extent to which annotators agree, and more. We then evaluate the potential of vision foundation models (DINO, OpenCLIP, BioCLIP) for automating image-based benthic habitat classification. We find that despite ambiguity in the dataset, a well chosen pre-trained feature extractor with linear probing can match the performance of manual annotators when evaluated in known locations. However, generalization across time and place is an important challenge.
@inproceedings{Humblot-Renaux_2024_ECCVW,title={Underwater Uncertainty: A Multi-Annotator Image Dataset for Benthic Habitat Classification},author={Humblot-Renaux, Galadrielle and Johansen, Anders Skaarup and Schmidt, Jonathan Eichild and Irlind, Amanda Frederikke and Madsen, Niels and Moeslund, Thomas B. and Pedersen, Malte},booktitle={Computer Vision -- ECCV 2024 Workshops},year={2024},eprint={https://vbn.aau.dk/files/738329849/JAMBO_ECCV_vbn.pdf}}
CVPR’24
A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?
Galadrielle Humblot-Renaux, Sergio Escalera, and Thomas B. Moeslund
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2024
The ability to detect unfamiliar or unexpected images is essential for safe deployment of computer vision systems. In the context of classification, the task of detecting images outside of a model’s training domain is known as out-of-distribution (OOD) detection. While there has been a growing research interest in developing post-hoc OOD detection methods, there has been comparably little discussion around how these methods perform when the underlying classifier is not trained on a clean, carefully curated dataset. In this work, we take a closer look at 20 state-of-the-art OOD detection methods in the (more realistic) scenario where the labels used to train the underlying classifier are unreliable (e.g. crowd-sourced or web-scraped labels). Extensive experiments across different datasets, noise types & levels, architectures and checkpointing strategies provide insights into the effect of class label noise on OOD detection, and show that poor separation between incorrectly classified ID samples vs. OOD samples is an overlooked yet important limitation of existing methods. Code: https://github.com/glhr/ood-labelnoise
@inproceedings{Humblot-Renaux_2024_CVPR,title={A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?},author={Humblot-Renaux, Galadrielle and Escalera, Sergio and Moeslund, Thomas B.},booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},year={2024},eprint={https://openaccess.thecvf.com/content/CVPR2024/html/Humblot-Renaux_A_Noisy_Elephant_in_the_Room_Is_Your_Out-of-Distribution_Detector_CVPR_2024_paper.html}}
2023
CVPRW’23
Beyond AUROC & Co. for Evaluating Out-of-Distribution Detection Performance
Galadrielle Humblot-Renaux, Sergio Escalera, and Thomas B. Moeslund
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , Jun 2023
While there has been a growing research interest in developing out-of-distribution (OOD) detection methods, there has been comparably little discussion around how these methods should be evaluated. Given their relevance for safe(r) AI, it is important to examine whether the basis for comparing OOD detection methods is consistent with practical needs. In this work, we take a closer look at the go-to metrics for evaluating OOD detection, and question the approach of exclusively reducing OOD detection to a binary classification task with little consideration for the detection threshold. We illustrate the limitations of current metrics (AUROC & its friends) and propose a new metric - Area Under the Threshold Curve (AUTC), which explicitly penalizes poor separation between ID and OOD samples. Scripts and data are available at https://github.com/glhr/beyond-auroc
@inproceedings{Humblot-Renaux_2023_CVPRW,author={Humblot-Renaux, Galadrielle and Escalera, Sergio and Moeslund, Thomas B.},title={Beyond AUROC \& Co. for Evaluating Out-of-Distribution Detection Performance},booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},doi={10.1109/CVPRW59228.2023.00402},month=jun,year={2023},pages={3881-3890},}
2022
ICRA’22
Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning to Segment Driveability in Egocentric Images
Galadrielle Humblot-Renaux, Letizia Marchegiani, Thomas B. Moeslund, and Rikke Gade
@article{Humblot-Renaux_2021_RAL,author={Humblot-Renaux, Galadrielle and Marchegiani, Letizia and Moeslund, Thomas B. and Gade, Rikke},journal={IEEE Robotics and Automation Letters},title={Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning to Segment Driveability in Egocentric Images},year={2022},volume={7},number={2},pages={2913-2920},doi={10.1109/LRA.2022.3144491},}