Articles | Volume 21, issue 11
https://doi.org/10.5194/bg-21-2909-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/bg-21-2909-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
From simple labels to semantic image segmentation: leveraging citizen science plant photographs for tree species mapping in drone imagery
Salim Soltani
CORRESPONDING AUTHOR
Sensor-based Geoinformatics (geosense), University of Freiburg, Freiburg, Germany
Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI), Leipzig University, Leipzig, Germany
German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Leipzig, Germany
Olga Ferlian
German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Leipzig, Germany
Institute of Biology, Leipzig University, Leipzig, Germany
Nico Eisenhauer
German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Leipzig, Germany
Institute of Biology, Leipzig University, Leipzig, Germany
Hannes Feilhauer
Remote Sensing Centre for Earth System Research (RSC4Earth), Leipzig University, Leipzig, Germany
Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI), Leipzig University, Leipzig, Germany
German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Leipzig, Germany
Remote Sensing, Helmholtz Centre for Environmental Research, Leipzig, Germany
Teja Kattenborn
Sensor-based Geoinformatics (geosense), University of Freiburg, Freiburg, Germany
German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Leipzig, Germany
Related authors
No articles found.
Kevin Wolf, Evelyn Jäkel, André Ehrlich, Michael Schäfer, Hannes Feilhauer, Andreas Huth, Alexandra Weigelt, and Manfred Wendisch
Biogeosciences, 22, 2909–2933, https://doi.org/10.5194/bg-22-2909-2025, https://doi.org/10.5194/bg-22-2909-2025, 2025
Short summary
Short summary
This paper reports an investigation of the influence of clouds on vegetation albedo using a coupled atmosphere–vegetation radiative transfer model. Both models are iteratively linked to simulate cloud–vegetation–radiation interactions over canopies more realistically. Solar, spectral, and broadband irradiances have been simulated under varying cloud conditions. The simulated irradiances were used to investigate the spectral and broadband effect of clouds on vegetation albedo.
Simon Lotz, Teja Kattenborn, Julian Frey, Salim Soltani, Anna Göritz, Tom Jakszat, and Negin Katal
EGUsphere, https://doi.org/10.5194/egusphere-2025-1496, https://doi.org/10.5194/egusphere-2025-1496, 2025
This preprint is open for discussion and under review for Biogeosciences (BG).
Short summary
Short summary
Digital hemispherical photography (DHP) is a valuable tool for monitoring leaf area index (LAI), a key factor in ecosystem productivity and climate interactions. We compared DHP with litter traps in a temperate forest and found that at a 20° view angle, both methods aligned well. We applied a calibration model to assess site variability which significantly improved accuracy. Our findings enhance the reliability of ground-based LAI monitoring, supporting better ecosystem assessments.
Kevin Wolf, Evelyn Jäkel, André Ehrlich, Michael Schäfer, Hannes Feilhauer, Andreas Huth, and Manfred Wendisch
EGUsphere, https://doi.org/10.5194/egusphere-2025-2082, https://doi.org/10.5194/egusphere-2025-2082, 2025
Short summary
Short summary
This paper presents combined atmosphere-vegetation radiative transfer simulations to systematically investigate cloud-induced biases in remotely sensed vegetation indices (VIs) derived from below-cloud measurements. The biases in VIs have been investigated for the general case of two-band VIs, and for the special cases of the normalized difference vegetation index (NDVI), the normalized difference water index (NDWI), and the enhanced vegetation index (EVI).
Eya Cherif, Teja Kattenborn, Luke A. Brown, Michael Ewald, Katja Berger, Phuong D. Dao, Tobias B. Hank, Etienne Laliberté, Bing Lu, and Hannes Feilhauer
EGUsphere, https://doi.org/10.5194/egusphere-2025-1284, https://doi.org/10.5194/egusphere-2025-1284, 2025
Short summary
Short summary
Hyperspectral imagery combined with machine learning enables accurate large-scale mapping of plant traits but struggles with uncertainty when facing unfamiliar environmental conditions. This study introduces a distance-based method that measures dissimilarities between new and training data to reliably quantify uncertainty. Results show it effectively identifies uncertain predictions, greatly improving the reliability of global vegetation monitoring compared to traditional methods.
Salim Soltani, Lauren E. Gillespie, Moises Exposito-Alonso, Olga Ferlian, Nico Eisenhauer, Hannes Feilhauer, and Teja Kattenborn
EGUsphere, https://doi.org/10.5194/egusphere-2025-662, https://doi.org/10.5194/egusphere-2025-662, 2025
Short summary
Short summary
We introduce an automated approach for generating segmentation masks for citizen science plant photos, making them applicable to computer vision models. This framework effectively transforms citizen science data into a data treasure for segmentation models for plant species identification in aerial imagery. Using automatically labeled photos, we train segmentation models for mapping tree species in drone imagery, showcasing their potential for forestry, agriculture, and biodiversity monitoring.
Simon Scheiter, Sophie Wolf, and Teja Kattenborn
Biogeosciences, 21, 4909–4926, https://doi.org/10.5194/bg-21-4909-2024, https://doi.org/10.5194/bg-21-4909-2024, 2024
Short summary
Short summary
Biomes are widely used to map vegetation patterns at large spatial scales and to assess impacts of climate change, yet there is no consensus on a generally valid biome classification scheme. We used crowd-sourced species distribution data and trait data to assess whether trait information is suitable for delimiting biomes. Although the trait data were heterogeneous and had large gaps with respect to the spatial distribution, we found that a global trait-based biome classification was possible.
Cited articles
Bayraktar, E., Basarkan, M. E., and Celebi, N.: A low-cost UAV framework towards ornamental plant detection and counting in the wild, ISPRS J. Photogramm., 167, 1–11, https://doi.org/10.1016/j.isprsjprs.2020.06.012, 2020. a
Bouguettaya, A., Zarzour, H., Kechida, A., and Taberkit, A. M.: Deep learning techniques to classify agricultural crops through UAV imagery: A review, Neural Comput. Appl., 34, 9511–9536, 2022. a
Braga, G., J. R., Peripato, V., Dalagnol, R., P. Ferreira, M., Tarabalka, Y., OC Aragão, L. E., F. de Campos Velho, H., Shiguemori, E. H., and Wagner, F. H.: Tree crown delineation algorithm based on a convolutional neural network, Remote Sens., 12, 1288, https://doi.org/10.3390/rs12081288, 2020. a
Brandt, M., Tucker, C. J., Kariryaa, A., Rasmussen, K., Abel, C., Small, J., Chave, J., Rasmussen, L. V., Hiernaux, P., Diouf, A. A., Kergoat, L., Mertz, O., Igel, C., Gieseke, F., Schöning, J., Li, S., Melocik, K., Meyer, J., Sinno, S., Romero, E., Glennie, E., Montagu, A., Dendoncker, M., and Fensholt, R.: An unexpectedly large count of trees in the West African Sahara and Sahel, Nature, 587, 78–82, https://doi.org/10.1038/s41586-020-2824-5, 2020. a
Brodrick, P. G., Davies, A. B., and Asner, G. P.: Uncovering ecological patterns with convolutional neural networks, Trends Ecol. Evol., 34, 734–745, https://doi.org/10.1016/j.tree.2019.03.006, 2019. a
Chandler, M., See, L., Copas, K., Bonde, A. M., López, B. C., Danielsen, F., Legind, J. K., Masinde, S., Miller-Rushing, A. J., Newman, G., Rosemartin, A., and Turak, E.: Contribution of citizen science towards international biodiversity monitoring, Biol. Conserv., 213, 280–294, https://doi.org/10.1016/j.biocon.2016.09.004, 2017. a
Cherif, E., Feilhauer, H., Berger, K., Dao, P. D., Ewald, M., Hank, T. B., He, Y., Kovach, K. R., Lu, B., Townsend, P. A., and Kattenborn, T.: From spectra to plant functional traits: Transferable multi-trait models from heterogeneous and sparse data, Remote Sens. Environ., 292, 113580, https://doi.org/10.1016/j.rse.2023.113580, 2023. a
Cloutier, M., Germain, M., and Laliberté, E.: Influence of Temperate Forest Autumn Leaf Phenology on Segmentation of Tree Species from UAV Imagery Using Deep Learning, bioRxiv, 2023–08, https://doi.org/10.1101/2023.08.03.548604, 2023. a, b, c
Curnick, D. J., Davies, A. J., Duncan, C., Freeman, R., Jacoby, D. M., Shelley, H. T., Rossi, C., Wearn, O. R., Williamson, M. J., and Pettorelli, N.: SmallSats: a new technological frontier in ecology and conservation?, Remote Sensing in Ecology and Conservation, 8, 139–150, https://doi.org/10.1002/rse2.239, 2021. a
De Sa, N. C., Castro, P., Carvalho, S., Marchante, E., López-Núñez, F. A., and Marchante, H.: Mapping the flowering of an invasive plant using unmanned aerial vehicles: is there potential for biocontrol monitoring?, Front. Plant Sci., 9, 293, https://doi.org/10.3389/fpls.2018.00293, 2018. a
Di Cecco, G. J., Barve, V., Belitz, M. W., Stucky, B. J., Guralnick, R. P., and Hurlbert, A. H.: Observing the observers: How participants contribute data to iNaturalist and implications for biodiversity science, BioScience, 71, 1179–1188, https://doi.org/10.1093/biosci/biab093, 2021. a, b, c
Fassnacht, F. E., Latifi, H., Stereńczak, K., Modzelewska, A., Lefsky, M., Waser, L. T., Straub, C., and Ghosh, A.: Review of studies on tree species classification from remotely sensed data, Remote Sens. Environ., 186, 64–87, https://doi.org/10.1016/j.rse.2016.08.013, 2016. a, b, c
Ferlian, O., Cesarz, S., Craven, D., Hines, J., Barry, K. E., Bruelheide, H., Buscot, F., Haider, S., Heklau, H., Herrmann, S., Kühn, P.,Pruschitzki, U., Schädler, M., Wagg, C., Weigelt, A., Wubet, T., and Eisenhauer, N.: Mycorrhiza in tree diversity–ecosystem function relationships: conceptual framework and experimental implementation, Ecosphere, 9, e02226, https://doi.org/10.1002/ecs2.2226, 2018. a
Fraisl, D., Hager, G., Bedessem, B., Gold, M., Hsing, P.-Y., Danielsen, F., Hitchcock, C. B., Hulbert, J. M., Piera, J., Spiers, H., Thiel, M., and Haklay, M.: Citizen science in environmental and ecological sciences, Nature Reviews Methods Primers, 2, 64, https://doi.org/10.1038/s43586-022-00144-4, 2022. a
Fricker, G. A., Ventura, J. D., Wolf, J. A., North, M. P., Davis, F. W., and Franklin, J.: A convolutional neural network classifier identifies tree species in mixed-conifer forest from hyperspectral imagery, Remote Sens., 11, 2326, https://doi.org/10.3390/rs11192326, 2019. a
Galuszynski, N. C., Duker, R., Potts, A. J., and Kattenborn, T.: Automated mapping of Portulacaria afra canopies for restoration monitoring with convolutional neural networks and heterogeneous unmanned aerial vehicle imagery, PeerJ, 10, e14219, https://doi.org/10.7717/peerj.14219, 2022. a, b
GBIF: GBIF: the global biodiversity information facility, 2019. a
Hoeser, T. and Kuenzer, C.: Object detection and image segmentation with deep learning on earth observation data: A review-part i: Evolution and recent trends, Remote Sens., 12, 1667, https://doi.org/10.3390/rs12101667, 2020. a
Ivanova, N. and Shashkov, M.: The possibilities of GBIF data use in ecological research, Russ. J. Ecol., 52, 1–8, 2021. a
Johnston, A., Matechou, E., and Dennis, E. B.: Outstanding challenges and future directions for biodiversity monitoring using citizen science data, Methods in Ecol. Evol., 14, 103–116, https://doi.org/10.1111/2041-210X.13834, 2023. a
Kattenborn, T. and Soltani, S.: CrowdVision2TreeSegment, Zenodo [data set], https://doi.org/10.5281/zenodo.10019552, 2023. a, b
Kattenborn, T., Eichel, J., and Fassnacht, F. E.: Convolutional Neural Networks enable efficient, accurate and fine-grained segmentation of plant species and communities from high-resolution UAV imagery, Sci. Rep., 9, 1–9, https://doi.org/10.1038/s41598-019-53797-9, 2019a. a
Kattenborn, T., Lopatin, J., Förster, M., Braun, A. C., and Fassnacht, F. E.: UAV data as alternative to field sampling to map woody invasive species based on combined Sentinel-1 and Sentinel-2 data, Remote Sens. Environ., 227, 61–73, https://doi.org/10.1016/j.rse.2019.03.025, 2019b. a
Kattenborn, T., Schiefer, F., Frey, J., Feilhauer, H., Mahecha, M. D., and Dormann, C. F.: Spatially autocorrelated training and validation samples inflate performance assessment of convolutional neural networks, ISPRS Open Journal of Photogrammetry and Remote Sensing, 5, 100018, https://doi.org/10.1016/j.ophoto.2022.100018, 2022. a
Leitão, P. J., Schwieder, M., Pötzschner, F., Pinto, J. R., Teixeira, A. M., Pedroni, F., Sanchez, M., Rogass, C., van der Linden, S., Bustamante, M. M. and Hostert, P.: From sample to pixel: multi-scale remote sensing data for upscaling aboveground carbon data in heterogeneous landscapes, Ecosphere, 9, e02298, https://doi.org/10.1002/ecs2.2298, 2018. a
Lopatin, J., Fassnacht, F. E., Kattenborn, T., and Schmidtlein, S.: Mapping plant species in mixed grassland communities using close range imaging spectroscopy, Remote Sens. Environ., 201, 12–23, https://doi.org/10.1016/j.rse.2017.08.031, 2017. a
Lopatin, J., Dolos, K., Kattenborn, T., and Fassnacht, F. E.: How canopy shadow affects invasive plant species classification in high spatial resolution remote sensing, Remote Sens. Ecol. Conserv., 5, 302–317, https://doi.org/10.1002/rse2.109, 2019. a, b
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., and Johnson, B. A.: Deep learning in remote sensing applications: A meta-analysis and review, ISPRS J. Photogramm., 152, 166–177, https://doi.org/10.1016/j.isprsjprs.2019.04.015, 2019. a
Mäder, P., Boho, D., Rzanny, M., Seeland, M., Wittich, H. C., Deggelmann, A., and Wäldchen, J.: The flora incognita app–interactive plant species identification, Methods in Ecol. Evol., 12, 1335–1342, https://doi.org/10.1111/2041-210X.13611, 2021. a
Maes, W. H. and Steppe, K.: Perspectives for remote sensing with unmanned aerial vehicles in precision agriculture, Trends Plant Sci., 24, 152–164, https://doi.org/10.1016/j.tplants.2018.11.007, 2019. a
Milas, A. S., Arend, K., Mayer, C., Simonson, M. A., and Mackey, S.: Different colours of shadows: Classification of UAV images, Int. J. Remote Sens., 38, 3084–3100, https://doi.org/10.1080/01431161.2016.1274449, 2017. a
Molls, C.: The Obs-Services and their potentials for biodiversity data assessments with a test of the current reliability of photo-identification of Coleoptera in the field, Tijdschrift voor Entomologie, 164, 143–153, 2021. a
Müllerová, J., Brundu, G., Große-Stoltenberg, A., Kattenborn, T., and Richardson, D. M.: Pattern to process, research to practice: remote sensing of plant invasions, Biol. Invasions, 25, 3651–3676, https://doi.org/10.1007/s10530-023-03150-z, 2023. a
Ronneberger, O., Fischer, P., and Brox, T.: U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer, 234–241, https://doi.org/10.1007/978-3-319-24574-4_28, 2015. a, b
Rzanny, M., Mäder, P., Deggelmann, A., Chen, M., and Wäldchen, J.: Flowers, leaves or both? How to obtain suitable images for automated plant identification, Plant Methods, 15, 1–11, https://doi.org/10.1186/s13007-019-0462-4, 2019. a
Schiefer, F., Kattenborn, T., Frick, A., Frey, J., Schall, P., Koch, B., and Schmidtlein, S.: Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks, ISPRS J. Photogramm., 170, 205–215, https://doi.org/10.1016/j.isprsjprs.2020.10.015, 2020. a, b, c, d, e, f, g
Schiefer, F., Schmidtlein, S., Frick, A., Frey, J., Klinke, R., Zielewska-Büttner, K., Junttila, S., Uhl, A., and Kattenborn, T.: UAV-based reference data for the prediction of fractional cover of standing deadwood from Sentinel time series, ISPRS Open Journal of Photogrammetry and Remote Sensing, 8, 100034, https://doi.org/10.1016/j.ophoto.2023.100034, 2023. a
Schmitt, M., Prexl, J., Ebel, P., Liebel, L., and Zhu, X. X.: Weakly supervised semantic segmentation of satellite images for land cover mapping–challenges and opportunities, arXiv [preprint], https://doi.org/10.48550/arXiv.2002.08254, 2020. a
Soltani, S., Feilhauer, H., Duker, R., and Kattenborn, T.: Transfer learning from citizen science photographs enables plant species identification in UAVs imagery, ISPRS Open Journal of Photogrammetry and Remote Sensing, 5, 100016, https://doi.org/10.1016/j.ophoto.2022.100016, 2022. a, b, c, d, e, f, g, h, i
Sun, Z., Wang, X., Wang, Z., Yang, L., Xie, Y., and Huang, Y.: UAVs as remote sensing platforms in plant ecology: review of applications and challenges, J. Plant Ecol., 14, 1003–1023, https://doi.org/10.1093/jpe/rtab089, 2021. a, b
Tan, M. and Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks, in: International conference on machine learning, 6105–6114, PMLR, Long Beach, California, 10–15 June 2019, https://doi.org/10.48550/arXiv.1905.11946, 2019. a
van Der Velde, M., Goëau, H., Bonnet, P., d’Andrimont, R., Yordanov, M., Affouard, A., Claverie, M., Czúcz, B., Elvekjær, N., Martinez-Sanchez, L., and Rotllan-Puig, X.: Pl@ ntNet Crops: merging citizen science observations and structured survey data to improve crop recognition for agri-food-environment applications, Environ. Res. Lett., 18, 025005, https://doi.org/10.1088/1748-9326/acadf3, 2023. a
Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S.: The inaturalist species classification and detection dataset, in: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, USA 18–22 June 2018, 8769–8778, 2018. a, b
Van Horn, G., Cole, E., Beery, S., Wilber, K., Belongie, S., and Mac Aodha, O.: Benchmarking Representation Learning for Natural World Image Collections, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, virtual, 19–25 June 2021, https://doi.org/10.48550/arXiv.2103.16483, 12884–12893, 2021. a
Wagner, F. H.: The flowering of Atlantic Forest Pleroma trees, Sci. Rep., 11, 1–20, https://doi.org/10.1038/s41598-021-99304-x, 2021. a
Zhou, Z.-H.: A brief introduction to weakly supervised learning, Natl. Sci. Rev., 5, 44–53, https://doi.org/10.1093/nsr/nwx106, 2018. a
Short summary
In this research, we developed a novel method using citizen science data as alternative training data for computer vision models to map plant species in unoccupied aerial vehicle (UAV) images. We use citizen science plant photographs to train models and apply them to UAV images. We tested our approach on UAV images of a test site with 10 different tree species, yielding accurate results. This research shows the potential of citizen science data to advance our ability to monitor plant species.
In this research, we developed a novel method using citizen science data as alternative training...
Altmetrics
Final-revised paper
Preprint