<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "https://jats.nlm.nih.gov/nlm-dtd/publishing/3.0/journalpublishing3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="3.0" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="publisher">ISPRS-Archives</journal-id>
<journal-title-group>
<journal-title>The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences</journal-title>
<abbrev-journal-title abbrev-type="publisher">ISPRS-Archives</abbrev-journal-title>
<abbrev-journal-title abbrev-type="nlm-ta">Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.</abbrev-journal-title>
</journal-title-group>
<issn pub-type="epub">2194-9034</issn>
<publisher><publisher-name>Copernicus Publications</publisher-name>
<publisher-loc>Göttingen, Germany</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.5194/isprs-archives-XLVIII-2-W3-2023-137-2023</article-id>
<title-group>
<article-title>MONITORING OF DYNAMIC OBJECTS ON A 2D OCCUPANCY MAP USING NEURAL NETWORKS AND MULTIMODAL DATA</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Krishtopik</surname>
<given-names>A. S.</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Yudin</surname>
<given-names>D. A.</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
</contrib>
</contrib-group><aff id="aff1">
<label>1</label>
<addr-line>Moscow Institute of Physics and Technology (National Research University), Moscow, 141701, Russia</addr-line>
</aff>
<aff id="aff2">
<label>2</label>
<addr-line>Federal Research Center Computer Science and Control, Russian Academy of Sciences, Moscow, 119333, Russia</addr-line>
</aff>
<pub-date pub-type="epub">
<day>12</day>
<month>05</month>
<year>2023</year>
</pub-date>
<volume>XLVIII-2/W3-2023</volume>
<fpage>137</fpage>
<lpage>143</lpage>
<permissions>
<copyright-statement>Copyright: &#x000a9; 2023 A. S. Krishtopik</copyright-statement>
<copyright-year>2023</copyright-year>
<license license-type="open-access">
<license-p>This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this licence, visit <ext-link ext-link-type="uri"  xlink:href="https://creativecommons.org/licenses/by/4.0/">https://creativecommons.org/licenses/by/4.0/</ext-link></license-p>
</license>
</permissions>
<self-uri xlink:href="https://isprs-archives.copernicus.org/articles/XLVIII-2-W3-2023/137/2023/isprs-archives-XLVIII-2-W3-2023-137-2023.html">This article is available from https://isprs-archives.copernicus.org/articles/XLVIII-2-W3-2023/137/2023/isprs-archives-XLVIII-2-W3-2023-137-2023.html</self-uri>
<self-uri xlink:href="https://isprs-archives.copernicus.org/articles/XLVIII-2-W3-2023/137/2023/isprs-archives-XLVIII-2-W3-2023-137-2023.pdf">The full text article is available as a PDF file from https://isprs-archives.copernicus.org/articles/XLVIII-2-W3-2023/137/2023/isprs-archives-XLVIII-2-W3-2023-137-2023.pdf</self-uri>
<abstract>
<p>The paper deals with the construction of dynamic occupancy maps, where the grid cell can contain not only information about the presence or absence of an obstacle, but also information about its velocity. We propose a multimodal approach to constructing 2D dynamic occupancy maps from LiDAR point clouds and camera images. The approach involves building a static occupancy map from LiDAR data and then adding information about cell velocities based on neural network instance segmentation and object tracking in monocular onboard camera images. Pedestrians and vehicles were considered as dynamic objects. One of the important stages is the projection of the object masks found in the images onto a 2D occupancy map. We compared the proposed approach with the classical method of constructing dynamic occupancy maps from LiDAR data based on the Monte Carlo method. An experimental quality evaluation of the approaches was carried out using the popular Mapillary Vistas and Waymo Open Datasets containing street scenes and a large number of dynamic objects. We demonstrate that the considered approaches can work in real time, which indicates the possibility of their application as part of the on-board control systems of autonomous cars or ground mobile robots. The software implementation of the proposed dynamic occupancy reconstruction approach is publicly available at the link: https://github.com/andrey1908/nn-dynamic-occupancy.</p>
</abstract>
<counts><page-count count="7"/></counts>
</article-meta>
</front>
<body/>
<back>
</back>
</article>