DATASET FOR POSITIONING AND TRACKING CARS AND PEDESTRIANS FROM UAV IMAGERY AND STATIC

: The development of autonomous vehicles, both terrestrial and aerial ones (unmanned aerial system (UAS)), is causing the need of properly formulating appropriate solutions for ensuring a safe interaction between them, human beings and the infrastructures and environment in their operating area. To such aim, the knowledge of the positions of different platforms moving in the considered area is fundamental. GNSS (Global Navigation Satellite System) is by far the most used positioning technique in order to determine positions all over the world. Nevertheless, there are several conditions in which its use is unfortunately impossible or unreliable. Hence, different techniques, based on the use of sensors either mounted on the moving platforms or on an ad-hoc infrastructure, shall be used in order to determine the absolute and relative positions of the involved platforms. To this aim, this work proposes the use of vision, in particular from UAS imagery, static LiDAR (Light Detection and Ranging) and UWB (Ultra Wide-Band) transceivers, with initial encouraging results.


INTRODUCTION
The number of Unmanned Aerial Vehicles (UAVs) civil applications is continuously increasing thanks to their flexibility of usage.In this work their use is proposed to support traffic monitoring, vehicle and pedestrian tracking and collaborative navigation, along with the use of a properly developed infrastructure, i.e. a static LiDAR (Light Detection and Ranging) and a set of static UWB (Ultra Wide-Band) transceivers, ensuring Vehicle-to-Infrastructure (V2I) ranging (and, in particular, Pedestrian-to-Infrastructure ranging).UWB Vehicle-to-Vehicle (V2V) ranging, i.e.UWB ranging and communications between moving platforms, is also taken into account, in particular in the Pedestrian-to-Pedestrian case.
The aim of this work is twofold: • first, traffic monitoring and tracking can be useful to study car driver and pedestrian behaviors, in particular when interacting with each other, which can be of fundamental importance in order to develop safe and reliable autonomous driving solutions (Puri et al., 2007, Khan et al., 2017, Kanistras et al., 2013, Coifman et al., 2006) (generating datasets to this aim has already been considered by some research groups, such as in (Krajewski et al., 2018, Bock et al., n.d., Krajewski et al., 2020)).Furthermore, crowd monitoring can support public event organizers and decision makers to properly deal with certain emergency conditions.
• On the other hand, despite the availability of affordable GNSS (Global Navigation Satellite System) receivers enabled effective outdoor navigation on consumer devices * Corresponding author almost everywhere, there are a number of critical conditions where GNSS-based navigation does not provide an accurate and reliable solution, such as indoors, in urban canyons and tunnels (Nistér et al., 2004, Konolige et al., 2010, Howard, 2008, Forster et al., 2014, Gurturk et al., 2021, Masiero and Vettore, 2016).Mitigating the unreliability and/or compensating the unavailability of a GNSSbased solution usually involves the integration of information provided by several sensors (Grejner-Brzezinska et al., 2016, El-Sheimy et al., 2006), often including inertial sensors (El-Sheimy and Youssef, 2020), magnetometer and radio signals (Zhuang et al., 2016, Dabove et al., 2018, Li et al., 2018, Adegoke et al., 2019, Sakr et al., 2020), in order to reach a trustworthy solution (de Groot et al., 2018, Hsu et al., 2015, Zeng et al., 2017).
This work is part of a project, conducted as a collaboration between the ISPRS WG I/2 "Mobile Mapping Technology" and the IAG WG 4.1.4"Computer Vision in Navigation", aiming at both assessing the accuracy of vision based positioning and navigation.More specifically, this paper focuses on a data collection campaign made in order to investigate UAV and static LiDAR-based pedestrian and ground vehicle tracking, and also to exploit this information in a collaborative navigation approach (Yao et al., 2011, Alam and Dempster, 2013, Ansari, 2019, Masiero et al., 2021).In accordance with the aims of this project, all the involved devices, mounted on aerial platforms, ground vehicles and pedestrians, are assumed to be connected: thanks to their communication abilities, each moving platform can exploit the information shared by the others when computing its own solution.
Vision-based positioning and tracking techniques are quite popular, in particular for what concerns visual simultaneous localization and mapping (SLAM) techniques (Mur-Artal et al., 2015, Leonard and Durrant-Whyte, 1991, Strasdat et al., 2012, Whelan et al., 2016).Furthermore, LiDAR can also be exploited in order to both determine the ego-motion of a platform (Zhang andSingh, 2014, Zhang andSingh, 2017) and to assess movement of other platforms and persons (Masiero et al., 2022).
This paper will present the conducted data collection campaign and some initial results related to pedestrian and car tracking, which can be used either as stand-alone solutions or complementary information when GNSS is available.

CASE STUDY
A three-day data collection test, including three pedestrians, two UAS platforms, one terrestrial vehicle was organized at the Agripolis Campus of the University of Padua.All the terrestrial platforms were provided of reliable positioning systems, GNSS based, with corrections from a permanent base station at distance ≤ 150 m.Additionally, all the moving platforms were provided with a camera, in video acquisition mode, and an UWB transceiver (see Fig. 1).Some targets, properly surveyed (at few centimeter-level of accuracy), have also been distributed on the ground, mostly in correspondence with the UWB anchor locations, to be used as reference ground information for the UAVs, as shown in Fig. 2.  In the portion of the dataset considered in this paper, the static LiDAR was mounted on a building close to the case study area, in order to have a quite view of the scene (Fig. 3).Finally, a UAV monitored the scene flying at few tens of meters of altitude from the ground (Fig. 4 shows a top view of most of the test area).
Fig. 5 shows an example of the pedestrian tracks (5-minute long example).

PRELIMINARY RESULTS
First, UWB ranging has been tested, comparing UWB measurments with reference distances computed by means of GNSS.For what concerns Pedestrian-to-Pedestrian UWB ranging, Fig. 6 provides the results of such comparison.It is worth to notice that, despite some outliers are quite visible, the two curves are reasonably similar.The success percentage of ranging is also a very important factor, remarkably impacting in a trilateration-based positioning system.Hence, Fig. 7 and Fig. 8 show the success rates as a function of the distance between the considered devices in the Pedestrian-to-Pedestrian and in the Pedestrian-to-Infrastructure case.Such two figures show a higher success rate for the Pedestrian-to-Pedestrian case in the (0-30) m interval, whereas a much larger maximum range is obtained in the Pedestrian-to-Infrastructure case.In an initial assessment of the UWB only solution for pedestrian tracking, it allowed to obtain a median 2D positioning error at meter level, which is more than expected (see (MacGougan et al., 2009)), but probably caused by the presence of many metallic objects (e.g.cars) in the scene.
Instead, the results of LiDAR-based tracking were at decimeter accuracy level, when available.Actually, solution was not available on those parts of the tracks outside of the LiDAR fieldof-view.
Finally, UAV vision was used for car tracking, ensuring a reliable reference solution by fixing a geodetic GNSS receiver on the top of the vehicle.In order to study the driver behavior it is quite important in this case to assess the vehicle speed, which is reported in Fig. 9 (results are reported for a 30 s interval), where the reference speed (blue solid line) is compared with the estimated speed (red dot marks), with an average discrepancy of around 0.1 m/s.

CONCLUSIONS
This paper shows a portion of the data collected in data collection campaign aiming at providing a dataset useful to investigate the combined use of different sensors in order to track pedestrians and terrestrial vehicles.To such aim, different kind of sensors, both static (on the ground and from a quite high point of view) and mounted on moving platforms (UAV and carried by the pedestrians) have been considered, including radio (UWB), vision (standard RGB camera) and LiDAR.
The initial results show that there is a quite decent potential in the use of the considered sensors.Nevertheless, each of them has some strengths and weaknesses, e.g.LiDAR and camera imagery can ensure very good positioning results, but only when the tracked object/person is in the sensor field-of-view.Despite radio connection is also quite limited at large distances, overall it allowed to obtain much more continuous solutions, i.e. with much less gaps.
Our future investigations will aim at a more in depth comparison between the performance that can be obtained with these sensors, at investigating their combined use, improve the collaborative aspect of the collected dataset, focusing in particular in the development of a scalable, non-centralized positioning approach (Pascacio et al., 2021, Mu et al., 2011, Kerr, 1987, Steinmetz et al., 2019), and at implementing also some machine/deep learning based techniques in order to detect and recognize objects, obstacles, pedestrians and other vehicles in the neighborhood of the each vehicle.

Figure 2 .
Figure 2. Example of target and Pozyx UWB anchor.

Figure 3 .
Figure 3. LiDAR view of the case study area.

The
Figure 4. Top view of the test area.

Figure 7 .
Figure 7. Pedestrian-to-pedestrian UWB ranging: success rate as a function of the distance.

Figure 8 .
Figure 8. Pedestrian-to-infrastructure UWB ranging: success rate as a function of the distance.

Figure 9 .
Figure 9.Comparison between the vision-based estimated car speed (red dot marks) and the GNSS-based reference velocity (blue solid line).