The high versatility and affordable price made Unmanned Aerial Vehicles (UAVs) very popular in many civil applications. Despite their usage often takes advantage of automatic flight mode, this flight modeality is usually limited to the good working conditions of Global Navigation Satellite Systems (GNSS) positioning. Extending their operability to cases when GNSS is not available or reliable, including for instance departure and landing in not open-sky areas, is clearly of great importance for the drone market. This paper aims at investigating the performance of a vision-based system to monitor, and support, if needed, UAV movements, within some tens of meters from a ground camera, used to track the UAV. In this work, a uncalibrated camera is used to track the UAV: UAV detection on the camera frames is implented with a background subtraction approach, even if neural network-based approaches can be used as well. Then, the UAV centroid on the camera frames along with its area in pixels are used as inputs for the machine learning predictors. While using an uncalibrated camera is clearly suboptimal in terms of performance, it eases the usage of the proposed method in for non-expert operators. The proposed approach is tested both on a synthetic and a real dataset, collected at the Agripolis campus of the University of Padua, in order to determine whether the performance is limited by the size of available training dataset. The results reported in this work show that the usage of tree ensamble regression can lead to submeter errors in tracking a UAV with a ground camera, when the UAV is at less than approximately 100 meters from the camera.