AUTOMATIC ROAD EXTRACTION BASED ON INTEGRATION OF HIGH RESOLUTION LIDAR AND AERIAL IMAGERY

In recent years, the rapid increase in the demand for road information together with the availability of large volumes of high resolution Earth Observation (EO) images, have drawn remarkable interest to the use of EO images for road extraction. Among the proposed methods, the unsupervised fully-automatic ones are more efficient since they do not require human effort. Considering the proposed methods, the focus is usually to improve the road network detection, while the roads’ precise delineation has been less attended to. In this paper, we propose a new unsupervised fully-automatic road extraction method, based on the integration of the high resolution LiDAR and aerial images of a scene using Principal Component Analysis (PCA). This method discriminates the existing roads in a scene; and then precisely delineates them. Hough transform is then applied to the integrated information to extract straight lines; which are further used to segment the scene and discriminate the existing roads. The roads’ edges are then precisely localized using a projection-based technique, and the round corners are further refined. Experimental results demonstrate that our proposed method extracts and delineates the roads with a high accuracy.


INTRODUCTION
In recent years, various tasks such as traffic navigation and urban planning have been affected by the rapid growth of transportation networks, therefore, increasing the interest in road extraction methods in fields such as EO and Geographic Information System.Recent advances in EO imaging have made it possible to obtain high resolution aerial and satellite images which represent land cover, such as roads, in more detail.In order to use this large quantity of data for road extraction while avoiding the costs of the manual processing, developing efficient automatic road extraction methods are in high demand.In order to automatize road extraction, previous works have followed two main directions, namely semi-automatic and fully-automatic methods.While semi-automatic methods require human interaction (Miao et al., 2014, Anil andNatarajan, 2010), most of the proposed fully automatic methods require human supervision as a ground truth either for training or parameter estimation (Shi et al., 2014, Samadzadegan et al., 2009).Since both human interaction and ground truth generation requires considerable human effort, several unsupervised fully-automatic methods have been recently introduced (Bae et al., 2015, Singh andGarg, 2014).While in many recent works, the focus is to improve road network detection, the roads' precise delineation has been studied to a lesser degree.However, in various applications such as road map updating, a precise determination of the roads' components (e.g., edges) is essential.
In this paper, we propose a new method for road extraction and precise delineation, using the integration of a LiDAR image; and the red, green, and blue (RGB) bands of a high resolution aerial image.While the LiDAR image provides objects' height information, the aerial image represents their color properties and textures from a bird's eye view.Figure 1 shows the overview of our proposed method.As a first step, in order to integrate the height and color information, we use the PCA (Pearson, 1901, Hotelling, 1933) method and convert the 4 information bands (i.e., R, G, B, height) into 4 principal components.Among the resulting components, we select the one which provides the largest distinction between the roads and their neighboring objects.In the next step, a Hough transform (Hough, 1962) is applied to the selected component in order to extract straight lines.The lines are then intersected to segment the scene into different regions.Using the LiDAR and the aerial images, we verify whether the segments contain road parts, and discard the ones which do not.To this end, since the roads' color and height are usually more homogeneous than their surrounding objects (e.g., buildings), we suppose that the pixel value deviation of the segments containing road parts is smaller than those of the other segments.Thus, we discard the segments with the pixel value deviations larger than a certain threshold.Additionally, using the LiDAR image, the segments with average height values larger than a certain threshold are removed.
After extracting the road segments, we merge the segments which belong to the same road.The points within each segment are then projected on a perpendicular plane to the segment's main orientation, so that the majority of the points are able to generate the road's profile.The profile is generated for both the height and the gray value of the aerial image, in which the road edges are supposed to be the points with a significant change in either height or gray value.The road's edges are then detected by integrating the two profiles (height and gray value), and taking the derivative of the result.The maximum points of the resulting derivative localize the roads' edges.As a final step, the edges at the round corners, which usually occur at the road intersections, are refined.Experimental results demonstrate that our proposed method both detects the road network, and delineates the road edges with a high accuracy.
The rest of this paper is organized as follows: Section 2., provides an introduction to Hough transform.Section 3., presents our proposed method in detail.Section 4., discusses our experimental results, and Section 5., concludes this paper.Figure 1: The overview of our proposed method; in which the process within the green rectangle is repeated for every road segment.

HOUGH TRANSFORM
Hough transform is an image feature extraction method usually used to detect simple structures such as straight lines, circles or ellipses (Hough, 1962, Duda andHart, 1972).The Hough transform is employed when, due to the image properties or the edge detection performance, grouping the extracted edges into a structure (e.g., straight line) is not possible.It considers a set of parametrized structure candidates.It then groups the image edges (for example, the ones extracted by Canny edge detector) into an appropriate structure candidate through a voting procedure.Assume detecting a straight line is required.Each line is defined by the slope parameter α and the y-intercept parameter b, y = αx + b.However, in practice, since the infinite value of the slope parameter for the vertical lines causes computational burden, using Hesse normal form of the line is suggested in (Duda and Hart, 1972) as the following, where ρ is the distance of the line from origin, and θ is the angle between the x-axis and the line's normal vector which passes through origin (θ ]).Using this line form, any point in the x-y space is equal to a sinusoidal curve in ρ-θ space.In order to detect the existing straight lines, Hough transform quantizes the two parameters (ρ and θ), and for each quantization considers a bin in a histogram.Then for every pixel, it tries to find evidences of a straight line and assign it to one of the histogram bins.In a next step, the bins with higher values are selected as the parameters of the existing straight lines.Finally, since the line lengths are not determined (as an essential step) the lines are localized on the image.

METHODOLOGY
The main contribution of our paper is the extraction of road edges in urban area by integrating high resolution LiDAR and aerial imagery.Figure 1 shows the overview of our proposed method.In the following, we explain every step in detail.

Apply PCA to the Images
In order to extract the existing roads in a scene, the first step is to discriminate the roads from the other objects.Thus, we integrate the LiDAR and aerial images in order to make them more discriminative.To this end, we apply PCA to the 4 available information bands (e.g., R, G, B, and height) to convert them into 4 principal components, where each component has a unique property.Among them, we empirically find out that the second component discriminates the roads from the rest of the scene better than the others.Therefore, we use it as the input for the road extraction step.

Decompose the Image into Segments
In urban areas, roads are usually in the form of straight lines.Using this assumption, we apply Hough transform to the PCA's second component in order to extract the existing straight lines in the scene.We then elongate these lines up to the scene borders in order to segment the scene (Figure 2.a).Among the segments, some are aligned with the roads and contain road parts.To find these segments (which contain part of a road), we assess all the segments in the next step.

Discard Irrelevant Segments
In this step, we perform two strategies in order to discard the segments which contain no road parts.The first strategy is based on the homogeneity of the segments in the aerial image.In order to measure the homogeneity, we compute the pixel values' average standard deviation.Since roads are usually homogeneous in color, the standard deviation of the segments which contain road parts are supposed to be smaller than those of the other segments.Thus we discard the segments with standard deviation larger that a certain threshold.Figure 2.b presents the remaining segments, and as depicted, most of the irrelevant segments are removed according to their homogeneity.However, there are segments which contain homogeneous objects, except roads, covered almost by grass and partially by a homogeneous house roof.In order to handle these situations, we apply a second strategy which uses the height information taken from the LiDAR image.According to how we segment the scene, the segments containing road parts usually have a lower average height than the ones containing other homogeneous objects.Thus, we discard the segments with the average height larger that a threshold.Figure 2.c shows the remaining segments after applying the second strategy.
According to the figure, although most of the irrelevant segments are removed, there are still segments which contain no road parts.These segments are homogeneous and their average height value is small; and are therefore, not detected by the two strategies.
In the next step, we propose a method to find and remove these segments.

Merge the Related Segments
In order to represent each road as a whole in a single segment, we find and merge the segments containing parts of the same road.
To this end, we extract the corner points of every segment using Douglas-Peucker line simplification algorithm (Douglas and Peucker, 2011).The corner points are considered as intersections of any two adjacent segments' edges with dissimilar orientations.Due to the roads' structure, we usually obtain four corner points for each segment.To merge the segments, we randomly select a segment and determine its main orientation as the orientation of its longest adjacent Hough line.Then we randomly take one of its edges (a line between a pair of corner points) which is not parallel to the segment's main orientation.Using the aerial image, the mean gray value of the pixels along the selected edge is then compared to that of the closest edge of its neighboring segment.If the edges' mean gray values are close enough, the segments are considered to be containing the same road; and thus, they are merged.This procedure is repeated until the main orientation of the newly merged segment is changed.The same procedure is repeated, by randomly selecting a segment from the remaining ones, until all the smaller segments are merged into larger ones.This results in connected segments representing the network structure of the roads.The results show that since the remaining irrelevant segments from the previous step do not contain a road part, they cannot connect to the discovered road network.Thus, we can detect and remove them.Figure 2.d shows the obtained segments.

Project Points within each Segment on a Plane
Although the resulting segments from the previous step localize the roads, the precise locations of the roads' edges are still missing.In order to delineate the roads accurately, in this step, we integrate the points within each segment along the segment's main orientation to project them on a perpendicular plane to the segment's main orientation.The idea behind this projection is that a majority of the points can represent the road's profile within a segment.In our experiments, we generate the roads' profiles using both LiDAR and aerial images.In the profiles, the roads' edges are the points where they experience a significant change either in height value (when LiDAR image is used), or the pixels' gray value (when the aerial image is used).
In order to increase the area for searching the road edges, we expand the segments to the sides (i.e., the perpendicular direction to the segment's main orientation) and increase each segment's width by 65%.Considering the LiDAR image (Figure 3 In order to localize the road edges as the points of significant changes, we integrate the two road profiles (height and gray value), and take the absolute value of the result's first derivative.The local maxima represent the places of significant change.Figure 3.e shows the computed derivative for the two road profiles.Due to the existence of objects with large contrasts either in height (e.g., parked cars) or in color (e.g., artifacts in sidewalks), the resulting derivate usually has more than two local maxima.In order to identify the ones which correspond to the road's edges, we take the 4 larger local maxima indicating the edge candidates' locations.In contrast to the road edges, which continue along the road, the high contrast objects occur only at some points.Therefore, in order to find and discard the irrelevant edge candidates, we randomly sample 50 points along each edge candidate.After that, we compute each point's gradient in an orientation perpendicular to that of its corresponding edge candidate.For each can-Figure 4: Refinement of the road edges at the round corners.In this figure, the magenta lines are the extracted straight road edges, the green lines are the search lines, the red points are the search results for the round corner edges, and the blue line is the curve fitted to the points determining the road's round corner edge.
Figure 5: Visualization of the extracted road network and the roads' delineation using our proposed method.
didate, the computed gradients are then averaged.Due to their noncontinuous effect, the high contrast objects' average gradient is usually smaller than that of the road edges.Thus, we localize the road edges by the two edge candidates with the largest average gradient.In order to increase the stability of our method in the presence of random selection, we run the experiments three times; and then consider the average result.

Refine Road Edges at Round Corners
Since our road extraction method (up to this step) is based on the straight lines, the round corners, especially at the road intersections are missed.As Figure 4 shows, the extracted road edges depicted by magenta lines do not fit the roads' round corners.In this step, in order to improve the road delineation at round corners, we look for local round edges at the roads' intersections.To this end, we split the angle between every two intersected road edges into 10 equal sized angles.Then we compute the gradient of the pixels' gray values along every smaller angles' side (depicted by green lines in Figure 4), up to a certain distance from the intersection point (70 pixels in our experiments).On each angle side, the point with the largest gradient value is then taken as a sample of the round edge (the red points in Figure 4).These points are then used to fit a quadratic polynomial which delineates the round corner (the blue curve in Figure 4).Table 1: RMS errors computed for our method's result after road discrimination based on the Hough transform, and after delineation of the roads and refinement of their round corners.

EXPERIMENTAL RESULTS AND DISCUSSION
In our experiments, we use images of an area in Zeebruges , Belgium, which were acquired on March 13th, 20111 .We use its high resolution LiDAR and RGB aerial images.The LiDAR image has a spatial resolution of 10 cm, while the aerial image's spatial resolution is 5 cm.Therefore, before using them in our experiments, we downsample the aerial image to have the same resolution as the LiDAR image.
Figure 5 shows the extracted road network and the roads' delineation by our proposed method.According to the results, our method is able to detect all the existing roads in the scene.Moreover, Figure 6 demonstrates our proposed method's delineation precision.In this figure, the green lines are the ground truth, the red lines are the segment boundaries obtained by Hough transform, and the blue lines are the road edges determined by our proposed method.The results show that using the projection-based technique for delineation of the roads, and refining the round corners, brings the extracted road edges close to their real location the ground truth.
For a quantitative evaluation, we compute the Root Mean Square (RMS) error of the roads' edges extracted by our method comparing to the ground truth, as proposed in (Heipke et al., 1997).
Table 1 shows the average RMS errors after extracting the road segments, and after delineating the roads and refining the round corners.As a further evaluation, we measure the average angular displacement of the extracted road edges from the ground truth, which is only 0.3 degrees.Considering the both evaluations, our method is able to precisely localize and delineate the roads in the scene.

CONCLUSION
In this paper, we propose an unsupervised automatic road extraction method which uses the integration of the LiDAR and RGB aerial image of a scene.As an integration technique, PCA is used on the 4 available information bands, namely R, G, B, and height.The most road discriminating principal component is then used in Hough transform to extract the possible locations of the roads as straight lines.Using these lines, the scene is segmented into various regions.The segments containing a road part are then detected and merged to shape the road network in the scene.The roads are then delineated using a projection-based method, and their round corners are further refined.For a quantitative evaluation, the extracted roads are compared to ground truth data using RMS and angular displacement measures.Experimental results demonstrate that our proposed method detects and delineates the roads precisely in a given scene.

Figure 2 :
Figure 2: Segmentation of the image using the Hough lines.(a) All the segments, (b) Segments after discarding the irrelevant ones based on their homogeneity.(c) The remaining segments after removing the ones with a high average height value, (d) The road segments after merging and discarding ones which are not connected.

Figure 3 :
Figure 3: Projection of the points within each segment on a plane perpendicular to the main orientation of the segment.(a) A road depicted on the LiDAR image, (b) The same road depicted on the aerial image, (c) The road profile obtained from the LiDAR image, (d) The road profile obtained from the aerial image, (d) The first derivative of the integrated road profiles.
.a) and the aerial image (Figure 3.b), we compute the height profile (Figure 3.c) and the gray value profile (Figure 3.d).

Figure 6 :
Figure 6: A closer look at some roads' delineations resulted by our proposed method (the blue lines); comparing to the ground truth (the green lines), and the segment boundaries computed by Hough transform (the red lines).Road Segments Delineation & Refinement RMS 5.75 2.57