PHOTOGRAMMETRIC POINT CLOUD CLASSIFICATION BASED ON GEOMETRIC AND RADIOMETRIC DATA INTEGRATION

Abstract

The extraction of information from point cloud is usually done after the application of classification methods based on the geometric characteristics of the objects. However, the classification of photogrammetric point clouds can be carried out using radiometric information combined with geometric information to minimize possible classification issues. With this in mind, this work proposes an approach to the classification of photogrammetric point cloud, generated by correspondence of aerial images acquired by Remotely Piloted Aircraft System (RPAS). The proposed approach for classifying photogrammetric point clouds consists of a pixel-supervised classification method, based on a decision tree. To achieve this, three data sets were used, one to define which attributes allow discrimination between the classes and the definition of the thresholds. Initially, several attributes were extracted based on a training sample. The average and standard deviation values for the attributes of each class extracted were used to guide the decision tree definition. The defined decision tree was applied to the other two point clouds to validate the approach and for thematic accuracy assessment. The quantitative analyses of the classifications based on kappa coefficient of agreement, applied to both validation areas, reached values higher than 0.938.

Keywords Classification; Photogrammetric Point Cloud; RPAS

1. Introduction

Aerial images, LiDAR point cloud and photogrammetric point cloud are indispensable data sources for studies of urban environment such as LULC (Land Use and Land Cover) mapping (Zhang and Lin, 2017; Rau et al., 2015; Ali-Sisto and Packalen, 2017), building modelling (Haala and Kada, 2010), and Digital Terrain Model (DTM) extraction (Sithole and Vosselman, 2003; Sithole and Vosselman, 2005; Susaki, 2012), etc.

Vosselman (2013) considers that 3D point cloud classification is one of the most important steps in the information extraction process. However, despite the researcher’s efforts to develop automatic classification procedures, obtaining accurate results remains as a challenge (Gilani et al., 2015). The algorithms must be robust to deal with the several objects present in the scenes, both natural (vegetation and hydrography, for example) and anthropic (buildings, power lines, etc.).

According to Zhang and Lin (2017), images and LiDAR point clouds are the two major data sources for the photogrammetry and remote sensing community. This availability creates the need for new tools to integrate such data since images and LiDAR point clouds have different but complementary characteristics. The complementarity of these two data sources is of great importance in several areas, especially for classification in urban environments, as mentioned by Hartfield et al. (2011) and Man et al. (2015).

The scene complexity in urban environments is one of the main factors making it difficult to discriminate objects, mainly by automatic methods. In addition, the lack of information or occlusions and the characteristics of the equipment are further aspects that hamper the classification process (Sohn and Dowman, 2007).

The synergy between radiometric and geometric data has motivated the integration of images and LiDAR point clouds aiming at exploring the potential of these combined data (Hermosilla et al., 2011; Buján et al., 2012; Malpica et al., 2013). Recent advances in the development of photogrammetric platforms, sensors and photogrammetric software have enabled the exploration of the potential of information extraction from photogrammetric point clouds (Rau et al., 2015; and Ali-Sisto and Packalen, 2017). As a result of these developments, photogrammetric point clouds that integrate geometric and radiometric information have emerged as an alternative for the integration of images and LiDAR point cloud.

Hartfield et al. (2011) can be mentioned as one example of integration. The authors explored the feasibility of integrating remotely sensed multispectral reflectance data and LiDAR-derived height information to improve land use and land cover classification and used classification and regression trees to analyze the data. Man et al. (2015) focused on the performance of fused lidar and hyperspectral data for urban land-use classification, especially the contribution of lidar intensity and height information, for land-use classification in shadow areas using Support Vector Machine (SVM), Maximum Likelihood Classification (MLC) and object-based classifiers. Both studies investigated the contribution of hyperspectral and LASER data integration in information extraction by means of robust classifiers. In this work, we explore the potential of classifiers based on hierarchical decision trees in the attributes extracted from photogrammetric point clouds obtained by multispectral cameras with a limited number of bands.

The development of RPAS concomitant with the popularization of Global Navigation Satellite System (GNSS) receivers, inertial navigation systems, and off-the-shelf imaging sensors have boosted the generation of terrestrial surface geospatial information (Nex and Remondino, 2014). In addition, the development of algorithms for image processing and computer vision allows the extraction of three-dimensional information from images with a reduced number of human operations.

Although the development of RPAS has enabled major advances in data acquisition, the payload capacity sets limitations to the amount and quality of the embedded sensors (Nex and Remondino, 2014 and Colomina and Molina, 2014). In the case of point clouds resulting from the photogrammetric processing of images obtained by RPAS, the availability of radiometric information can vary as a function of the sensors embedded, ranging from few bands to many in the case of hyperspectral sensors, as can be seen in Aasen et al. (2015) and Näsi et al. (2018). Thus, urban area classification based on data obtained by RPAS is a challenge. In these cases, the study and determination of the attributes necessary to minimize classification errors are important. For this, the classes must be defined based on a previous study of the scene and the attributes that allow the discrimination between the classes must be defined based on the data available.

Considering the previous studies presented, we propose a classification approach based on a hierarchical decision tree. From one point cloud, different criteria were used to determine the attributes and thresholds enabling the discrimination of different objects present in an urban environment: Fiber cement tiles, Clay tiles, Bare soil, Asphalt, Grass, Trees, Shaded asphalt, Shaded grass and High shadow. Then, based on the attributes, two other point cloud were classified, aiming to estimate the quality of the classification. The approach uses a point cloud generated from aerial images acquired by RPAS, combining geometric and radiometric data.

2. Experimental areas and data description

The data used in this work correspond to photogrammetric point clouds generated from two aerial surveys. The first was carried out in the Espigão district, in the city of Regente Feijó - SP, in the year 2016 (called Mission 1). The second aerial survey was carried out in the city of Ribeirão dos Índios - SP, in the year 2015 (called Mission 2).

An RPAS Sensefly eBee equipped with a positioning system based on GNSS receivers and Micro Electro Mechanical Systems (MEMS) inertial systems was used for both missions. In addition, two dual frequency GNSS receivers were used to obtain 3D coordinates of Ground Control Points (GCP) and check points.

The flight plans were developed with Sensefly Emotion2 software using the parameters shown in Table 1. A Cannon S110 NIR (Near Infra-Red) digital camera with focal length of 4.40 mm was used to acquire the images, whose sensitivity curves of the Green, Red and NIR bands, with central wavelengths positioned at wavelengths 550 nm, 625 nm and 850 nm, respectively, are shown in Fig. 1.

Table 1:
Flight parameter configuration for image acquisition.

Mission	Flight height (m)	GSD* (cm)	Along-track overlap (%)	Cross-track overlap (%)
1	157	5.5	80	70
2	140	4.0	80	70

• *GSD - Ground Sample Distance.

Figure 1:
Sensitivity curves of the Green, Red and NIR bands (wavelength x percentage of response). Source: Canon S110 RGB/NIR/RE User Manual (2018).

The photogrammetric processing was carried out using Pix4D Mapper Pro software which employs a sequence of operations known as Structure from Motion (SfM), which consists of the measurement of photogrammetric points of interest, bundle block adjustment and dense image matching for point cloud generation. The following reference is suggested for additional details related to the point cloud generation: Pix4Dmapper software manual (2018). GCP and check points were used in both missions to guarantee and assess the accuracy of the results. Eight GCP and six check points were used in processing mission 1, reaching RMSE of 0.10 m in planimetry and 0.12 m in altimetry. Eight GCP and ten check points were used in processing mission 2, reaching RMSE of 0.08 m in planimetry and 0.11 m in altimetry.

Following the Photogrammetric processing of the RPAS images, a set of 3D coordinates and n radiometric information is associated with each point of the clouds, where n is the number of bands of the imaging sensor used to acquire the images. In this work the imaging sensor captured three bands: Green, Red and NIR.

Two clippings were extracted for the processing of the point cloud resulting from mission 1, Area t (Fig. 2 (a)) and Area 1 (Fig. 2 (b)). In the second mission, another point cloud was generated, named Area 2 in Fig. 2 (c). Details of the extracted clippings are presented in Table 2. It should be noted that the low point density of Area 2 is the result of photogrammetric overlay failures caused by bad weather conditions at the time the images were acquired.

The three selected point clouds clippings comprise urbanized areas in which most of the buildings are covered with clay tiles or fiber cement tiles. In addition, it is possible to observe trees on the sidewalks, ground areas covered by undergrowth and paved roads (asphalt). As indicated in Fig. 2, the first cropped area (Area t) was used to extract the sample for the training of the classification algorithm, while samples were obtained from the other clippings (Area 1 and Area 2) for validation of the results.

Figure 2:
Point clouds in false color composition. (a) algorithm training area (Area t); (b) validation Area 1; and (c) validation Area 2.

Table 2:
Clipping photogrammetric point clouds parameters.

Mission	Point cloud	Average density (points/m²)	Nº points	Area (m²)
1	Area t	26.6	2862758	110020
	Area 1	25.9	5340739	206400
2	Area 2	4.6	1054572	230256

3. Proposed classification method

The classification aims to determine in which class or category a given pixel/point should be included. The classification methods can be divided according to the a priori knowledge of the study area (supervised or unsupervised), the metrics used (parametric or non-parametric), or the analysis method considered (per pixel or by region) (Jensen, 2005). Alternatively, they may use radiometric spectral data only, geometric data, or both.

The proposed approach consists of a per pixel supervised classification method, based on decision tree, where a sequence of thresholds is applied, aiming to discriminate previously defined classes. It was necessary to analyze the available data to identify land cover types present in the area and to collect reference samples for further training of the classifier (Jensen, 2005). The training data allowed the estimation of a set of metrics and thresholds which were used as the basis for the discrimination and classification of the other pixels or regions of the scene. The per pixel approaches use only pixel data, disregarding the neighborhood data and the relationship between neighboring pixels (Jensen, 2005). The decision tree aiming to classify the point cloud was implemented in C++ using Code::Blocks development environment.

The first step of the method consists in the definition of the classes and the acquisition of training data. The definition of the classes should be based on a previous study of the objects present in the scene. For each class defined, a data set should be acquired for training and to guide the attribute and threshold determination step. The attributes and threshold determination consider the definition of which attributes provide the discrimination of the predicted classes. It should be emphasized that the determination of the attributes and thresholds is carried out in an empirical way and details on this step are presented on Section 3.2. Thus, the attributes and thresholds defined are applied in a hypercritical structure in order to provide the data classification. Finally, the generated decision tree was applied to two data sets for validation. The accuracy of the classification approach was evaluated based on the Kappa coefficient of agreement and omission and commission errors.

The 3D coordinates and the values of the R, G and NIR were associated with each point of the point cloud generated from the RPAS images. Based on this information, the attributes and the thresholds used in the decision tree were defined. In the following two sections the classes, the attributes and thresholds are defined, based on the analysis performed considering the data from the training area (Area t).

3.1 Specification of the classes and training data acquisition

Based on a previous study of the experimental areas, the Fiber cement tiles, Clay tiles, Bare soil, Asphalt, Grass and Trees classes were defined. As well as the classes mentioned, it was necessary to define three classes of shade: Shaded asphalt, Shaded grass and High shadow. To evaluate the attributes relevant for the class discrimination, the first cropped area (Area t) was used, from which a total of 46 regions were selected, as shown in Fig. 3. The distribution of the sample elements was based on the visual interpretation of the image, in order to ensure that all classes were sampled. From this area, a total of 43904 points were selected and the number of points extracted per class is shown in Table 3.

Figure 3:
Distribution of training sample points collected for attribute and threshold estimation.

Table 3:
Number of of points per class sampled for the classifier training.

Class	Fiber Cement Tiles	Clay Tiles	Asphalt	Bare Soil	Grass	Trees	Shaded Grass	Shaded Asphalt	High Shadow
points	3017	2546	4479	3140	17800	6367	4164	1781	610

3.2 Features and thresholds determination

For each class, the average and standard deviation per spectral band were estimated from the training data. The average and standard deviation per band considering the nine classes proposed is presented in Fig. 4.

The proximity between the average values of the different classes for the three bands in Fig. 4 indicates the similarity between the classes considered. It can be verified that the Fiber cement tiles and Clay tiles classes show high dispersion and can be confused with other classes. There is also great confusion among Grass, Trees, Fiber cement tiles, Clay tiles and Bare soil classes.

The difficulty in classifying the data using only the available spectral responses of the objects is clearly evident. In this case, one alternative is to apply arithmetic of bands techniques to generate attributes that enable discrimination between the classes.

Figure 4:
Attribute average and standard deviation graphs (y-axis contains the normalized Digital Number (DN). (a) Results regarding Red band; (b) Results regarding Green band; and (c) Results regarding NIR band.

The discrimination between vegetation classes (Grass and Trees) and building classes (Fiber cement tiles and Clay tiles) can be done from a vegetation index, such as normalized difference vegetation index (NDVI - Eq. 1). However, the application of shadow indexes proposed in the literature is not achievable because of the non-availability of spectral bands needed to compute this index (Ma et al., 2008; and Polidoro et al., 2003). The geometric height attribute may assist in discriminating between objects belonging to the terrain and elevated objects. Radiometric attributes were extracted in this way based on the radiometry according to Eq(s). 1, 2, 3 and 4. The geometric height attribute was generated from the application of tools present in the rapidlasso LASTools package, which uses a modified version of the filtering algorithm proposed by Axelsson (2000), specifically lasground and lasheight.

$$NDVI=\frac{NIR-R}{NIR+R}$$ (1)

$$M=Maximum(R,G,NIR)$$ (2)

$$m=Minimum(R,G,NIR)$$ (3)

$$S=\frac{M-m}{M+m}$$ (4)

in which R is the value in the Red band; G is the value in the Green band; NIR is the value in the NIR band; M is the highest value between Red, Green and NIR bands; m is the lowest value between Red, Green and NIR bands; and S is the ratio determined from M-m and M+m.

The motivation to include the attribute S is based in some color models, as in the HSL model for example (see Pedrini and Schwartz, 2008), in which H is the hue, S is the saturation and L is the lightness, and where the saturation is estimated from equations similar to Eq(s). 2 and 3 considering the R, G and B bands.

The average and standard deviation were estimated for all classes regarding the following attributes: height, NDVI, M, m and S. The average and standard deviation were plotted in graphs presented in Fig. 5. As can be seen in Fig. 5(d), the highest S values correspond to the three classes in which shadow is present, i. e., shaded grass, shaded asphalt and high shadow.

Figure 5:
Attribute average and standard deviation graphs (y-axis contains the possible values for the attributes). (a) attribute Height; (b) attribute NDVI; (c) attribute m; and (d) attribute S.

3.3 Definition of the decision tree and thresholds estimation

One hierarchical supervised decision tree aiming to classify the objects was defined based on the proposed attributes, as summarized in flowchart of Fig. 6. Initially, height information is used to distinguish whether points belong to the terrain or not. Two sets of points result from this step: points belonging to terrain (Grass, Asphalt, among others) and elevated points (Buildings, Trees, among others). These two sets of points are processed individually to obtain the final classification. In Fig. 5 (a) it is possible to verify that the use of height above the ground enables discrimination of the terrain and non-terrain objects by adopting a threshold (t _height) of 0.7 m. Points that have values lower than this threshold will be considered as terrain points (belonging to the Grass, Asphalt, Bare soil, Shaded grass or Shaded asphalt classes). The others will be considered elevated points (belonging to the Fiber cement tiles, Clay tiles, Trees or High shadow classes).

The Shadow classes are discriminated from the other classes, considering the responses to the m attribute and the threshold (t _m) of 0.30 (Fig. 5 (c)). Despite the high deviation of this attribute for some classes, the t _m threshold adopted can discriminate the Shadow classes from other classes. In the set of elevated points, points below the threshold (t _m = 0.30) are classified as High shadow points. Considering terrain points that are below the threshold applied to the m attribute, the algorithm classifies the points as Shaded grass or Shaded asphalt by the application of a threshold (t _NIR) of 0.25 for the attribute NIR (Fig. 4 (c)). In this way, points above that threshold are classified as Shaded grass or as Shaded asphalt.

For vegetation discrimination, the response of the two vegetation classes (Trees and Grass) to NDVI values (Fig. 5(b)) and a threshold (t _NDVI) of 0.10 were considered. Non-shaded terrain points above the threshold are classified as Grass and the non-shaded elevated points above the threshold are classified as Trees.

Terrain points above the shadow threshold and below the NDVI threshold are differentiated as Asphalt and Bare soil. Considering the values for the NIR attribute (Fig. 4 (c)) and a threshold (t _NIR) of 0.50, points above this threshold are classified as Bare soil and the others as Asphalt. The elevated points above the shadow threshold (t _m) and below the NDVI threshold (t _NDVI) are differentiated considering the values presented for attribute S (Fig. 5 (d)) and the threshold (t _S) of 0.09. Points above this threshold are classified as Clay tiles, the others as Fiber cement tiles. In Fig. 6 the decision tree designed is shown.

Figure 6:
Decision tree designed for the classification of urban land cover classes from the integration of radiometric and geometric data.

4. Results of validation experiments

The point clouds selected from both areas, Areas 1 and 2, were used to perform the classification validation based on the thresholds presented in Section 3.3 determined from the graphs in Fig. 5 and Fig. 4. By quantitative analysis of the classification accuracy, three regions per class were sampled in both areas. Table 4 shows the total number of points sampled per class for these two areas. Both point clouds were submitted to the same classification algorithm and the same thresholds mentioned. These classified point clouds are presented in Fig. 7.

Table 4:
Total validation sample point number per class.

Class	Fiber cement tiles	Clay tiles	Asphalt	Bare soil	Grass	Trees	Shaded grass	Shaded asphalt	High shadow
Area 1	2095	1108	2144	1573	2742	2540	2292	626	1458
Area 2	200	327	319	567	771	197	157	162	101

Figure 7:
Classification results. (a) orthophoto of Area 1 (left), Area 1 classified point cloud (right); and (b) orthophoto of Area 2 (left), Area 2 classified point cloud (right).

Although nine classes were initially considered, some objects that were not initially foreseen in the predicted classes were mistakenly included in predicted classes. For example, points of the sidewalk regions and the pavement traffic markings of streets and roads were included in the Bare soil class. Errors in the filtering process responsible for extracting the MDT and giving subsidy for estimating the heights of points have led to some classification problems, such as the classification of some points belonging to the streets in the classes of Fiber cement tiles and Clay tiles. Some of the aforementioned errors can be seen in Fig. 8.

Figure 8:
Examples of regions where confusion occurred in the classification. (a) the occurrence of sidewalk points included in the Bare Soil class; and (b) occurrence of points belonging to the streets and included in the class Fiber cement tiles.

The reliability of classification results is mainly dependent on the input data, the method used and the complexity of the scene. It is therefore necessary to know at least one indication of the accuracy obtained in the classification process which can be obtained by the error matrix, as indicated by Congalton and Green (2009). It consists of a matrix arrangement of numbers that expresses the number of sample data classified in each of the classes.

In order to analyze the accuracy of the classification quantitatively, the two sets of validation points presented in Table 4 were submitted to the classification algorithm and the results were compared with the reference classes known a priori. Finally, the error matrices (Tables 5 and 6) were constructed, referring to both areas (1 and 2).

Several metrics can be extracted from the error matrix, among them: Kappa coefficient of agreement (Eq.5) and omission and commission errors (Eqs. 6 and 7). The error matrix is an appropriate subsidy for several analyses as mention by Congalton and Green (2009).

$$\widehat{\text{K}\mathrm{}}\text{=}\frac{\text{n}\mathrm{}\sum _{\text{i=1}}^{\text{c}}{C}_{i,i}\text{-}\sum _{\text{i=1}}^{\text{c}}{C}_{i,+}*C_{+,i}}{{\text{n}}^{\text{2}}\text{-}\sum _{\text{i=1}}^{\text{n}}{C}_{i,+}*C_{+,i}}$$ (5)

$${omission}_j=\frac{\left(\sum _{i=1}^c{C}_{i,j}\right)-C_{j,j}}{\sum _{i=1}^n{C}_{i,j}}$$ (6)

$$c{omission}_i=\frac{\left(\sum _{j=1}^c{C}_{i,j}\right)-C_{i,i}}{\sum _{j=1}^c{C}_{i,j}}$$ (7)

in which i and j are respectively row and column indices of the error matrix elements; C_i,j is the element of the error matrix located at line i and column j; n is the total number of points analyzed; and c is the number of classes.

Table 5:
Error matrix relative to Area 1 classification.

	Fiber cement tiles	Clay tiles	Asphalt	Bare soil	Grass	Trees	Shaded grass	Shaded asphalt	High shadow	Total
Fiber cement tiles	2091	1	70	0	0	0	0	0	60	2222
Clay tiles	1	1009	0	0	0	0	0	0	9	1019
Asphalt	0	0	2045	3	0	0	11	0	0	2059
Bare soil	0	0	2	1566	3	0	0	0	0	1571
Grass	0	0	0	4	2728	0	20	0	0	2752
Trees	3	91	0	0	5	2481	0	0	3	2583
Shaded grass	0	0	0	0	6	0	2250	107	22	2385
Shaded asphalt	0	0	26	0	0	0	11	497	0	544
High shadow	0	7	1	0	0	59	0	12	1364	1443
Total	2095	1108	2144	1573	2742	2540	2292	626	1458	16578

Table 6:
Error matrix relative to Area 2 classification.

	Fiber cement tiles	Clay tiles	Asphalt	Bare soil	Grass	Trees	Shaded grass	Shaded asphalt	High shadow	Total
Fiber cement tiles	200	1	7	0	0	1	0	0	1	210
Clay tiles	0	281	0	0	0	0	0	0	0	281
Asphalt	0	0	310	0	0	0	0	14	0	324
Bare soil	0	45	2	564	2	0	0	1	0	614
Grass	0	0	0	3	768	0	8	0	1	780
Trees	0	0	0	0	1	195	0	0	5	201
Shaded grass	0	0	0	0	0	0	148	40	2	190
Shaded asphalt	0	0	0	0	0	0	0	98	0	98
High shadow	0	0	0	0	0	1	1	9	92	103
Total	200	327	319	567	771	197	157	162	101	2801

The kappa coefficient of agreement of the classification was around 0.962 for Area 1 and 0.938 for Area 2. However, some classification errors occur in classes that can be grouped, as is the case of Shaded grass and Shaded asphalt classes, in both experiments. Also based on the error matrix presented in Tables 5 and 6, the omission and commission errors were calculated and the results are presented in Table 7.

Table 7:
Omission and Commission error for Areas 1 and 2.

	Area 1		Area 2
Class	Omission	Commission	Omission	Commission
Fiber cement tiles	0.002	0.059	0.000	0.048
Clay tiles	0.089	0.010	0.141	0.000
Asphalt	0.046	0.007	0.028	0.045
Bare soil	0.004	0.003	0.005	0.081
Grass	0.005	0.009	0.004	0.015
Trees	0.023	0.039	0.010	0.030
Shaded grass	0.018	0.057	0.057	0.221
Shaded asphalt	0.190	0.078	0.395	0.000
High shadow	0.064	0.055	0.089	0.107

The omission and commission errors show that the Shaded grass and Shaded asphalt classes have achieved errors greater than 19% for the data in Area 1 and greater than 39.5% for the data in Area 2. For Area 2 the relatively higher error may be explained by possible differences in illumination considering the moment of image acquisition. The attribute used to separate Shaded grass and Shaded asphalt is directly related to the brightness of the scene. So, in more brightly lit scenes, the tendency is for points belonging to the Shaded asphalt class to be included in the Shaded grass class, and in less brightly lit scenes the tendency is for points belonging to the Shaded grass class to be included in the Shaded asphalt class.

Also, based on the error matrices presented in Tables 5 and 6, it can be seen that a small percentage of points belonging to the Asphalt class were erroneously included in the Fiber cement tiles class. Such inclusion can be a consequence of problems during the filtering stage and estimation of the Height attribute. The same can be said of the small confusion between Shaded grass and Shaded asphalt classes with the High shadow class.

5. Conclusion

It was observed from the quantitative analysis of thematic accuracy that the proposed classification method presents confusion among some classes. However, grouping some classes into more general classes, such as grouping the Shaded grass and Shaded asphalt classes into a Shaded terrain class, an improvement in overall classification accuracy is observed.

The use of radiometric information and the possibility of using a radiometric index allows the extraction of some classes, for example the detection of points in shaded regions and the separation of vegetation points from other classes.

The geometric data played a fundamental role in the separation of land and object points, that can be further processed independently. It is important to consider that the classification of the points is also a function of the altimetric accuracy of the photogrammetric cloud that is dependent on the dense matching process. In the validation experiments, considering Area 1 and Area 2, based on the thresholds determined in one independent data set, the Kappa coefficient of agreement of the classification was around 96.2% and 93.8%, respectively.

Future work should investigate alternatives to the classification. The geometric data can be further used to assist in the classification process, but as previously mentioned there are noises resulting from difficulties in the matching process. An alternative would be to perform a treatment of the cloud seeking to eliminate these noises. For this purpose, some alternatives can be considered: applying noise filtering to reduce the mentioned noises and then consider other attributes as those based on entropy and principal components analysis.

Acknowledgments

The authors would like to thank the Coordination for the Improvement of Higher Education Personnel - CAPES, for the grant to the first author (Process 1481349); the São Paulo Research Foundation - FAPESP (2014/01841-1); and also National Council for Scientific and Technological Development - CNPq (Process 304189/2016-2) for supporting this research.

REFERENCES

• Aasen, H., Burkart, A., Bolten, A. and Bareth, G. 2015. Generating 3D hyperspectral information with lightweight UAV snapshot cameras for vegetation monitoring: From camera calibration to quality assurance. ISPRS Journal of Photogrammetry and Remote Sensing, 108, pp.245-259.
• Ali-Sisto, D. and Packalen, P. 2017. Forest change detection by using point clouds from dense image matching together with a LiDAR-derived terrain model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(3), pp.1197-1206.
• Axelsson, P. 2000. DEM generation from laser scanner data using adaptive TIN models. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 33, pp.110-117.
• Buján, S., González‐Ferreiro, E., Reyes‐Bueno, F., Barreiro‐Fernández, L., Crecente, R. and Miranda, D. 2012. Land use classification from LiDAR data and ortho-images in a rural area. The Photogrammetric Record, 27(140), pp.401-422.
• Canon S110 RGB/NIR/RE User Manual (2018). Retrieved from: http://95.110.228.56/documentUAV/camera%20manual/%5BENG%5D_2014_user_manual_s110_v3.pdf
  » http://95.110.228.56/documentUAV/camera%20manual/%5BENG%5D_2014_user_manual_s110_v3.pdf
• Colomina, I. and Molina, P. 2014. Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS Journal of Photogrammetry and Remote Sensing, 92, pp.79-97.
• Congalton, R. G. and Green, K. 2009. Assessing the Accuracy of Remotely Sensed Data: Principles and Practices.
• Gilani, S., Awrangjeb, M. and Lu, G. 2015. Fusion of LIDAR data and multispectral imagery for effective building detection based on graph and connected component analysis. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 40(3), pp.65-72.
• Haala, N. and Kada, M. 2010. An update on automatic 3D building reconstruction. ISPRS Journal of Photogrammetry and Remote Sensing. 65, pp.570-580.
• Hartfield, K. A., Landau, K. I. and Van Leeuwen, W. J. D. 2011. Fusion of high resolution aerial multispectral and LiDAR data: land cover in the context of urban mosquito habitat. Remote Sensing, 3(11), p.2364-2383.
• Hermosilla, T., Ruiz, L. A., Recio, J. A. and Estornell, J. 2011. Evaluation of automatic building approaches combining high resolution images and LiDAR data. Remote Sensing, 3(6), pp.1188-1210.
• Jensen, J. R. 2005. Introductory digital image processing: a remote sensing perspective. 3. ed. New Jersey: Prentice Hall.
• Ma, H., Qin, Q. and Shen, X. 2008. Shadow segmentation and compensation in high resolution satellite images. In: Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008. IEEE International. IEEE, pp.1036-1039.
• Malpica, J. A., Alonso, M. C., Papí, F., Arozarena, A. and Martínez De Agirre, A. 2013. Change detection of buildings from satellite imagery and LiDAR data. International Journal of Remote Sensing, 34(5), pp.1652-1675.
• Man, Q., Dong, P. and Guo, H. 2015. Pixel-and feature-level fusion of hyperspectral and lidar data for urban land-use classification. International Journal of Remote Sensing, 36(6), pp.1618-1644
• Näsi, R., Honkavaara, E., Blomqvist, M., Lyytikäinen-Saarenmaa, P., Hakala, T., Viljanen, N., Kantola, T. and Holopainen, M. 2018. Remote sensing of bark beetle damage in urban forests at individual tree level using a novel hyperspectral camera from UAV and aircraft. Urban Forestry & Urban Greening, 30, pp.72-83.
• Nex, F. and Remondino, F. 2014. UAV for 3D mapping applications: a review. Applied geomatics, 6(1), pp.1-15.
• Pedrini, H. and Schwartz, W. R. 2008. Análise de Imagens Digitais: princípios, algoritmos e aplicações. São Paulo: Thomson Learning. 508 p.
• Pix4Dmapper software manual 2018. Retrieved from: https://support.pix4d.com/hc/en-us/articles/204272989-Offline-Getting-Started-and-Manual-pdf-
  » https://support.pix4d.com/hc/en-us/articles/204272989-Offline-Getting-Started-and-Manual-pdf-
• Polidorio, A. M., Flores, F. C., Imai, N. N., Tommaselli, A. M. and Franco, C. 2003. Automatic shadow segmentation in aerial color images. In: Computer Graphics and Image Processing, 2003. SIBGRAPI 2003. XVI Brazilian Symposium on. IEEE. pp.270-277.
• Rau, J. Y., Jhan, J. P. and Hsu, Y. C. 2015. Analysis of oblique aerial images for land cover and point cloud classification in an urban environment. IEEE Transactions on Geoscience and Remote Sensing, 53(3), pp.1304-1319.
• Sithole, G. and Vosselman, G. 2003. Report: ISPRS comparison of filters. ISPRS commission III, Working Group 3.
• Sithole, G. and Vosselman, G. 2005. Filtering of airborne laser scanner data based on segmented point clouds. In: International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 36(3), pp. W19.
• Sohn, G. and Dowman, I. 2007. Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction. ISPRS Journal of Photogrammetry and Remote Sensing, (68), pp.43-63.
• Susaki, J. 2012. Adaptive slope filtering of airborne LiDAR data in urban areas for digital terrain model (DTM) generation. Remote Sensing, 4(6), pp.1804-1819.
• Vosselman, G. 2013. Point cloud segmentation for urban scene classification. In: ISPRS Int. Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 1, pp.257-262.
• Zhang, J. and Lin, X. 2017. Advances in fusion of optical imagery and LiDAR point cloud applied to photogrammetry and remote sensing. International Journal of Image and Data Fusion, 8(1), pp.1-31.

• Special Issue - X CBCG

Publication Dates

• Publication in this collection
  10 July 2019
• Date of issue
  2019

History

• Received
  19 Nov 2018
• Accepted
  21 Feb 2019