LiDAR Elevation and DEM Errors in Forested Settings

Digital elevation models (DEMs) form the basis of LiDAR derived tree height measurements and other topographic modeling needs within natural resource applications. We compared 2873 digital total station elevations to the closest discrete LiDAR elevation point and DEM raster cell across several forest and topographic settings. We also examined limiting comparisons to points within 0.5 m and within one meter. Using all nearest LiDAR points, average total station plot elevation differences ranged from -0.06 m (SD 0.40) to -0.59 m (SD 0.23) indicating that LiDAR elevations are higher than actual elevations. LiDAR DEM differences ranged from -0.09 (SD 0.41) to -0.56 m (SD 0.70). We also compared mapping-grade GPS receiver measurements to LiDAR point elevation and DEMs. Average plot GPS elevation differences ranged from 0.24 (SD 1.55) to 2.82 m (SD 4.58) for the nearest LiDAR point, and from 0.27 (SD 2.33) to 2.69 m (SD 5.06) for LiDAR DEMs. We believe that our efforts represent one of the most robust studies of LiDAR measurement errors available in published literature. The relatively small measurement differences that we found between LiDAR elevations and our most reliable field-based method of elevations, the digital total station, demonstrate the potential for LiDAR in forestry and natural resource applications.


Introduction
Light Detection and Ranging (LiDAR), also known as airborne laser scanning, has emerged since its initial applications in the mid 1980's into many activities including forest management, urban planning, natural resource modeling, ice sheet mapping, and road design (Lim et al. 2003;Aguilar and Mills 2008).Forest terrain mapping (Reutebuch et al. 2003) and forest inventory (Reutebuch et al. 2005) are some of the primary LiDAR applications within natural resources.Research in topographic mapping with LiDAR began in the 1980s (Krabill et al. 1984) with the first commercial LiDAR sensor fielded in 1993 (NOAA 2010).
In aerial mapping applications, LiDAR is an active sensor that directly measures elevations of objects between the aircraft and the ground.LiDAR relies on the principle that a laser pulse moves at the speed of light, thus the time it takes for a laser pulse to travel from the sensor to an object and back to the sensor enables calculation of the distance between the sensor and object.These distances allow for the calculation of object heights based on an aircraft's altitude (Evans et al. 2009).An aerial LiDAR system configuration for terrain mapping consists of a laser scanning sensor mounted on an aircraft (either fixed or rotary wing), Inertial Measurement Unit (IMU), and global positioning system (GPS) (Hodgson et al. 2005;Reutebuch et al. 2005;Liu 2008).To obtain accurate ground coordinate measurements, the sensor location must be known throughout data collection.The GPS and IMU work in concert to continually determine the absolute location and orientation of the sensor (Hodgson et al. 2005).The IMU continually measures the attitude of the aircraft for pitch, roll, and yaw (Liu 2008).This information is for measurement correction and calibration based on the time indexed position of the aircraft and on-board instrumentation (Watershed Sciences 2008).The laser sensor consists of a pulse generator, infrared laser in the wavelength range of 0.8 µm to 1.6 µm, and a pulse receiver.Higher pulse rates equate to greater data density.In forestry, thousands of LiDAR points are used per hectare to measure and classify features such as vegetation or ground terrain.Final LiDAR output includes measurements referenced to X, Y, and Z coordinates (Liu 2008).Pfeifer and Briese (2007) provide a comprehensive review of LiDAR mechanics.
Many forest attributes can be measured by LiDAR over large areas including subcanopy topography, vertical canopy distribution (Lim et al. 2003), and individual tree heights (Andersen et al. 2006).LiDAR tree height measurements are influenced by size and reflectivity of the tree, sampling density, LiDAR pulse diameter, and tree crown shape.A primary source of error in LiDAR tree height measurement of conifer species occurs when laser pulses miss the tree apex resulting in height underestimates (Popescu et al. 2002;Anderson et al. 2006).Another potentially significant impact on tree height is terrain height measurement (Andersen et al. 2006).LiDAR tree height estimates are calculated by subtracting the terrain surface as represented by a digital elevation model (DEM) from the highest point associated with an individual tree (Kraus & Pfeifer 1998;Lim et al. 2003).Not all emitted LiDAR pulses penetrate canopy and strike ground surfaces.Kraus and Pfeifer (1998) found that less than 25% of LiDAR pulses reached the ground in wooded areas.Means (2000) found that 1-5% of LiDAR pulses struck the ground in conifer forest.Creating a DEM first requires eliminating all points classified as non-ground from the three dimensional LiDAR point cloud by automated and manual computer methods.The automated method of identifying ground points is an iterative process using spatial algorithms to identify the lowest points within a spatial search pattern.Identified points are assumed to represent the ground.The specific neighborhood method applied by LiDAR vendors to identify ground surface points is proprietary.Once the automated method is complete, an analyst will manually refine the ground candidate set usually by comparing it to orthophotography (Hodgson & Bresnahan 2004) or viewing cross-sections of the ground surface points.DEMs created from bare earth LiDAR can be interpolated to resolutions as fine as 0.1 m (Anderson et al. 2006).The choice of interpolation method is important as some methods are considered more exacting than others (Popescu et al. 2002;Chaplot et al. 2006).Regardless of interpolation method, if the distribution of discrete points used for interpolation are not regularly spaced at the desired resolution of the DEM grid, interpolation error can be introduced (Fisher and Tate 2006).Laser points used to interpolate an elevation surface are typically irregularly spaced, especially when under dense canopy (Kraus and Pfeifer 1998).
A DEM usually consists of a collection of equally-sized grid cells with each cell containing an elevation (Chaplot et al. 2006).The opportunity for error propagation exists in many phases of DEM generation and endemic error may exist in many DEMs (Fisher and Tate 2006).LiDAR users usually rely on DEM providers for nominal accuracy specifications of raw data (McGaughey et al. 2004).Most accuracy confirmation involves a limited sample of ground control check points obtained by real time kinematic (RTK) GPS surveying.LiDAR-derived elevations are compared to ground control check points and residual error is presumed to follow a statistically normal distribution.Errors, however, are likely enhanced in LiDAR elevations captured over non-open terrain.Errors are generally positive in a forest meaning that LiDAR derived elevations are usually higher than actual ground surface, and are likely from false ground samples where LiDAR returns reflect low vegetation and logs (Reutebuch et al. 2003;Hodgson & Bresnahan 2004;Aguilar & Mills 2008;Liu 2008).Canopy cover also affects LiDAR elevations since canopy obstructs pulses from reaching the ground, reducing LiDAR point density for surface interpolation.Smaller LiDAR footprints increase the chance that a laser pulse will pass through canopy breaks.Chen (2007) argues that discrete-return LiDAR can achieve better tree height accuracy than field measurements and can also provide ground truth height information.Previous research has found error in LiDAR tree height measurements.Some height error can be attributed to the difficulty LiDAR has in hitting the actual apex of conifer tree species but LiDAR terrain measurement error may also be significant (Andersen et al. 2006).The relative accuracy of LiDAR for establishing elevation has been studied but little empirical research exists that examines LiDAR derived vertical DEM accuracy (Leckie et al. 2003;Reutebuch et al. 2003).While typical elevation accuracies stated by LiDAR vendors is around 0.15 m root-mean-squared-error (RMSE), this accuracy is generally only achievable under ideal circumstances such as those found in flat, open terrain (Hodgson & Bresnahan 2004;Su & Bork 2006).
Our primary study objective was to compare specific elevations (orthometric heights) determined by trigonometric leveling with a total station to the closest discrete LiDAR ground point across a range of different forest canopy and topographic conditions.We believe that this comparison may help quantify the absolute elevation error resulting from LiDAR sensor measurements.We also examined the influence of limiting discrete point comparisons to only points that were within 0.5 m and only those that were within one meter.Our comparison used 2873 ground surveyed locations.To our knowledge, no previous peer-reviewed study exists that has made a similar comparison of this magnitude.A secondary objective was to determine the accuracy of LiDAR derived DEMs within a range of different forest canopy and topographic conditions through comparison to elevations determined by total station.A third objective was to compare elevations measured by a mapping-grade GPS receiver to DEM interpolated elevations created from trigonometric leveling measurements.A fourth objective was to compare elevations determined by mapping-grade GPS to the vendor provided DEM interpolated from LiDAR; we compared 8150 GPS and LiDAR elevation locations.

Background
Previous studies have compared LiDAR-derived DEMs to field-based measurements.Reutebuch et al. (2003) compared 1.5 m post spacing DEM elevations derived from LiDAR data to 347 checkpoints located in forest settings including clearcut, heavily thinned, lightly thinned, and uncut.The mean error (ME) in elevation between the LiDAR DEM and the checkpoints was 0.22 m (0.24 m Standard Deviation (SD)).The elevation ME for clearcut, heavy thinned, lightly thinned, and uncut canopy classes was 0.16 m (0.23 SD), 0.18 m (0.14 SD), 0.18 m (0.18 SD), and 0.31 m (0.29 SD), respectively.Hodgson and Bresnahan (2004) found a DEM elevation error range between 0.17 and 0.26 m RMSE in different land-cover classes.They compared 2 m post spacing DEM elevations to elevations determined from surveyed points.Areas of lowest elevation errors included conifer forest, pavement, and high grass, whereas areas of deciduous forest (leaf-off) and brush or low trees had the highest errors.Hodgson et al. (2005) conducted a similar study within terrain described as gently rolling with elevations ranging from 44 to 136 m above mean sea level.They reported an error range from 0.15 to 0.36 m RMSE with the highest error occurring in scrub/shrub land cover at 0.36 m RMSE.Errors in pine and leaf-off deciduous forest were approximately the same at 0.28 and 0.27 m RMSE, respectively, while mixed forest error was lower at 0.24 m RMSE.Su and Bork (2006) evaluated LiDAR derived DEMs with three main treatment effects of slope, LiDAR off-nadir sampling angle, and four dominant feature types consisting of upland grassland, deciduous forest [leaf-on trembling aspen (Populus tremuloides)], shrubland, and riparian meadow.Measurement errors were evaluated using kriging, splining, and inverse distance weighting (IDW) interpolations.Mean error values were reported of 0.0035 (RMSE 0.133), 0.0028 (RMSE 0.140), and 0.0028 m (RMSE 0.116) for kriging, splining, and IDW, respectively.When limiting the data to slopes above 15º, they found mean error values of -0.0063 m (RMSE 0.200 m), -0.0060 m (RMSE 0.205 m), and 0.0018 m (RMSE 0.186 m) for kriging, splining, and IDW, respectively.With respect to vegetation types, elevations determined on a DEM interpolated by IDW were compared to total station measurements.ME values of 0.20 and -0.22 m for deciduous forest and lowland meadows, respectively, were reported.The reported overall project average IDW derived DEM elevation error and RMSE was 0.02 and 0.59 m, respectively, compared to total station measurements.
According to Li et al. (2005), GPS elevation points are not likely to match grid height points in any DEM.Although not specifically stated in Li et al. (2005), we interpret that the statement is based on the fact that DEMs are interpolated from relatively few discrete, known elevation points, and GPS receiver measurements are generally not used for DEM generation due to poor elevation precision and accuracy.There are few empirical studies to confirm this suggestion.Clark and Lee (1998) investigated the accuracy of elevations for a precision farming study.Topographic maps were created with GPS data collected using a Leica Wild SR299E base station with a Leica Wild SR399E rover.Two GPS collection modes were used: 1) In the stop and go mode, stationary GPS readings were collected for 4 seconds before moving to the next collection point; 2) the kinematic mode involved a tractor-mounted roving RTK GPS receiver moving in a gridded pattern.Vertical accuracy was reported using a concept described as "true error points."The ten randomly placed true error points were control points collected with the base station receiver in rapid static configuration for 3 to 5 minutes."Mass points" were GPS points collected by the rover GPS and were used to generate the topographic maps.The true error points were only used ground truth calculations.Error was reported as the standard deviation of the difference between the elevation at the true error point compared to the topographic map at the same horizontal location.Stop and go error ranged from 0.02 to 0.03 m.The error for the kinematic DGPS ranged from 0.03 to 0.04 m.RTK error ranged from 0.04 to 0.09 m.In a related precision farming study, Yao and Clark (2000) found that a single frequency GPS receiver could support DEM creation with vertical accuracies from 0.12 to 0.14 m.They used a Trimble Pro XRS with integrated GPS-Omnistar antenna mounted on a vehicle driving at a speed of 6-9 km and making 22 passes across a flat agriculture field.
In another DEM study, Holmes et al. (2000) compared 252 post processed DGPS points collected using a Trimble 4400 GPS to 30 m USGS DEMs created by contour digitizing from either photogrammetric sources or existing maps.Elevations varied by 1000 m within a 2700 ha study area in diverse terrain ranging from floodplain to low foothills and high relief.DEM error was calculated by subtracting USGS nearest neighbor elevations from the GPS points.The reported error mean was -0.10 m (SD 4.1), indicating that the USGS overestimated elevation.Li et al. (2005) examined vehicular mounted GPS receivers collecting data on roads in Wales, United Kingdom compared to 10 m resolution DTMs (DEMs) created by the British Ordnance Survey (OS) using a contour file.Three different interpolation methods (linear, bilinear, and biquintic) were used to obtain heights from the OS DEM.Several methods of collection were tested including autonomous coarse acquisition C/A code GPS, real-time kinematic (RTK), and post processed GPS using a road reduction filter (RRF).The RRF snaps the horizontal location of the coarse acquisition (C/A) code GPS height readings to the road center line where the GPS readings were collected.The RRF method significantly increased the GPS elevation accuracy when compared to autonomous C/A code GPS and real-time kinematic (RTK).The RRF GPS RMSE values ranged from 0.79 to 0.82 m in the three different DEM interpolation methods.
Previous studies have also considered comparisons of mapping-grade and survey-grade GPS receivers.Sigrist et al. (1999) studied the impact of forest canopy on vertical GPS measurements by comparing a Trimble ProXL mapping-grade to a survey-grade GPS receiver.Sites included an open agriculture field, broadleaf forest, and dense, mature white pine forest.RMSE results varied widely from 0.77 in the open to 11.89 m under pine canopy.In another study comparing a mapping-grade GPS to a survey-grade GPS, Yoshimura and Hasegawa (2003) examined Trimble Pathfinder Pro XR and Trimble 4600LS autonomous and DGPS measurements.Sites included a forest landing under open sky, cedar plantation canopy, forest road, and broad leaf forest.Average autonomous error was 2.17, 4.23, 3.65, and 3.74 m for the landing, plantation, road, and natural forest, respectively.Average DGPS error was 0.44, 4.52, 4.90, and 4.17 m in the same respect.

Materials and Methods
The study was conducted on the 5475 ha McDonald-Dunn research forest located in western Oregon, USA (Figure 1).The forest spans a portion of the eastern foothills of the Oregon Coast Range with elevations ranging from 75 to 660 m.Conifers dominate the forest with Douglas-fir (Pseudotsuga menziesii) and grand fir (Abies grandis) being the apex species.The primary deciduous tree species is bigleaf maple (Acer macrophyllum) and shrub species is California hazel (Corylus cornuta var.california).Eleven plots were intensively ground surveyed for feature and terrain measurements.Plot strata consisted of clearcut (four plots), even-aged (two plots), old growth/mature (referred to as old growth) (two plots), and uneven-aged (three plots).Each plot spans one hectare (10,000 m2) and was selected by stratified random sample (Figure 2).Plot naming throughout this study corresponds to silviculture treatment (C = clearcut; E = even-aged; O = old growth; and U = uneven-aged) and plot number.

Field Measurements
Elevation field measurements were collected using a digital total station instrument and mapping-grade GPS receivers as part of a comprehensive inventory of all tree, shrub, and downed woody debris locations.Large tree locations were measured using a two meter offset while other features were measured at their actual location.A total station with a rated angular accuracy of five-seconds collected multiple elevations at five of the 11 plots in order to address the initial three study objectives.Three plots were surveyed by closed traverse (plots E200, O16, and U13) while two others used a radial traverse method (plots O69 and U8).The radial traverse method increases the speed of traverse measurements as the instrument remains stationary throughout all measurements, consequently no closure error is calculated (Table 1).Post processed survey-grade GPS measurements were used to establish two control stations associated within each plot to associate actual coordinates to total stations measurements.GPS data were collected on all eleven plots to address study objectives three and four.We used three similar Trimble mapping-grade receivers for GPS data collection in order to expedite data collection.Receivers included GeoXT, GeoXH, and ProXH equipped with an external antenna (Table 2).The GeoXT receiver has a slightly higher expected error and was used in the clearcut and younger even aged (E412) plots where canopy was unobstructive.The remaining plots were measured using the ProXH with the exception of a final plot (U8) that was collected using the GeoXH.The GeoXH and ProXH receivers have identical manufacturer accuracy specifications.Trimble (2006b;2007;2009a;2009b).
All GPS receivers were configured identically throughout the survey.To reduce atmospheric attenuation and improve measurement accuracy, we used a 15º horizon mask, a PDOP mask of 6 (Van Sickle 2008), and a manufacturer recommended signal-to-noise ratio of 39 (Trimble 2006).The GPS receiver was attached to a pole with the antenna mounted 2.2 meters above the ground to avoid multipath and/or signal attenuation from the ground and operator.A minimum of thirty and not more than sixty points were averaged per position.GPS receiver files were differentially corrected with automatic carrier and C/A code processing using multiple base station providers selected through plot proximity.

LiDAR Collection
LiDAR data were collected on April 2, 2008 under clear weather conditions with a Leica ALS50 Phase II laser system configured with a ± 14º scan angle from nadir and pulse rate designed to achieve a point density of ≥ 8 points per m2 (Watershed Sciences 2008).To reduce laser shadows and increase laser coverage, each flight line had ≥ 50% sidelap, which equates to ≥ 100% overlap.The system is capable of a maximum number of four returns per pulse.The onboard differential GPS unit measured aircraft position twice per second (2 Hz) and the inertial measurement unit (IMU) measured aircraft attitude 200 times per second (200 Hz).
Ground control was conducted simultaneously during the LiDAR survey with a static GPS observation.A RTK GPS survey of 510 points from a limited, open area in the northern portion of the study area served as ground truth LiDAR point coordinates (Table 3).We used three main methods for comparing elevations: 1) discrete point to discrete point, 2) discrete point to DEM, and 3) DEM to DEM.The discrete point to discrete point method compared the ground surveyed elevations from both total station and GPS to the closest discrete LiDAR point elevations.Our discrete point comparisons included elevations differences between all points, only points within one meter of each other, and only points within a half meter of each other.We were interested in different proximity comparisons as previous studies have determined that horizontal displacement can contribute to vertical DEM error (Hodgson and Bresnahan 2004;Su and Bork 2006).We chose a one meter threshold based on the one meter resolution DEM provided by the vendor.The half meter threshold distance was selected so that we could further explore the influence of point proximity on accuracy.The discrete point to DEM method compared the ground surveyed points collected with total station and GPS to the vendor provided LiDAR DEM.The DEM to DEM method compared our GIS-derived DEMs to the vendor DEM.An additional comparison included total station derived DEM to GPS derived DEM.
The GIS software package ArcGIS 9.3 was used to interpolate total station and GPS elevation data for DEM creation.The ArcGIS 9.3 spatial analyst tool "Topo-to-Raster" was used, which applies the ANUDEM algorithm for interpolation.We left the default option of a maximum number of iterations to 40.Andersen et al. (2006) compared DEMs created from portions of a LiDAR database.Increased elevation overestimation occurred when LiDAR data were reduced and it was speculated that the ANUDEM interpolation algorithm may introduce systematic error.Overestimates, however, were not statistically significant and limited to <0.03 m compared to a DEM of all LiDAR data.ANUDEM may have a similar affect on the DEMs we created from the total station data based on the limited point density.
DEM error is commonly reported by RMSE although another commonly used DEM error statistic is mean error (ME).Because ME can either be positive or negative, thus illustrating any bias in the model, some researchers consider it to be a more complete error description when reported with a standard deviation (SD) (Fisher and Tate 2006).
ME= ∑(Z_DEM-Z_Ref) / n ME and SD were calculated for LiDAR and mapping-grade GPS derived DEMs compared to total station derived DEMs.Total station elevations were treated as ground truth.Errors were determined by subtracting the LiDAR DEM and GPS DEM from the total station DEM using ArcGIS software.ME, SD, and RMSE were calculated for LiDAR DEM elevations compared to discrete elevation points collected by total station by calculating zonal statistics in ArcGIS.Zonal statistics assigned the LiDAR DEM elevation to each total station point in an output database, thus allowing for the subtraction of the LiDAR DEM elevation at the discrete total station point from the total station surveyed elevation at the same point.This also enabled manual calculation of RMSE, whereas ArcGIS did not calculate RMSE in the raster subtraction.ME, SD, and RMSE were calculated for LiDAR ground point elevations compared to the closest total station elevations.ME, SD, and RMSE were also calculated from the comparison of mapping-grade GPS elevations to LiDAR derived DEMs and closest LiDAR ground points.GPS elevations were treated as ground truth.The LiDAR DEM was compared to both GPS discrete elevations and GPS derived DEMs.Based on the nonparametric characteristics of the elevation differences, we determined statistical significance with Wilcoxon rank-sum tests.All spatial databases in this study were referenced to a NAD83 (CORS96) horizontal datum and NAVD88 vertical datum with Geoid03.

Results
Total station and LiDAR elevation comparison.
Total station data were collected in five of the eleven study plots (Table 4).LiDAR elevation points were subtracted from total station elevations for comparative statistics.When only points that were within 0.5 m of each other were selected, resulting ME values ranged from -0.06 (SD 0.37 ) to -0.60 m (SD 0.17) (Table 5).All ME values were negative indicating that the LiDAR elevations are above total station elevations on average.Although ME values were as high as -0.60 m, none of the point-to-point comparisons displayed a statistically significant difference based on Wilcoxon rank-sum tests.Horizontal displacement can have an impact on elevation error.We chose 0.5 m based on half the distance of a one meter resolution DEM provided by the vendor.While many of the closest LiDAR points to total station points were between 0.5 and 1.0 m apart, some were further than 1.0 m apart.Counter to expectations, the mean error did not improve in four of the plots, and only improved by 0.02 m in plot E200 as the comparison data were reduced to only those points within 0.5 m, and in three of the plots ME actually slightly increased (Table 6).In all plots, the SD decreased when limiting the data set from all the closest points to points within 0.5 m.The plot with the least amount of LiDAR ground points within 0.5 m of a total station point was E200.The high density of even-aged trees in this plot likely limited the LiDAR canopy penetration to the ground.The only plot that displayed statistically significant error between total station points and LiDAR points is E200 using all matched points (Table 6).The error was not statistically significant using only those points within 0.5 m.Curiously this was the plot with the least slope in the study (13%, Table 4).Since this was a young even-aged stand with the second greatest crown cover (65%), this likely had an impact on the accuracy of the DEM as it had by far the lowest number of LiDAR ground points, and the second greatest percent crown cover.An important fact related to this is that this plot had a large area of redundant LiDAR sampling with sidelap in the flight path/LiDAR swath.The sidelap did not apparently improve the ground point sampling count; the density of points was the least in all the plots due to the even-age stem and crown density.One would presume that sidelap would improve the canopy penetration, but it is possible that canopy penetration requires an increased pulse rate, thus a greater sampling density.Comparing the total station points to the LiDAR DEM, ME generally increased with the exception of plot U13.The only plot displaying statistically significant (p = 0.01) mean elevation differences is plot E200.The error increases when comparing DEM to DEM.Although ME values change little from the total station point and LiDAR DEM comparison to the total station DEM and LiDAR DEM comparison, the mean elevations between the total station and LiDAR DEMs were all significantly different (p ≤ 0.007) except in plot U8.The increase in error likely reflects error in the interpolation of the DEM because the errors in the total station point to LiDAR point comparison were not statistically significant (p > 0.18).The LiDAR DEM is likely to reflect higher precision and accuracy than the total station DEM due to the far greater density of LiDAR points used in interpolating the DEM.

GPS Receiver and LiDAR Elevation Comparison
GPS receiver measurements were collected in all eleven study plots (Table 4).The clear cut plots, and one even-aged plot (E412) were collected with C/A code and the remaining plots were collected using C/A code and carrier phase frequencies.Plot E412 was generally free of GPS signal obstruction from canopy due to the relatively young age of trees in the plot.The elevation error between unobstructed C/A code only GPS points and LiDAR elevation points was relatively low and ranged from -0.50 to 0.49 m (Table 7).The plots with the lowest ME were E412 (-0.03 m) and C27 (-0.10 m).These ME values are similar to the errors in the total station comparison noting that these data were collected under generally open sky conditions with little or no canopy obstruction.In comparing the GPS point data compared to the LiDAR DEM, ME ranged from -0.46 (SD 0.41) to 0.53 m (SD 0.75).Although there is no apparent consistency in the LiDAR DEM being above or below the GPS elevations, the ME absolute values are similar to those found when comparing the total station to the LiDAR DEM (Table 8).Although the highest error occurred on plot C20 (ME 0.53) where the most rugged terrain occurred with some slash, a similarly high ME (-0.46) occurred on plot C110 which was on relatively flat slope.Plot C110 did have many perennial shrubs including poison oak (Rhus diversiloba) and Oregon grape (Mahonia aquifolium) which may have contributed to errors in the LiDAR DEM ground classification.The ME was generally improved over the point-to-point analysis when comparing the GPS DEM to the LiDAR DEM elevations plot by plot, although the range in elevation differences was slightly higher with ME values ranging from -0.45 (SD 0.32) to 0.59 (SD 0.64). is only slight, however, and the error actually increased on plot C20 and E412.In every case, the SD decreased as a result of interpolation smoothing.The least error occurred in plot C27 (ME 0.02 m, SD 0.34), which is surprising given that this plot had many large stumps and small slash piles that could have contributed to false ground identification.*The GPS points used in this analysis were C/A code corrected in the unobstructed plots and C/A code and carrier phase corrected in the obstructed plots.This also applies to GPS points used to generate respective DEMs.
In comparing measurement differences from obstructed GPS data collected under canopy using carrier phase and C/A code differential corrections, error increased dramatically with the exception of one plot.The GPS point to closest LiDAR point elevation comparison results in a ME range from 1.61 m to 2.82 m when excluding plot U13 (Table 7).The plot U13 ME of 0.24 m (SD 1.55 m) was low compared to the other plots.The ordinal variation in these plots compared to that of the total station to LiDAR (Table 3.6) does not display the same pattern from low to high error.In these comparisons, all of the LiDAR ME's indicate that the LiDAR DEM elevations are below the GPS elevations with one exception.Observing the GPS to LiDAR elevation comparisons, seven of the eleven plots had little effect on the ME (0.00-0.07m) when limited to only points within a meter or half-meter distance were analyzed (Table 3.8).The ME in plot E412 improved by 0.13 m when restricting point comparisons to those within a half meter.This improvement can be explained by the fact that this plot had a very high tree density in its SW quadrant that severely restricted the number of LiDAR points that struck the ground.Plots O69 and U8 had the lowest density of GPS points compared to the other plots (Table 4).This low point density and relatively steep slopes in comparison to other plots appears to have contributed to the improvements in ME within plots O69 and U8.We cannot explain the error increase of 0.12 m in plot U13 but do note that the MEs were the lowest for this plot in comparison to all other plots that were canopy obstructed.
The comparison between GPS point to LiDAR DEM ME values ranged from 0.27 (SD 2.33) to 2.69 m (SD 5.06) for GPS data collected under canopy (Table 7).With the exception of plot U13, the ME ranged from 1.67 to 2.69 m.This may be attributed mostly to canopy cover and steep slopes.Plot E200 (ME 1.67 m, SD 3.59) had a very high crown cover percentage and plot O69 (ME 2.63 m, SD 4.58) was on a very steep slope.Plot O16 was also on a steep slope and based on field observations appeared to have the greatest slope variation of all plots.The crown cover percentage was relatively high and included many deciduous trees.Plot U13 had low error values (ME 0.27 m, SD 2.33) and, although located on a relatively steep slope with many large canopy openings, had a relatively low tree/canopy density and was the only plot with a SE slope aspect (Table 4).In every case except plot U13, the GPS DEM compared to the LiDAR DEM had an improved ME over the GPS point to LiDAR DEM.The ME values ranged from -0.29 (SD 2.02) to 1.71 m (SD 2.50).The SD values also improved in all but one plot (O16).The plot O16 increase in SD may be attributed to the high slope variation in the plot.It appears that the GPS DEM interpolation may have caused a drop in the overall GPS plot elevation in comparison to the GPS points.The least amount of ME difference occurred in plot U56 (ME decrease = 1.79 m -1.71m = 0.08) and the greatest amount was plot O69 (ME decrease = 2.63 m -1.22 m = 1.41 m).This decrease in elevation however resulted in a slightly increased ME, but in the opposite direction on plot U13 (ME decrease = 0.27 m -(-0.29 m) = 0.56 m).
Since the errors dramatically increased in the LiDAR data collected with dual frequency GPS under canopy compared to the C/A code only GPS data collected in the open, we also analyzed whether C/A code only GPS elevations compared to LiDAR elevation would differ from elevation comparisons from carrier phase and C/A code measurements.We used the point-to-point method and compared GPS elevations to the nearest LiDAR elevation point in plots with canopy cover.In every plot elevation ME increased with C/A code only GPS.The ME ranged from 0.85 m (SD 1.64) to 4.46 m (SD 5.20) (Table 7).The least amount of error increase occurred on plot O16 where ME only increased 0.02 m.The greatest error increase occurred in plot O69 where ME increased 2.33 m.This does not relate notable information regarding LiDAR elevation accuracy, but demonstrates that GPS elevation accuracy under forest canopy is higher using the combined C/A code and dual frequency carrier phase ranging over C/A code only ranging in mapping-grade GPS receivers.
Most of the plots with unobstructed GPS reception did not display statistically significant elevation differences except in the GPS DEM to LiDAR DEM comparison.Three of five comparisons resulted in definitive significant differences with p-values < 0.00 and the remaining two had p-values of 0.08, which is marginally significant.This may be explained by the interpolation of relatively few GPS points compared to LiDAR points.Plot C110 was the only clearcut plot that included a large amount of short, woody ground cover including Oregon grape and poison oak.This likely had an impact on separating the ground points from vegetation.Plots C20 and C27 had a large amount of downed wood and slash piles.We noticed that these features were easily identified and separated from true ground points, thus they were likely removed by the vendor when creating the ground file.Most of the GPS to LiDAR comparisons collected under canopy reflected significant differences.Plot U13 was the only obstructed plot that did not demonstrate significant elevation differences and displayed relatively low ME in the phase and C/A code GPS to LiDAR point comparison.This is likely attributable to the relatively small crown cover (38%, Table 4), lack of understory vegetation, and easily identified large downed wood features.

Total Station to Differentially Corrected GPS Elevation
We compared the elevations between DEMs derived from total station and GPS measurements.In all but one of the plots the ME values are negative indicating that mean GPS elevation values are above the mean total station values (Table 7).Excluding plot U13, the ME ranges from -1.12 m (SD 3.38 m) to -1.84 m (SD 5.22 m).In plot U13, the ME of 0.27 m (SD 2.02 m) is markedly smaller than the other plots, which coincidently is the same ME between the GPS point and the LiDAR DEM.The average error for the GPS DEM compared to the total station DEM (-1.15 m) is much higher than the average error for the LiDAR DEM compared to the total station DEM (-0.39 m).This indicates that greater elevation error is coming from GPS measurements.

Discussion
LiDAR and total station elevation comparison.
Among the measurement methods used in this study, the most reliable is the digital total station.We compared 2873 total station elevations to the nearest LiDAR elevation point in five plots.We are unaware of a previous study where individual total station elevation points were compared to discrete elevations acquired using LiDAR.
We feel that this is the most meaningful evaluation of LiDAR vertical measurements since it more directly evaluates LiDAR absolute elevation accuracy.Using all of the nearest points, the lowest ME was -0.06 m (SD 0.40 m) and the highest was -0.59 m (SD 0.23 m) (Table 6).This varies only slightly from a data comparison that included only those nearest LiDAR elevation points that were within 0.5 m of a total station point.The LiDAR elevation point accuracy in this comparison is higher in general than in the comparisons that involved DEMs.
A critically important elevation comparison for our interests was the LiDAR DEM contrasted to the total station data, as we intended to measure tree heights using LiDAR for future work.LiDAR tree heights typically underestimate actual tree heights.LiDAR DEM error ranged from -0.09 (SD 0.41) to -0.56 m (SD 0.70) (Table 5).The LiDAR DEM accuracies we determined are consistent with previous studies.Kraus and Pfeifer (1998) found an overall DEM accuracy of 0.57 m RMSE in wooded terrain whereas Gomes-Pereira and Janssen (1999) found DEM ME between 0.25 and 0.38 m on sloping terrain.Hodgson and Bresnahan (2005) reported RMSE elevation errors in pine (0.28 m) and leaf-off deciduous (0.27 m) forest that were approximately the same while mixed forest error was lower (0.24 m).The most similar research we found to our study was that conducted by Reutebuch et al. (2003), which was conducted within various canopy conditions.The overall project DEM ME, SD, and RMSE were 0.22, 0.24, and 0.32, respectively, with a ME range of 0.16 m (SD 0.23) in clearcut stands and 0.31 m (SD 0.29) in uncut thick forest canopy.In our study the highest ME (-0.56 m) occurred in plot O69 (Table 5), which is located on very steep slopes (55%) in a 138 year old stand.The least amount of error (-0.09 m) occurred in plot U8 which is located on a relatively steep slope (32%) in an 85 year old stand.This further confirms that LiDAR is capable of achieving high vertical accuracies under forest canopy and steep slopes, but at the same time these environmental characteristics can degrade accuracy.

LiDAR and GPS Elevation Comparison
Although we treated the GPS measurements as actual elevations for comparative purposes, LiDAR elevations and associated horizontal positions are likely more accurate and consistent than mapping-grade GPS receiver measurements.This portion of the study demonstrated how mapping-grade GPS elevation accuracy is degraded with more severe forest conditions such as topography canopy.Our results are somewhat biased in that we used several similar yet not identical GPS receivers for our measurements.Based on the dramatic overall error increase witnessed in comparing GPS to LiDAR elevations, as contrasted to total station to LiDAR, it appears that much of the elevation error may be attributed to the GPS receiver.*The GPS points used in this analysis were C/A code corrected in the unobstructed plots and C/A code and carrier phase corrected in the obstructed plots.
Like the total station comparison, the GPS point to LiDAR closest point comparisons gave us the best indication of LiDAR elevation accuracy as no interpolation is involved.We compared 8,150 GPS and LiDAR elevation locations in eleven plots.In the open, GPS elevation accuracy is surprisingly high when compared to the LiDAR elevations.Using all positions, ME ranged from -0.02 (SD 0.50) to 0.49 m (SD 0.75) (Table 8).Overall, using all the positions slightly improved the accuracy over limiting the matched points to closer than a half meter.Analysis of the plots under canopy reveals that error significantly increases, and that accuracy actually decreased in using C/A code only, in fact it decreased.Error in the LiDAR points compared to all carrier phase GPS positions resulted in ME ranging from a low of 0.24 m (SD 1.55) in plot U13 to a high of 2.82 m (SD 4.58) in plot O16 (Table 7).After the lowest ME of 0.24 m, the next lowest ME is 1.61 (SD 3.05).Using the total station comparison as the criterion, we believe that these errors are more a reflection of GPS rather than LiDAR measurements.The low error in plot U13 compared to the other plots is likely due to canopy cover.Plot U13 is an uneven-aged plot with many large canopy openings of well dispersed large trees.Using plot O16 with the highest ME as an example, although plot O16 had a greater number of ground points (11,994) compared to U13 (8,995) the ME was much higher (2.82 v. 0.24 m).The overall density and the spatial configuration of the ground points are both influential.Although the ground point density was higher in plot O16, compared to U13 there were larger contiguous areas without ground returns that may influence elevation error (Figure 3).The GPS points are also associated with offsets from tree locations.Plot U13 had a lower average stand age (Table 4) and had many more small trees than did O16.These small trees tended to be in the open, and not under forest canopy, thus allowing for potentially more accurate GPS readings.Another observation comparing open to canopy cover plots is that using all the matched GPS points tended to improve the overall accuracy in the clearcut plots as discussed above, but in the plots with canopy cover the opposite occurred.Under canopy conditions, using all the points in the GPS point to LiDAR point elevation comparison increased ME in contrast to limiting the matches to those within 0.5 m.Our results differed somewhat from Li et al. (2005).Similar to the point-to-point comparison above, the LiDAR DEM demonstrates surprisingly accurate elevations in open areas compared to mapping-grade GPS elevations; but the DEM analysis further confirms the inaccuracy of GPS acquired elevations when collected under forest canopy and on steep terrain.The C/A code GPS data collected in the open canopy conditions had ME values that ranged from -0.46 m (SD 0.49 m) to 0.53 m (SD 0.75 m), with the most accurate being -0.17 m (SD 0.49 m) when comparing the GPS point data to the LiDAR DEM (Table 7).These values remained relatively consistent when comparing the GPS derived DEM to the LiDAR DEM with values ranging from -0.45 m (SD 0.32 m) to 0.59 m (SD 0.64 m).Two plots demonstrated accuracy improvement as a result of GPS DEM interpolation and subsequent comparison to the LiDAR DEM.In plot E412, ME improved from 0.21 m (SD 0.92 m) to 0.15 (SD 0.65 m) and plot C27 improved slightly from -0.04 m (SD 0.45 m) to 0.02 m (SD 0.34).This is notable with respect to C27 as there was significant slash on the plot which could have degraded LiDAR elevation data.Within plot E412, over ninety percent of the trees were taller than the GPS antenna and ranged in height from four meters to eleven meters.The GPS data collected under canopy was collected using dual carrier frequencies and both the C/A code and carrier phase were used in differential correction.The GPS elevation accuracy degraded significantly under canopy with one exception.The exception occurred on plot U13 where the ME of the GPS DEM compared to the total station DEM was 0.27 m (SD 2.02) (Table 5), and the carrier phase plus C/A code GPS compared to the LiDAR point elevation ME equaled 0.12 using all comparisons (Table 8) Although the ME was surprisingly low, the canopy had an obvious impact on the SD when compared to GPS data collected in the open.The low error in this plot compared to the other plots with canopy cover is likely a result of many large openings in the canopy and relatively low density of large trees.As this plot is uneven aged, it had historically been aggressively selectively cut.This is also the only plot with a SE slope aspect.We have noticed during this study and during other GPS work in the study area that GPS reception on slopes seems to improve when on south facing slopes.This observation may warrant further research.The remaining plots under canopy demonstrated relatively poor elevation accuracy when compared to the total station elevations.
The GPS to LiDAR DEM accuracies we found are consistent with other studies and in some examples demonstrate improved accuracies.For studies conducted in open canopy comparing GPS to DEM elevations, Clark and Lee (1998) reported standard deviation of the error in stop and go GPS surveying between 0.02 and 0.03 m, kinematic DGPS between 0.03 and 0.04 m, and RTK between 0.04 and 0.09 m.Yao and Clark (2000) found that a single frequency GPS receiver rated with sub-meter accuracy can provide a DEM with vertical accuracy in the range of 0.12-0.14m.Holmes et al. (2000) reported an ME of -0.10 m (SD 4.11) when comparing DGPS points collected using a Trimble 4400 GPS receiver to a 30 m USGS DEMs.Li et al. (2005) found RRF GPS RMSE errors on roads ranging from 0.79 to 0.82 m compared to 10 m British Ordnance Survey (OS) DEMs.
In other studies comparing GPS elevations in forests, Sigrist et al. (1999) compared mapping-grade to survey-grade GPS measurements.Vertical comparisons demonstrated a wide range of RMSE values from 0.77 m in the open to 11.89 m under pine canopy.In another comparison of mapping-grade to survey-grade GPS, Yoshimura and Hasegawa (2003) found average autonomous errors of 2.17, 4.23, 3.65, and 3.74 m for forest landing, forest plantation, forest road, and natural forest settings, respectively.Average DGPS error was calculated at 0.44, 4.52, 4.90, and 4.17 m, respectively.
The total station comparison subtracted LiDAR point and DEM elevations from total station elevations.All results indicated negative mean error demonstrating that LiDAR elevations are above true elevations on average.The similar LiDAR elevation comparison to GPS also indicated negative mean error in the open terrain study, although not as consistently.In most cases under canopy, the GPS elevations minus the LiDAR elevations resulted in positive differences.Subtracting the GPS DEM from the total station DEM, however, resulted in consistently negative values, with one exception where a 0.27 m difference resulted, indicating that the GPS DEM also tends to overestimate ground elevations.We found that LiDAR overestimates ground elevation in this study, meaning that LiDAR DEM error will contribute to underestimating tree heights.

Conclusion
LiDAR data is now a standard data source that is used across disciplines to provide elevation surfaces.We closely examined differences between elevations that were derived between LiDAR data, digital total stations, and GPS receivers.Based on the number of ground truth survey points, as captured by a digital total stations, this is the most robust study of its kind of which we are aware.Furthermore, we know of no other studies that have matched and compared ground surveyed points to the closest discrete LiDAR ground point across a range of forest settings.The LiDAR and GPS errors reported in this study were consistent with other previous research.While the differences between the LiDAR DEM and our most accurate ground measurement method using a total station survey instrument were statistically insignificant, error significance is based on the application.In determining tree heights based on a DEM, a half meter may not be significant in the resulting amount of milled board feet, however, a half meter could be a costly error in many civil engineering projects.This study contains many potential sources of error including those committed by the LiDAR vendor, survey crews, GIS analysis, and statistical analysis.Nevertheless, based on results that demonstrated relatively small measurement differences, we believe that our reported accuracies are conservative, and demonstrate the potential for LiDAR in forestry and natural resource applications.

Figure 1 .
Figure 1.McDonald-Dunn Forest and surrounding communities within Oregon, USA Figure 3. U13 and O16 LiDAR ground point density and topography comparison

Table 1 .
Total station survey closure error.A radial traverse was used for plots O69 and U8 and resulted in no closure statistics

Table 3 .
Laser point density and accuracy reported by vendorLaser point coordinates were computed with Leica Inertial and Positioning System (IPAS) and Airborne Laser Scanner (ALS) post processing software, then resolved to correct for aircraft pitch, roll, heading, and scale, and filtered for non-terrestrial returns caused by birds and atmospheric effects.Internal calibration of the LiDAR data was refined using TerraMatch software in which automated sensor attitude and scale corrections yielded 0.03-0.05m relative accuracy improvements, and then GPS drift was removed per flight line improving relative accuracy by < 0.01 m.TerraScan software was used to classify and model near-ground points, which were then manually inspected and refined for improved ground detail and modeled again (Watershed Sciences 2008).A DEM was created from a triangulated irregular network (TIN) based on ground classified LiDAR points and a modeling algorithm within TerraSolid software.

Table 4 .
Plot characteristics and measurement counts.Total station elevations were collected on five plots while GPS receiver and LiDAR elevations were collected in all Stand age in years based on oldest trees in the stand at time of LiDAR acquisition.With the exception of plot U8, LiDAR DEM elevations compared to the discrete total station points had a ME ranging from -0.45 m (SD 0.10 m) in E200 and O16 (SD 0.18) to -0.56 m (SD 0.70 m) in plot O69 (Table5).Similar errors existed when comparing the LiDAR DEM to the total station DEM with a ME range of -0.35 m (SD 0.19 m) in plot O16 to -0.56 m (SD 0.23 m) in plot U13.The LiDAR DEM in plot U8 had a low ME of -0.09 m (SD 0.41 m) and -0.04 m (SD 0.45 m) compared to discrete total station points and the total station DEM, respectively.In every comparison involving LiDAR measurements, ME is a negative value and indicates that, on average, LiDAR derived elevations are higher than ground truth elevations.In all but one of the plots, as ME decreases, SD increases.Plot O69 has the highest ME and SD which stands to reason as this plot is on very steep and rugged terrain under old growth canopy with several downed logs.
* which is likely due to the steep slopes where slight variation in horizontal distance results in rapid elevation change.ME on plot U13 decreased with interpolation from -0.60 to -0.50 m, a 0.10 m difference.While the ME in U13 was consistent with three other plots, it was relatively high (-0.50) in comparison to plot U8.ME in plot U8 was less than 0.10 m for the total station point to LiDAR DEM comparison.Only 300 m separate these two plots with plot U8 having a greater canopy cover and slightly greater slope.There are many potential causes for the larger ME at plot U13, including survey crew error and relatively higher variance at GPS control stations.

Table 6 .
Mean elevation error (ME), standard deviation (SD), and root mean square error (RMSE) (m) of the closest LiDAR ground point to a total station point including all closest points, closest points within 1 m, and closest points within 0.5 m

Table 7 .
Elevation error (m) of carrier phase (Φ) with C/A code corrected and C/A code only corrected GPS receiver* elevation points and DEM compared to LiDAR

Table 8 .
Mean elevation error (ME), standard deviation (SD), and root mean square error (RMSE) (m) of the closest LiDAR ground point to a GPS point*, including all closest points, closest points within 1 m, and closest points within 0.5 m