I have been working with detecting trees using LiDAR data for a while now, one of the biggest challenges that are still not completely solved by the community is the detection of individual tree when trees are highly overlap. Most detection methods underperform for overlapped trees, we will continue to work on improving detection accuracy of occluded trees, using some of the techniques adopted by occluded human detection.
What? You can do that?
Actually kind of.. anyone who had done any geomatics work will know some distance search function. So if you have a dataset that has classification label (dataset A), you can theoretically transfer the label to a second dataset (dataset B) that doesn’t already contain any labels. Simple.
For each point in dataset B, search a point that is closest in dataset A, then copy the label over and it’s done. There are so many existing functions for calculating distance, my colleague started with matlab. However, to search 500,000,000 points, that takes forever, after looking through several methods, I end up with using “Closest Point Set” with cloudcompare. Although I still have to split them in 1gb tiles, the processing time is much faster!
The tools I want to write about today is lastools software suite and is my earliest exploration tools I have used when I first learn about LiDAR. The very first tool I have used is las2txt (https://rapidlasso.com/las2txt/) because back in the days I didn’t know how a *.las file look like so I turned the file into text for studying. In this post, I will talk about few of the commands that I use most often and include my study notes for you.
Download the tools and double click las2txt.ext, you will be able to input your file (red box 1), and select the attributes (e.g. x, y, z, intensity…) (red box 2) that you want to write to your text file.
Another way to run las2txt (or any lastools) is to run it in command prompt, like the above figure. The syntax is straight forward, it involves calling the exe (las2txt); the input file (-i); the parameters (-parse); and the output file (-o). The details can be found in the readme file that you have downloaded.
More information: https://rapidlasso.com/lastools/lasground/
http://lastools.org/download/lasground_README.txt
More information: https://rapidlasso.com/lastools/lasnoise/
http://lastools.org/download/lasnoise_README.txt
More information: https://rapidlasso.com/lastools/lasclassify/
I hope my notes can help you get started with LiDAR data.
I decided to write up this post because many students have had asked how they can start learning and visualizing LiDAR data. LiDAR data is becoming more and more available to the public and many want to learn about it.
Software: CloudCompare
According to their website, CloudCompare is “free to use them for any purpose, including commercially or for education”. This is good because if you are just trying to learn and wanted to make some maps, this is a good starting point without paying for it.
CloudCompare will read and write the following file formats, personally I have had to dealt with *.las format in the past. *.laz is the “zipped” version of *.las.
I wanted to go over few of the most common functions that I use
Let’s first talk about the interface, you can click to the “Properties” tab to look at the properties of the las file. Then, you can change the colorization according to the attribute that come with the file, here I change the colorization to “classification” and so 1) Tree points are yellow 2) Ground points are green and 3) small buildings are orange.
Note that there is a bounding box around the size of the file and the box is bigger than it should. This is because there are a lot of “noise” points in the air, there are not a lot of them so you cannot see them very well but looking at the bounding box, it is bigger than it should indicating that there are probably noise above the trees.
You can clean them by
•Tools -> SOR Filter •https://www.cloudcompare.org/doc/wiki/index.php?title=SOR_filter •
Tools -> Noise Filter •https://www.cloudcompare.org/doc/wiki/index.php?title=Noise_filter
According to https://www.cloudcompare.org/doc/wiki/index.php?title=Cloud-to-Cloud_Distance, the tool compute the distance comparison between two sets of point clouds
Using their example, two sets of points are being compared and the blue area has very little distance differences (two point clouds are almost the same there) and red areas have large differences (two point clouds are most different there). This tool is useful in change detection as well, imagine you acquire one set of LiDAR points from summer and one set in winter times on top of forested area, you will be able to detect all the missing tree crowns due to leaves fallen on the ground.
CloudCompare also lets you do the following comparisons which I often use:
•Cloud-to-cloud (C2C)
•Cloud-to-mesh (C2M)
•Cloud-to-primitive (C2P)
•Robust cloud-to-cloud (M3C2) – Using ‘clean’ normals is very important in M3C2
https://www.cloudcompare.org/doc/wiki/index.php?title=Apply_Transformation
This is a useful tool if you want to simply apply a transformation matrix to a set of point cloud.
‘Edit -> Apply Transformation’ tool
You can even apply the inverse transformation by checking the corresponding checkbox (e.g. to go back to the previous position of the entity)
In this example, I am applying a translation of 100 in x-direction.
Alignment is used when I want to align a set of point cloud with respect to another one, or some known points (reference points – think ground control points GCPs for referencing an image). https://www.cloudcompare.org/doc/wiki/index.php?title=Align
This can be done three ways:
•Match bounding-box centers – simply select two (or more) entities, then call ‘Tools > Register > Match bounding-box centers’
•Manual transformation •’Edit > Rotate/Translate’ menu
•Picking point pairs – Tools > Registration > Align (point pairs picking). This is the most useful one, you have to decide which one is reference and which one is the one you want to align. Then, pick the point pairs (at least 3 pairs). Click “Align” button and you will also get RMS results for evaluation.
In this example provided from CloudCompare:
“As soon as you have selected at least 3 or more pairs (you must have picked exactly the same number of points in each cloud) CC will display the resulting RMS and you can preview the result with the ‘align’ button. You’ll also see the error contribution for each pair next to each point in the table (so as to remove and pick again the worst pairs for instance). You can add new points to both sets anytime (even after pressing the ‘align’ button) in order to add more constraints and get a more reliable result. And as suggested before, you can also remove points with the dedicated icons above each tables or with the ‘X’ icon next to each point.”
Also, if you have a reference point in 3D that you already know (e.g. by ground survey, or corners of the building) you may use them as reference and enter those points manually for alignment.
https://www.cloudcompare.org/doc/wiki/index.php?title=ICP
This two register two sets of point clouds automatically, using ICP (Iterative Closest Point)
The rest is pretty straight forward.
In this example provided by CloudCompare, the final transformation matrix can be calculated with RMS produced automatically. Theorical overlap is 50% which means the two sets of point cloud overlap by 50%. I found this is an important parameter to set in obtaining a “good” RMS.
These are my five top picks for CloudCompare, stay tuned for self-learn part 2.
Contributors: Sidharth Sudarsan, Dr. Gunho Sohn, Dr. Pio Caludio, Dr.Connie Ko, Sun Park, Oluwakiishi Dada, Afnan Ahmad
I didn’t realize it was so difficult to map indoor (3D) from AutoCAD floor plan drawings until I had to work with it this summer. It is just the beginning, and it already took us four months to convert 80+ buildings into somewhat 3D, floor by floor. Topologically there are a lot of problems with the original AutoCAD in GIS perspective, e.g. non-closing polygon, non-connecting lines, annotation in the wrong spot etc.. We manage to get a kick start this summer and definitely a lot more work to come in the near future!
Creating 3D floorplans have a lot of benefits, from simulation to modeling to navigation to visualization. I was very lucky to work with two talented under-graduate students at York University.
ABSTRACT
The automatic creation and update of tree inventory provides an opportunity to better manage tree as a precious resource in both natural and urban environment. Detecting and classifying individual tree species automatically is still one of the most challenging problems to be resolved. The goal of this project is to automatically detect 3D bounding boxes for individual trees from airborne LiDAR data using state-of-the art single-stage 3D object detection methods. According to Guo et al. (2019), there are five broad categories for 3D object detection methods, 1) Multi-view methods, 2) Segmentation-based methods, 3) Frustum-based methods, 4) Point cloud-based methods, and 5) BEV (Bird eye view) -based methods. The Deep Convolution Neural Network (DCNN) we are choosing fall into point cloud-based method using point cloud directly as input. Our study area is at the York University Keele Campus, Toronto, Ontario, Canada. We acquired three sets of data for the project 1) Airborne LiDAR 2) Mobile LiDAR and 3) field data. We acquired two sets of LiDAR data during September 2018, the ground (mobile) data and the airborne data. The ground data collection was carried out by a mobile mapping system consist of Lynx HS600 (Teledyne Optech, Toronto, Ontario) with and Ladybug 360° camera with six integrated cameras on 8th September 2018. The airborne mission took place on 23rd September 2018 with two ALS sensors Galaxy-PRIME and Titan (Teledyne Optech, Toronto, Ontario). The average flying height for the mission was 1829m (6000ft) agl. For the field data, 5717 trees on campus were recorded with species name, tree crown width [m], and locations obtained by a handheld GPS.
Training data is generated semi-automatically with a marker-controlled watershed segmentation method (Markan, 2018) to first over generate detected local maximums (treetops) and watersheds (tree crown boundaries). By merging the representative watershed candidates, we produced 5717 tree crown boundaries. One of the advantages of this method over a simple marker-controlled watershed segmentation is that this method will allow delineated tree crowns to overlap which is common even in the urban environment. Ground truth bounding boxes are then generated from each 2D watersheds and the height of individual trees. While current 3D object detection methods focus on the detection of vehicle and pedestrians for autonomous driving applications, we aim to test the applicability of existing methods on tree objects. Current state-of-the-art methods we are considering includes VoxelNet (Zhou and Tuzel, 2018), SECOND (Yan et al., 2018), Point pillars (Lang et al., 2018), and Part-A^2 (Shi et al., 2019). Regarding 3D object detection modules, tree point cloud data are sampled and processed into representations suitable for deep convolution neural networks. Several representations as 3D-voxelization-based representation and bird’s-eye-view-based representation are being explored. The representations will be passed through a trainable neural network to generate predicted 3D bounding box for trees. Improving detection accuracy not only affect classification results and will also improve existing tree locations that we traditionally recorded by handheld GPS or estimated from google street view imagery.
For the map, please visit: https://arcg.is/0KbH0q
The tree data was collected three years ago hosted by ESRI (http://hub.arcgis.com/datasets/bf0f4752b85c44318918e2a8be2847dc_0), I quickly created a web map to show you how the tree location is not aligned with the tree top. In order to use this data set (along with it’s attribute, e.g. dbh, height, species) for validation (for detection and classification), this problem must be addressed and corrected first.
Using airborne view, they can be identified as:
Using Airborne LiDAR and after conversion to canopy height model:
I have been studying these three separate but related problem for trees for a while now. After years of researchers in the community trying to solve these problems completely, it is still not completely solved. It is actually an extremely difficult task. One biggest challenge is the high variations of intra-class and small variations of inter-class.
Given a common pipeline is Detection -> Segmentation -> Classification, errors always propagate from one process to the next one. This approach suffer from this inherit architecture problem. However, the benefit is that they can be trained by three separate sets of datasets diversifying features of representation. Ideally, training of each problems will then require perfectly annotated dateset which is very rare.
I really enjoyed the keynote speech (https://hed.esri.ca/gisconference/keynote/) from Professor Michael Batty speaks about smart city and big data.
I spoke about LiDAR pre-processing, single tree delineation, and deep multi-task learning classification:
