International Journal of

Geo-Information

Article
UAV Low Altitude Photogrammetry for Power
Line Inspection
Yong Zhang 1 , Xiuxiao Yuan 1,2, *, Yi Fang 1 and Shiyu Chen 1
1 School of Remote Sensing and Information Engineering, Wuhan University, 129 Luoyu Road,
Wuhan 430079, China; zyong@whu.edu.cn (Y.Z.); fangyi@whu.edu.cn (Y.F.); csy_hy@whu.edu.cn (S.C.)
2 Collaborative Innovation Center of Geospatial Technology, Wuhan 430079, China
* Correspondence: yuanxx@whu.edu.cn; Tel.: +86-27-6877-1228

Academic Editor: Wolfgang Kainz

Received: 21 October 2016; Accepted: 5 January 2017; Published: 12 January 2017

Abstract: When the distance between an obstacle and a power line is less than the discharge distance,
a discharge arc can be generated, resulting in the interruption of power supplies. Therefore, regular
safety inspections are necessary to ensure the safe operation of power grids. Tall vegetation and
buildings are the key factors threatening the safe operation of extra high voltage transmission
lines within a power line corridor. Manual or laser intensity direction and ranging (LiDAR) based
inspections are time consuming and expensive. To make safety inspections more efficient and flexible,
a low-altitude unmanned aerial vehicle (UAV) remote-sensing platform, equipped with an optical
digital camera, was used to inspect power line corridors. We propose a semi-patch matching algorithm
based on epipolar constraints, using both the correlation coefficient (CC) and the shape of its curve to
extract three dimensional (3D) point clouds for a power line corridor. We use a stereo image pair from
inter-strip to improve power line measurement accuracy by transforming the power line direction to
an approximately perpendicular to epipolar line. The distance between the power lines and the 3D
point cloud is taken as a criterion for locating obstacles within the power line corridor automatically.
Experimental results show that our proposed method is a reliable, cost effective, and applicable
way for practical power line inspection and can locate obstacles within the power line corridor with
accuracy better than ±0.5 m.

Keywords: UAV remote sensing; power line inspection; dense matching; automatic detection of
obstacles in power line corridor

1. Introduction
The expansion of the 500 kV extra high voltage (EHV) transmission line system for inter-provincial
and large-span area power-supply tasks is presently a major project underway in China. Ensuring the
safe operation of these lines is one of the important tasks for the power maintenance department at
all levels. Buildings and tall plants, such as eucalyptus and bamboo, are typical obstacles threatening
the safe operation of EHV transmission lines within power line corridors. Thus, power maintenance
companies must devote a great deal of labor and material resources to power line corridor inspection,
striving for the timely elimination of these safety risks.
Existing methods for power line inspection fall into three categories; manual, airborne LiDAR,
and UAV inspection approaches. In manual ground-based inspection, each section in a segment of
a power line corridor is assigned to a fixed inspection team. Manual visual interpretation is used
to find and record obstacles along the power line corridor. It is difficult to conduct manual ground
inspection for the erection of EHV transmission lines because EHV transmission lines are mainly
located in areas that lack transportation. Obstacle interpretation depends on the conscientiousness
and experience of inspection personnel, making inspection subjective. Moreover, manual inspection is

ISPRS Int. J. Geo-Inf. 2017, 6, 14; doi:10.3390/ijgi6010014 www.mdpi.com/journal/ijgi

ISPRS Int. J. Geo-Inf. 2017, 6, 14 2 of 16

dependent on visibility, so blind spots are unavoidable [1]. In airborne LiDAR power line inspection,
a laser scanner is put onto a manned helicopter to obtain 3D point clouds of the objects and power line
within the power line corridor. During post processing, the point cloud data is classified to extract
objects and power lines within the line corridor. The distance between objects and power lines is used
to locate obstacles. Existing research shows that although airborne LiDAR point clouds yield more
accurate and less subjective inspection results than manual inspection [2], expensive airborne LiDAR
systems have high maintenance costs [3] in practical operation. This seriously restricts the application
and popularization of this method. For UAV power line inspection in China, the regulations on the
UAV payload, flight height, velocity, and flight range are very strict. UAV flights must fly within visual
range. If UAV flight does not meet the regulations, a license must be acquired from authority, and this
will cause inconvenience and delays for power line inspection. However, UAVs have the advantages of
low cost and ease of operation so are an attractive and popular option for power line inspection [4–6],
and both fixed wing UAVs and multi-rotor UAVs have been adopted. UAV systems are flexible; digital
cameras as well as video cameras are carried as UAV sensors [7]. Paper [8] comprehensively analyzed
how computer vision technology is used in the operation and maintenance of transmission lines.
Little attention has been paid to using low-altitude UAV images for the accurate determination of
distance between power lines and ground objects and for the automatic detection of obstacles found in
power line corridors. In this paper, we propose a method of UAV power line inspection, through the
automated detection of obstacles using photogrammetric approaches, thus fully exploiting the potential
of UAV power line inspection. To use UAV to detect the obstacles within the power line corridor, three
key problems need to be solved; 3D reconstruction of the ground of the power line corridor, power line
measurement, and automatic recognition of obstacles. For ground reconstruction of the power line
corridor, dense image matching techniques can be adopted to extract dense point clouds of the power
line corridor; currently the extraction of dense point clouds from images obtained by low-altitude
UAV is a very active research area in fields of photogrammetry and computer vision [9–11]. In UAV
power line inspection, image-matching algorithms must be adapted to ground conditions with dense
vegetation. At the same time, algorithms must be capable of extracting tree canopies and buildings
stably and reliably. When shooting images of a power line corridor, the flight direction of the UAV
must be consistent with the direction of the electric power line. Thus, the direction of the power
line will be approximately parallel to the direction of the image’s epipolar line. In a stereo mapping
environment, it is hard to mark a power line correctly, and it is difficult to control the measurement
error. Zheng et al. [12] proposed a method for power line positioning under the constraint of vertical
line, yet there still exist large measurement errors. As a target moves up and down along the vertical
line, the measure cursor moves along the epipolar line on the left and right images, and thus causes
a measurement error.
In this paper, semi patch matching based on epipolar constraints (SPMEC) was proposed to extract
the dense point clouds for the 3D reconstruction of ground objects in a power line corridor. It adapts
to ground conditions with dense vegetation and can successfully extract tree canopies and rooftops
stably and reliably. A stereo image pair from inter-strip is adopted to improve the power line stereo
measurement accuracy, which changes the direction of the power line from approximately parallel to
approximately perpendicular to the epipolar line for greater accuracy in stereo measurements. Finally,
a power line is used as a bus bar, and a buffer space analysis is carried out according to the safety
distance set in specification code so obstacles can be detected automatically.

2. Automatic Detection of Obstacles in Power Line Corridors

All the objects in the power line corridor with a distance to the power line less than the safety
distance are defined as obstacles. If the 3D surface reconstruction of a power line corridor is complete,
the power line measured, and a safe distance threshold given, then obstacles can be automatically
located. Low-altitude UAV images, obtained after global navigation satellite system (GNSS) supported
aerial triangulation [13], are the objects to be processed. First, SPMEC was proposed to extract
ISPRS Int. J. Geo-Inf. 2017, 6, 14 3 of 16

the dense
ISPRS point 2017,
Int. J. Geo-Inf. clouds
6, 14within the corridor for 3D reconstruction of the ground objects, including 3 of 15
canopies, buildings, and other ground attachments. Then, manual power line measurement was
measurement
applied waspower
to extract applied linetoinextract power pairs
stereo image line in stereo image
composed pairs composed
of corresponding of corresponding
images from different
images from different strips. Finally, along the power line direction at a set
strips. Finally, along the power line direction at a set interval, we calculate the distance interval, we calculate
betweenthe
distance
the power between
line andthethepower
surface line andcloud
point the surface point cloud
to determine to determine
whether it is withinwhether
the safe it is within
distance the
range.
safe distance range. An obstacle is identified when ground points are located within
An obstacle is identified when ground points are located within a distance less than the prescribed a distance less
than the
safety prescribed
distance safety distance threshold.
threshold.

2.1. Semi Patch Matching Algorithm Based on Epipolar Constraints

thispaper,
In this paper,epipolar
epipolar images
images areare
the the processing
processing unit aand
unit and coarse a coarse
to fine to finematching
image image matching
strategy
strategy
was was In
adopted. adopted. In ourmethod,
our proposed proposed undermethod, under
the initial the constraint,
parallax initial parallax
a largeconstraint, a large
matching window
matching window searches a one-dimensional image along the epipolar line
searches a one-dimensional image along the epipolar line and is used to extract the coarse matching and is used to extract
the coarse
seed points.matching seedispoints.
If a parallax If a parallax
continuous is continuous
within an within
object and the sameanobject
objecthas
and the same object
a consistent texturehas
in
a consistent
the image, then texture in the image,
the coarse matching then thepoint
seed coarseis matching
considered seed
the point
centerisofconsidered the center
the fine matching of the
window.
fine
In thematching
segmented window.
image, In the on
based segmented image,object
the segmented basedofon
thethe
seed segmented object
point, a patch of the seed
matching point, isa
constraint
patch matching
constructed constraint is constructed
(the segmentation (the segmentation
result of an object only occupiesresult
part of ofthe
an area
object only the
within occupies
w × wpart of
pixels
the area within the w × w pixels range around the seed point, so it is called a semi
range around the seed point, so it is called a semi patch). Then, the initial parallax of the points to be patch). Then, the
initial parallax
matched within of thethe points
semi patchtoisbe matched within
determined accordingthetosemi patch is determined
the geometric conditions of according
the semi to the
patch.
geometric
A conditions
one-dimensional of the
search is semi patch.within
conducted A one-dimensional
a smaller search search
range.is conducted
Finally, an within a smaller
outlier detection
search range.
algorithm Finally,toaneliminate
is applied outlier detection algorithm
the mismatched is applied to eliminate the mismatched points.
points.
(1) Construction of Parallax Triangulation Network
Triangulation Network
Relative orientation
Relative orientationisisconducted
conducted using
using thethe automatically
automatically measured
measured image image tie points
tie points in the in the
stereo
stereo image pairs, and epipolar image pairs are generated. Meanwhile, the
image pairs, and epipolar image pairs are generated. Meanwhile, the tie points are projected to the tie points are projected
to the epipolar
epipolar image,image,
obtainingobtaining the coordinate
the coordinate and parallaxand parallax of point
of the tie the tiepipoint
(x, y) p(ii (x,
= 1,y)2,(i...,
= 1,n)2,in…,
then)left
in
the left epipolar image, represented by P (x , y , d ). According to the generation
epipolar image, represented by Pei (xei , yei , dei ). According to the generation algorithm of an irregular
ei ei ei ei algorithm of an
irregular triangulation
triangulation network
network [14], [14], a triangulated
a triangulated irregularirregular network network (TIN) is constructed
(TIN) is constructed using n
using n relative
relative orientation points, and the corresponding pseudo colors are given
orientation points, and the corresponding pseudo colors are given to the triangle vertexes, according to the triangle vertexes,
according
to to the of
the difference difference of the
the parallax parallax
value dei . Avalue
parallax dei. ATIN parallax TINinisFigure
is shown shown1.in Figure 1.

Figure 1. Initial Parallax Triangulation Irregular Network.

In Figure
In Figure 1,
1, triangle
triangle vertexes
vertexes are
are composed
composed ofof relative
relative orientation
orientation points,
points, and
and the
the corresponding
corresponding
pseudo colors are given to the triangle vertexes according to the difference of the parallax value.
pseudo colors are given to the triangle vertexes according to the difference of the parallax value.
(2) Stereo
(2) Stereo Image
Image Matching Based on
Matching Based on aa Parallax
Parallax Controlled
Controlled Grid
Grid
First, a parallax controlled grid Pegi (xegi, yegi) is set to an interval of 15 pixels in the left epipolar
First, a parallax controlled grid P (xegi , yegi ) is set to an interval of 15 pixels in the left epipolar
image. Then, grid points are put intoegia parallax TIN, and the triangle ΔPeiPejPek where Pegi lies is
image. Then, grid points are put into a parallax TIN, and the triangle ∆Pei Pej Pek where Pegi lies is
obtained. Bilinear interpolation is conducted on the parallaxes of the three vertices of the triangle
[15] to obtain the initial parallax of Pegi. Then 61 × 61 pixels are used as the matching window. Slide
within [xegi − 100, xegi + 100] in the right epipolar image; for each pixel, the normalized correlation
ISPRS Int. J. Geo-Inf. 2017, 6, 14 4 of 16

obtained. Bilinear interpolation is conducted on the parallaxes of the three vertices of the triangle [15]
ISPRS
to Int. J.the
obtain Geo-Inf. 2017,
initial 6, 14
parallax 4 of 15
of Pegi . Then 61 × 61 pixels are used as the matching window. Slide within
[xegi − 100, xegi + 100] in the right epipolar image; for each pixel, the normalized correlation coefficient
coefficient
(CC) (CC) of the corresponding
of the corresponding windowaccording
window is calculated is calculated according
to Equation to Equation
(1), and (1),is and
a CC curve a CC
generated
curve is generated
as shown in Figure 2. as shown in Figure 2.
n n

∑ 
n n
[g ( j, i) - g ]× [g ( j + c, i + r ) − g ]
∑ [ g1 ( j, i ) − g1 ] × [ g2 ( j + c, i + r ) − g2 ]
1 1 2 2
i =0 ij==00 j = 0
ρc,rρ= =
c,r s (1)
(1)
n nn n nn nn


∑ ∑ [ g[1g(1j,( ij), i−
i =0i =j0=0j = 0
2

) - g1 ]] ×× ∑ ∑[[gg22((jj ++cc, r )r−) g−2 ]g2 ]
, i i++
i =0 j =0
i =0 j =0
2 2

where (c, r) are the

the coordinates
coordinates of the the corresponding
corresponding pixel of the center point pixel of the the matching
matching
window in thethe right
right epipolar
epipolar image,image, and the values gg11 and g22 are the average gray levels of the
the values
corresponding matching window in the left and right epipolar image, respectively. The size of the
matching windowisisn and
matching window n andi, j i,
is jthe
is line
the number and column
line number numbernumber
and column of the pixels within
of the thewithin
pixels matching
the
window, and g
matching window,1 ( j, i ) and g21 (j ,i ) and g 2 (j ,i ) are the gray values of the pixel (j, i) in the left and
and g ( j, i ) are the gray values of the pixel (j, i) in the left and right right epipolar
image, respectively.
right right epipolar image, respectively.

1.0
ρmax
0.8
Normalized correlation coefficient ρ

0.6

0.4

0.2

0.0

-0.2

-0.4

-0.6 ρl
ρr
-0.8
1 21 41 61 81 101 121 141 161 181 201
Region of search (pixels)

Figure 2. CC Curve.
Figure 2. CC Curve.

In Figure2,2,ρ ρ max
In Figure is the peak value of the normalized CC of the matching window and
max is the peak value of the normalized CC of the matching window and searching
searchingatwindow
window at the corresponding
the corresponding pixel.
pixel. ρl and ρl the
ρr are ρr are theCC
andnormalized normalized CC of
of the lowest the lowest
points of the
monotone decrease on both sides of the peak value respectively. To determine if the matching
points of the monotone decrease on both sides of the peak value respectively. To determine if the is
successful
matching iswe apply Equation
successful we apply(2) Equation
as follows:
(2) as follows:
successful
when, ρmax
when ≥ρ0.9
max ≥ 0.9

 success f ul,


successful
success f ul, when, ρmax
when <ρ0.9
max and l − ρρlr −
< 0.9|ρand |<ρ r 0.2
< 0and l −llrr| <
.2 and|ll l− <55.0
.0 and
and[ ρ[max
ρmax- ( ρ− ρlr +
l +( ρ )/2]ρr≥)/2
0.6] ≥ 0.6 (2)
(2)
 f ailed,  failed

,
else else


where llll and lrr are the distances in pixels between the peak value and the lowest points on both sides
where
respectively.
of the peak value respectively.
It can be seen from Equation (2), in contrast to most matching algorithms based on a normalized
CC, when designing
designingsimilarity
similaritycriterion,
criterion,we
weuse
usenot
notonly thethe
only value of of
value thethe
normalized
normalizedCCCC
butbut
alsoalso
the
curve
the characteristics
curve of the
characteristics ofCC.
the During image
CC. During correlation,
image whenwhen
correlation, sliding the matching
sliding window,
the matching if the
window,
corresponding
if the correspondingimageimage
point is found
point in the search
is found region region
in the search of the right
of theimage, then thethen
right image, CC the
willCC
change
will
following a law from small to large and then from large to small as the matching window gets closer
and moves far away from the corresponding point. Isolated peaks are evident in the data; two sides
of the peak value are approximately symmetrical (as shown in Figure 2). In Equation (2),
ISPRS Int. J. Geo-Inf. 2017, 6, 14 5 of 16

change
ISPRS Int.following a law
J. Geo-Inf. 2017, 6, 14from small to large and then from large to small as the matching window 5 of 15
gets closer and moves far away from the corresponding point. Isolated peaks are evident in the data;
ISPRS Int. J. Geo-Inf. 2017, 6, 14 5 of 15
ρl −sides
two ρr < of 0.2theand − lr <are5.0
peakll value are used respectively
approximately symmetrical to constrain
(as shown the
insymmetry
Figure 2). ofIn both sides(2),
Equation of
ρ − ρr |peak;
|the
ρll −CC [
< 0.2 and
ρr < 0.2 and ( ) ]
ρmax |l−lll−−ρllrl|r+<<ρ5.0 2areare
r 5.0 ≥used
0.6
used respectively
is respectively to constrain
used to constrain thethe
symmetry
the distinctiveness
to constrain symmetry of
of both
the sides
CC
of both of the
peak.
sides of
CC peak; [ρmax − (ρl + ρr )/2] ≥ 0.6 is used to constrain the distinctiveness of the CC peak.
the CC Aspeak; [
As can be ρseen−from
can be seen (
max from ) ]
Figure
ρl +Figure ρr 23, 3,≥matching
0.6 is used
matching
window
window
1 has rich
to constrain texture
the texture
1 has rich
and the peak
distinctiveness
and theof the CC
peak
of the CC is
peak.
of the CC is
greater than 0.9 within the search window, meaning a successful match; matching window 2 also
greater Asthan
can be 0.9 seen
within fromthe Figure
search 3,window,
matchingmeaning
window a successful
1 has richmatch; matching
texture and the window
peak of 2the alsoCChas is
has rich texture for it lies on the canopy, yet it is rather homogeneous. The CC peak is only 0.2;
rich texture
greater thanfor 0.9itwithin
lies on the the search
canopy,window,
yet it is rather
meaning homogeneous.
a successfulThe CC peak
match; is only
matching 0.2; however,
window 2 also
however, there is a symmetric shape in the CC curve near the peak and the CC difference between
thererich
has is a texture
symmetric for shape
it lies in onthetheCC curve near
canopy, yet ittheis peak and
rather the CC difference
homogeneous. The between
CC peakthe is peak
only and
0.2;
the peak and the trough is greater than 0.6. It is also a successful match. Matching window 3 lies in
the troughthere
however, is greater than 0.6. It
is a symmetric is also
shape in athe
successful
CC curve match. Matching
near the peak andwindow
the CC3 difference
lies in the between
shadow;
the shadow; the peak of the CC within the search window researches 0.6, but there is no obvious
the peak and of thetheCC within
trough is the search
greater window
than 0.6. It isresearches 0.6, butmatch.
also a successful there isMatching
no obvious pattern
window of the
3 lies in
pattern of the CC curve shape, hence a failed match. From these three typical matching states, we
CC shadow;
the curve shape, henceofathe
the peak failed
CC match.
within theFrom thesewindow
search three typical matching
researches states,
0.6, but thereweiscan see that
no obvious
can see that Equation (2) can improve successful matching rates without introducing too many
Equation
pattern of(2)thecan
CCimprove
curve shape, successful
hence matching
a failed rates
match. without introducing
From these too many
three typical mismatch
matching points,
states, we
mismatch points, compared to simply setting the threshold of the CC to a lower value in order to
compared
can see thatto simply
Equation setting
(2) can the threshold of the CC to
improve successful a lower value
matching rates in order to
without density match
introducing too points.
many
density match points.
mismatch points, compared to simply setting the threshold of the CC to a lower value in order to
density match points.

(a) (b)
(a) considering
Figure 3. Similarity judgment
Figure 3. Similarity judgment
considering the shape of CC curve. (a) Matching (b) (b)window
the shape of CC curve. (a) Matching window
(b)
Correlation
Correlation
coefficient coefficient curve.
Figure 3. curve.
Similarity judgment considering the shape of CC curve. (a) Matching window (b)
Correlation coefficient curve.
(3) Semi Patch Dense Image Matching
(3) Semi Patch Dense Image Matching
We use
(3) Semi Graph-Based
Patch Dense Image image segmentation method [16] to segment the left epipolar image and
Matching
the segmentation result is shown in Figure 4. method [16] to segment the left epipolar image and the
We use Graph-Based image segmentation
We use Graph-Based
segmentation image
result is shown segmentation
in Figure 4. method [16] to segment the left epipolar image and
the segmentation result is shown in Figure 4.

(a) (b)
(a)Segmentation Result. (a) Original image (b) Segmented
Figure 4. Image (b) image.

Figure
For a pixel 4. Image
pi to P =Result.
Segmentation
be matched, {
p1 , p(a) }
Figure 4. Image Segmentation Result. (a) Original image (b) Segmented image.
2 ,..., pn , the set of grid nodes located in the
Original image (b) Segmented image.

corresponding
For a pixelsegmented
p to be area
i
within the
matched, {p , p ,...,
P =parallax p } ,gird
regular the network, D = {dlocated
and nodes
set of grid , d ,..., din}, the
the1 2 n
1 2 n
For a pixel pi to be matched, P = { p1 , p2 , ..., pn }, the set of grid nodes located in the corresponding
parallax of pixels in the set,
corresponding
segmented areasegmented
within thearea
can be obtained.
within
parallax the parallax
regular
Since the pixels
regular
gird network, gird
andnetwork,
D = {d1and D d=n }{,dthe
in the same-segmented
, d2 , ..., ,..., d n }have
areas
1 , d 2parallax , the
of
similar texture features in a local neighborhood of objective space, assuming that the regular object
pixels in of
thepixels
set, can
in be
theobtained.
set, can Since the pixels in the
thesame-segmented areas have similar texture
parallax be obtained.
( Since
) pixels in the same-segmented
surface has a consistent image texture, π px , p y , d , the parallax space plane can be constructed using
similar texture features in a local neighborhood of objective space, assuming that the regular object
areas have

the image
surface haspoint coordinates
a consistent imagewithin
texture,the
π pset (
P and
x , py , d
the
, the )corresponding
parallax parallax.
space plane can bePut pi into πusing
constructed , the
plane equation; then di, the initial parallax, can be obtained. Then taking 11 × 11 pixels as a matching
the image point coordinates within the set P and the corresponding parallax. Put pi into π , the
window, slide within [xegi − 20, xegi + 20] in the right epipolar image. For each pixel, the normalized
plane equation; then di, the initial parallax, can be obtained. Then taking 11 × 11 pixels as a matching
ISPRS Int. J. Geo-Inf. 2017, 6, 14 6 of 16

features in a local neighborhood of objective space, assuming that the regular object surface has


a consistent image texture, π p x , py , d , the parallax space plane can be constructed using the image
point coordinates within the set P and the corresponding parallax. Put pi into π, the plane equation;
ISPRS d
then i , the
Int. initial
J. Geo-Inf. parallax,
2017, 6, 14 can be obtained. Then taking 11 × 11 pixels as a matching window, 6 of 15
slide within [xegi − 20, xegi + 20] in the right epipolar image. For each pixel, the normalized CC of the
corresponding window iswindow
CC of the corresponding calculated according according
is calculated to Equationto (1). Then we
Equation (1).determine if the matching
Then we determine if the
was successful
matching according to
was successful Equationto(2).
according A comparation
Equation of the matched
(2). A comparation point
of the cloud with
matched pointand without
cloud with
using the semi
and without patch
using theis semi
shown in Figure
patch is shown5. in Figure 5.

(a) (b)
Figure 5.
Figure 5. Comparation
Comparation of
of point
point cloud
cloud with/without
with/without using
using the semi patch
the semi patch dense
dense match.
match. (a)
(a) Match
Match
result without using semi patch (b) Match result using semi patch.
result without using semi patch (b) Match result using semi patch.

From Figure 5 we can see that the point cloud using the semi patch has fewer holes in the road
From Figure 5 we can see that the point cloud using the semi patch has fewer holes in the
and rooftop areas than that without using semi patch. Holes are caused by the homogenous texture
road and rooftop areas than that without using semi patch. Holes are caused by the homogenous
in the match window and the initial parallax of the surounding objects. After using the semi patch,
texture in the match window and the initial parallax of the surounding objects. After using the semi
because the initial parallax of the match window is determined by the corresponding segment, the
patch, because the initial parallax of the match window is determined by the corresponding segment,
successful match rate improves, especially in roads and rooftop areas. The successful match rate in
the successful match rate improves, especially in roads and rooftop areas. The successful match rate in
vegetation area improves as well.
vegetation area improves as well.
(4) Eliminating Mismatched Points
(4) Eliminating Mismatched Points
There always exists some outlier points in the point cloud, causing false alarms in power line
corridor
There inspection,
always existswhichsomeneed to be points
outlier eliminated.
in theThe point distance
cloud, between the outlier
causing false alarmspoints
in powerandline
the
rest points is different from the distance between non-outlier points and the rest
corridor inspection, which need to be eliminated. The distance between the outlier points and the rest points. For a point
cloud is
points containing
different fromN points, if the between
the distance longest non-outlier
distance L points amongand those non-outlier
the rest points. For points and
a point the
cloud
containing N points, if the longest distance L among those non-outlier points and the probability of thea
probability of the outlier b are set, a non-outlier point count, M, can be estimated. Then we count
numberb NUM
outlier are set,for each point PT
a non-outlier in the
point pointM,cloud.
count, can be If estimated.
NUM is less thanwe
Then M,count
an outlier is found.
a number NUM NUMfor
is calculated
each point PTasinfollows:
the pointforcloud.
each Ifpoint
NUM PTisinless
thethanpoint M,cloud, the initial
an outlier NUM
is found. NUM is set to zero, then
is calculated as
iterate through the point cloud. If the distance between current point and PT is
follows: for each point PT in the point cloud, the initial NUM is set to zero, then iterate through the less than L, add one
to NUM.
point cloud. If the distance between current point and PT is less than L, add one to NUM.
Assume that
Assume that the
thepoint
pointto tobebematched
matchedisisp(x,
p(x,y,y,z);z);the
thematching
matching T isTexpressed
results
results is expressed asset
as the theofset
p.
of p . If the set has a total of n points, the distance between any two points
If the set has a total of n points, the distance between any two points in the set can be expressed as: in the set can be
expressed as: q
( x2i − x j )2 + (yi −2 y j )2 + (zi − z j 2)2
(p , p ∈ T)


l (i, j) = pi , p j ∈ T (3)
l(i, j) = (xi − x j ) + (yi − y j ) + (zi − z j ) i j (3)

Assume that the probability of mismatched points in the set of matched points is b, so the
number of correctly matched points that may be included in the point set is M = (1.0 − b ) × n . For
o, any point in T , if L, the distance, is given, N, the number of points within the range with o as the
origin, L as the radius can be obtained. If N < M , o is an outlier; otherwise, it is a correctly matched
point. Thus, the criterion used in this paper depends on the two parameters of b and L. Experiments
ISPRS Int. J. Geo-Inf. 2017, 6, 14 7 of 16

Assume that the probability of mismatched points in the set of matched points is b, so the number
of correctly matched points that may be included in the point set is M = (1.0 − b) × n. For o, any point
in T, if L, the distance, is given, N, the number of points within the range with o as the origin, L as
the radius can be obtained. If N < M, o is an outlier; otherwise, it is a correctly matched point. Thus,
the
ISPRScriterion used2017,
Int. J. Geo-Inf. 6, 14 paper depends on the two parameters of b and L. Experiments have shown
in this 7 of 15
that it is more appropriate to take b = 20%, and L is derived by using RANSAC [17] method. For the
standard
ith iteration,deviation
randomly of the distance
sample between
a point from this point cloud,
the point and every
and point in the
calculate theremaining point cloud.
standard deviation of
After
the removing
distance thosethis
between points whose
point distance
and every is greater
point than threepoint
in the remaining timescloud.
of this standard
After removingdeviation,
those
take the
points whose biggest
distancedistance value
is greater thanasthree
thetimes
Li. We sample
of this thedeviation,
standard point cloud for biggest
take the 500 time and
distance
value as the L{
L = argmin iLi ; i = 0...500 .
. We sample the}point cloud for 500 time and L = argmin { L i ; i = 0, ..., 500 } .
Following
Following the 4 steps above, taking the epipolar images as objects to be
the 4 steps above, taking the epipolar images as objects to be matched,
matched, the the image
image tie
tie
points obtained by GNSS-supported aerial triangulation were used as an
points obtained by GNSS-supported aerial triangulation were used as an initial condition; ainitial condition; a triangular
irregular
triangularnetwork
irregularcontrolled by initialby
network controlled parallax was constructed.
initial parallax After parallax
was constructed. controlled
After parallax grid
controlled
interpolation,
grid interpolation, semi-patch matching,
semi-patch and mismatched
matching, and mismatchedpointspoints
eliminating, groundground
eliminating, dense pointdenseclouds
point
can be obtained as shown in Figure 6. The texture color of the point clouds
clouds can be obtained as shown in Figure 6. The texture color of the point clouds are taken from are taken from the left
the
epipolar image.
left epipolar image.

1 2

1 crown of trees

2 roofs

Figure 6. Dense 3D Point Clouds Extracted by SPMEC (the three colorful lines represent three power lines).
Figure 6. Dense 3D Point Clouds Extracted by SPMEC (the three colorful lines represent three
power lines).
As seen in Figure 6, the SPMEC achieves 3D ground reconstruction of the power line corridor,
with morphologically intact canopies, regular building edges, and clear small building components
As seen in Figure 6, the SPMEC achieves 3D ground reconstruction of the power line corridor,
such as chimneys and solar water heaters providing an accurate digital surface model for automated
with morphologically intact canopies, regular building edges, and clear small building components
detection of the obstacles within power line corridors. For more information about SPMEC, please
such as chimneys and solar water heaters providing an accurate digital surface model for automated
see Video S1 (Supplementary Materials). The video demonstrates the reconstruction of three typical
detection of the obstacles within power line corridors. For more information about SPMEC, please see
landscapes within power line corridor using SPMEC: the first scene includes both the power line and
Video S1 (Supplementary Materials). The video demonstrates the reconstruction of three typical
ground surface, second scene is of a rural settlement and the third scene is of a mountainous area.
landscapes within power line corridor using SPMEC: the first scene includes both the power line and
ground surface,
2.2. Power second sceneBased
Line Measurement is of on
a rural
Stereosettlement
Image Pairand
fromthe third scene is of a mountainous area.
Inter-Strips
ThereLine
2.2. Power are Measurement
two flight strip images
Based taken
on Stereo by Pair
Image UAV during
from power line corridor inspection; each
Inter-Strips
side of the power line has one strip. The epipolar line direction of the stereo images pair within a
There are two flight strip images taken by UAV during power line corridor inspection; each side of
strip is almost parallel to the power line, making it difficult for measure cursor to be placed on the
the power line has one strip. The epipolar line direction of the stereo images pair within a strip is almost
corresponding points on the power line, and the power line measurement may fail.
parallel to the power line, making it difficult for measure cursor to be placed on the corresponding
As shown in Figure 7, there are two flight strips along power line. Pyramids stand for the
points on the power line, and the power line measurement may fail.
position and orientation of the images. Pseudo color points are the pass points. During manual
As shown in Figure 7, there are two flight strips along power line. Pyramids stand for the
stereo observation, the height of the measurement cursor is adjusted by the x-parallax. Because the
position and orientation of the images. Pseudo color points are the pass points. During manual
direction of the power line is approximately parallel to the epipolar line, when adjusting the
stereo observation, the height of the measurement cursor is adjusted by the x-parallax. Because the
x-parallax, the movement track of the measurement cursor coincides with the epipolar line (shown
in Figure 8a), making it hard to measure the corresponding point on the power line. To make it easy
for stereo measure, we should make the epipolar line of the stereo image pair perpendicular to the
power line, such stereo image pairs can be acquired by using the images on different sides of the
power line.
ISPRS Int. J. Geo-Inf. 2017, 6, 14 8 of 16

direction of the power line is approximately parallel to the epipolar line, when adjusting the x-parallax,
the movement track of the measurement cursor coincides with the epipolar line (shown in Figure 8a),
making it hard to measure the corresponding point on the power line. To make it easy for stereo
measure, we should make the epipolar line of the stereo image pair perpendicular to the power line,
such
ISPRS stereo image
Int. J. Geo-Inf. pairs
2017, 6, 14can be acquired by using the images on different sides of the power line.8 of 15

ISPRS Int. J. Geo-Inf. 2017, 6, 14 8 of 15

Figure 7. Image position on both sides along the power line.

Figure 7. Image position on both sides along the power line.
Figure 7. Image position on both sides along the power line.

(a) (a)

(b)
(b)
Figure8.8. Stereo
Figure Stereo image
imagepair
pairforfor
power lineline
power measurement. (a) Stereo
measurement. image pair
(a) Stereo form
image inter-strips
pair (b)
form inter-strips
Stereo
(b) image
Stereo pair
image pairform inter-strips.
form
Figure 8. Stereo image pairinter-strips.
for power line measurement. (a) Stereo image pair form inter-strips (b)
Stereo image pair form inter-strips.
As seen in Figure 8b, by using stereo image pairs that form inter-strips for power line
As seen in Figure
measurement, the base8b,line
bydirection
using stereo image
of the pairs
stereo imagethat form
pair inter-strips forperpendicular
is approximately power line measurement,
to the
the As seen in Figure 8b, by using stereo image pairs that
direction of the power line. Therefore, the epipolar direction of the stereo to
base line direction of the stereo image pair is approximately form inter-strips
perpendicular the for
image power
direction
pairs is of line
the
measurement,
approximately perpendicular to the direction of the power line; the intersection point of the epipolar the
the base line direction of the stereo image pair is approximately perpendicular to
direction of power
line and the the power line.
line is the Therefore, point
corresponding the on epipolar
the powerdirection
line. of the stereo image pairs is
approximately perpendicular
When performing manualtomeasurement,
the directiontheof operator
the power line;
only the to
needs intersection point of cursor
adjust the measure the epipolar
line
(redand the at
cross) power line is the corresponding
the corresponding pointline.
point on the power on the
Duepower line.the power line should to be
to gravity,
modeled
Whenusing a polyline
performing insteadmeasurement,
manual of a single straight line. Thus,only
the operator the needs
whole power linethe
to adjust between
measuretwocursor
ISPRS Int. J. Geo-Inf. 2017, 6, 14 9 of 16

power line. Therefore, the epipolar direction of the stereo image pairs is approximately perpendicular
to the direction of the power line; the intersection point of the epipolar line and the power line is the
corresponding point on the power line.
When performing manual measurement, the operator only needs to adjust the measure cursor
(red cross) at the corresponding point on the power line. Due to gravity, the power line should
ISPRS
to be Int. J. Geo-Inf.using
modeled 2017, 6, a14 polyline instead of a single straight line. Thus, the whole power 9line
of 15

between two posts needs to measure about 30 pairs of corresponding points to model the polyline.
of a manual measurement of the power line only depends on the ground resolution of UAV images.
The accuracy of a manual measurement of the power line only depends on the ground resolution
In order to acquire sufficient image resolution, we designed a suitable flight height using the focal
of UAV images. In order to acquire sufficient image resolution, we designed a suitable flight height
length of the camera before power line corridor inspection. Hence, for power line measurement, the
using the focal length of the camera before power line corridor inspection. Hence, for power line
only thing required is to make sure that the left and right measuring cursor align at the
measurement, the only thing required is to make sure that the left and right measuring cursor align at
corresponding power line.
the corresponding power line.

2.3. Automatic
2.3. Automatic Detection
Detection of
of Obstacles
Obstaclesin
inPower
PowerLine
LineCorridor
Corridor
Afterextracting
After extractingthe
thepower
power line
line andand
3D 3D point
point clouds
clouds ofpower
of the the power line corridor,
line corridor, we takewe
thetake the
power
power line as the bus line and construct a spatial buffer zone around power line with a safe
line as the bus line and construct a spatial buffer zone around power line with a safe distance of r distance
of r (shown
(shown in Figure
in Figure 9). An9). An obstacle
obstacle can becan be detected
detected when
when the 3D the 3Dclouds
point point intersect
clouds intersect
with thewith the
spatial
spatial buffer
buffer zone. zone.

Figure9.9. Diagram
Figure Diagram of
of Principle
Principlefor
forAutomatic
AutomaticObstacle
ObstacleDetection.
Detection.

In Figure 9, the power line is divided into n segments with δ as step distance. The terminals can
In Figure 9, the power line is divided into n segments with δ as step distance. The terminals can
be expressed as δ1 , δ2 ,..., δn , δn + 1 . When δ is small enough, each segment can be regarded as a
be expressed as δ1 , δ2 , ..., δn , δn+1 . When δ is small enough, each segment can be regarded as a straight
i (1X
1 ( Xδi + +i 1, ,ZZi )i +1 ) is
i ,iY
straight
line. line. The
The direction direction
vector vector
of the line of the
segment line segment
constituted by δi ( Xi ,constituted
Yi , Zi ) and δi+ by ,Y and

( ) [ ] [ ]
→
h i T h T
i
+ 1 , Zi + 1 is v = i +A
vδi +=1 X i +A1 , YiB C = X 1 −B Xi CYi+1=− Y Xi i + 1Z−i +1X−i
ZiYi + .1 If− there
Yi Zexisti + 1 − Z i . If there exist points
points in digital surface
point
in digitalclouds that satisfying
surface the that
point clouds following condition:
satisfying the following condition:

(At − X + X i )2 +((Bt −X Y++X Yii ))22 ++(Ct

( Bt− −Z Y++ZY
22 2
i ) ≤ +r (Ct − Z + Zi ) ≤ r
i )
2
(
At − 2
 | AX + BY +CZ −( AX + BY +CZ )|+| AX + BY +CZ −( AXi+1 + BYi+1 +CZi+1 )| (4)
 AX + BY + CZ − (AX i + BYi +i CZ ii ) + i√AX + BY + CZ − (AX i + 1 + BYi + 1 +≤CZ
A2 + B2 + C 2
δi +1 ) (4)
 ≤ δ
 A2 + B 2 + C 2
then these points are considered as obstacles. The distance between these points and the straight line
isthen
the these
distance between
points the obstacles
are considered and the power
as obstacles. line. In this
The distance way, the
between exact
these position
points and theof the detected
straight line
obstacles is determined.
is the distance between the obstacles and the power line. In this way, the exact position of the
detected obstacles is determined.

3. Results and Discussion

3.1. Experiments and Analysis of the Obstacles Detection of Power Line Corridor
In order to verify the validity of the proposed method, the authors wrote an experimental
program using the VC++ language. A section of a 220 kV power line in Yichang, Hubei Province,
China, about 16.5 km in length, was selected for an inspection experiment, which is a typically
difficult area for power line corridor inspection. A fixed wing UAV (shown in Figure 10) with a
Nikon D810 camera (focal length 50 mm, CMOS Sensor, 35.9 mm × 24.0 mm, 7360 × 4912 pixels, pixel
ISPRS Int. J. Geo-Inf. 2017, 6, 14 10 of 16

3. Results and Discussion

3.1. Experiments and Analysis of the Obstacles Detection of Power Line Corridor
In order to verify the validity of the proposed method, the authors wrote an experimental
program using the VC++ language. A section of a 220 kV power line in Yichang, Hubei Province,
China, about 16.5 km in length, was selected for an inspection experiment, which is a typically difficult
area for power line corridor inspection. A fixed wing UAV (shown in Figure 10) with a Nikon D810
camera (focal length 50 mm, CMOS Sensor, 35.9 mm × 24.0 mm, 7360 × 4912 pixels, pixel size 4.88 µm)
was adopted for low-altitude photography of the power line corridor; images were shot from a fix
distanced. The strip overlap and the side-strip overlap were both 80%. The UAV was equipped
with
ISPRSaInt.
GNSS receiver
J. Geo-Inf. 2017, 6,(GPS
14 L1/L2). The exposure point coordinates are obtained using precise10 point
of 15
position (PPP). The coordinates were applied in the GNSS-supported aerial triangulation.
The ground sample distance (GSD) was about 3.5 cm; two sorties were flown and and 1066
1066 images
images
obtained. The Theimages
imagesare areclear,
clear,and
andthe
thecontrast
contrast is moderate, which can meet the requirements
is moderate, which can meet the requirements for
for power
power lineline inspection.
inspection. TheThe exterior
exterior orientation
orientation parametersofofthe
parameters theimages
images were
were obtained
obtained by
GNSS-Supported
GNSS-Supported aerial aerialtriangulation;
triangulation;we wesplit
splitthe
thelong
longstrip
stripinto
intosegments
segments ininevery
every1010
km, in in
km, order to
order
control the bad
to control effecteffect
the bad caused by long
caused bystrip
longinstrip
bundle in adjustment, and all the
bundle adjustment, andimages were
all the processed
images were
according
processed to the procedures
according mentioned mentioned
to the procedures in Section 2.inGround
Section3D point clouds
2. Ground extracted
3D point cloudsfrom stereo
extracted
image pairs along
from stereo imageeachpairs strip andeach
along somestrip
segments of power
and some line using
segments inter-strip
of power imageinter-strip
line using pairs are shown
image
in Figure
pairs 11.
are shown in Figure 11.
In Figure
Figure 11, the groundground 3D 3D point
point clouds
clouds were
were extracted
extracted by by SPMEC,
SPMEC, the the red
red points
points are
are obstacles,
obstacles,
and the red, yellow, and blue
the red, yellow, and blue wires wires are obtained
obtained by stereo image pair
pair from inter-strip and manual
from inter-strip and manual
measurement.
measurement. The towers are are existing
existing 3D
3D models
models putput inin the
the scene
scene to
to make
make itit more
more intuitive.
intuitive.

Fuselage length: 2.1 m

Wing span: 3.2 m

Payload: 3.0 kg

Maximum operation height: 3500m

Dynamical system: dual-cylinder

two-stroke 56cc petrol engine or

brushless electric motor

Takeoff and landing

system:catapult-assisted takeoff,

roll landing or parachute landing

Figure
Figure 10.
10. Diagram
Diagram of
of fixed
fixed wing
wing UAV.
UAV.

Figure 11. Diagram of Ground 3D Point Clouds and Power Lines within the Power Line Corridor.

3.2. Detection of Obstacles in the Power Line Corridor

Based on the results shown in Figure 11, when the safety distance threshold between the power
 Takeoff and landing

system:catapult-assisted takeoff,

roll landing or parachute landing

ISPRS Int. J. Geo-Inf. 2017, 6, 14 11 of 16
Figure 10. Diagram of fixed wing UAV.

Figure 11. Diagram of Ground 3D Point Clouds and Power Lines within the Power Line Corridor.
Figure 11. Diagram of Ground 3D Point Clouds and Power Lines within the Power Line Corridor.

3.2. Detection of Obstacles in the Power Line Corridor

3.2. Detection of Obstacles in the Power Line Corridor
Based on the results shown in Figure 11, when the safety distance threshold between the power
Based on the results shown in Figure 11, when the safety distance threshold between the power
line and 3D point clouds was set to 15 m, the locations of obstacles were listed in Table 1 using the
line and 3D point clouds was set to 15 m, the locations of obstacles were listed in Table 1 using the
method mentioned in Section 2.3; one obstacle is shown in Figure 12.
method mentioned in Section 2.3; one obstacle is shown in Figure 12.

Table 1. Distribution of Some Obstacles within the Power Line Corridor (safety distance threshold is 15 m).

Tower Section Position of the Obstacle (m) Distance between the Obstacle and the Power Line (m)
133.0–135.0 14.658
180.5–185.0 13.990
188.5–189.0 14.865
K182~K183 201.5–268.0 11.182
119.5–123.5 12.137
191.5–201.5 12.518
204.0–206.0 13.809

As shown in Table 1, the first column lists number of the towers, the second column represents
the distance between the obstacles and K182 tower, and the third column shows the distance between
the obstacles and power lines.
As shown in Figure 12, an obstacle has a certain continuity in space. Therefore, in Table 1, the
position of the obstacle is expressed as a continuous range. The minimum distance within this range
is taken as the distance between an obstacle and the power line. The measurement accuracy of the
obstacles will be further verified.
the distance between the obstacles and K182 tower, and the third column shows the distance
between the obstacles and power lines.
As shown in Figure 12, an obstacle has a certain continuity in space. Therefore, in Table 1, the
position of the obstacle is expressed as a continuous range. The minimum distance within this range
is taken
ISPRS Int. as the distance
J. Geo-Inf. 2017, 6, 14between an obstacle and the power line. The measurement accuracy 12
of ofthe
16
obstacles will be further verified.

Figure 12. Diagram of an obstacle within the Power Line Corridor.

Figure 12. Diagram of an obstacle within the Power Line Corridor.

Since the power line selected for experiment is operational, it is impossible to measure the
Sinceofthe
distance thepower
powerline selected
line and the forobjects
experiment is operational,
directly. it is impossible
It is also difficult to aim attothe
measure
powerthe linedistance
with a
of the power line and the objects directly. It is also difficult to aim at the power
total station instrument. Consequently, in this paper, two methods are used to check the accuracy line with a total station
of
instrument. Consequently, in this paper, two methods are used to check
power line inspection and the measurement of ground point clouds. A field investigation was the accuracy of power line
inspection and
conducted and the
the measurement of groundbypoint
results were reviewed clouds.
manual A fieldto
inspection investigation
confirm thewas conducted
validity and the
of the method
results were reviewed by manual inspection
and the accuracy of the automatic detection. to confirm the validity of the method and the accuracy of
the automatic
Method 1: detection.
A Trimble TX8 ground laser scanner was used for 3D scanning of the tower K182
Method 1:
and its power line. A Trimble TX8GPS
Firstly, the ground laser kinematic
Real-time scanner was usedTrimble
(RTK), for 3D R10,
scanning of the
was used fortower K182
accurately
and its power
locating line. Firstly,
the position of the the
TX8GPS Real-time
scanner, kinematic
and, then, (RTK), Trimble
the orientation pointR10, was used
coordinates of for
theaccurately
TX8 were
locating the position of the TX8 scanner, and, then, the orientation point
measured and equipment orientation was completed. The 3D laser point cloud was obtained coordinates of the TX8by were
3D
measured and equipment orientation was completed. The 3D laser point
scanning of the TX8 as shown in Figure 13. We overlaid the point cloud data collected by the TX8cloud was obtained by 3D
scanningand
scanner of the
the TX8
poweras shown in Figure
line vectors 13. We
measured byoverlaid
manual the point
work. cloudthe
Taking data collected
power by the TX8
line captured by
scanner and the power line vectors measured by manual work. Taking the power
the TX8 as the ‘ground trues’, we set a section with an interval of 1 m; every section has a gravity line captured by the
TX8 as of
center thethe‘ground
point trues’, we set
cloud and anaintersection
section withpoint
an interval
of the of 1 m; every
power line. Wesection has a gravity
calculated center
the elevation
of the pointbetween
difference cloud and theangravity
intersection
centerpoint
and ofthethe power line.point.
intersection We calculated
Then, thethe elevation
error in the difference
elevation
between the gravity center and the intersection point. Then, the error in the elevation difference was
taken as criterion to evaluate the measurement precision of the power line. Several sampling points
and the elevation differences are listed as shown in Table 2.
ISPRS Int. J. Geo-Inf. 2017, 6, 14 12 of 15

difference
ISPRS was taken
Int. J. Geo-Inf. 2017, 6,as
14criterion
to evaluate the measurement precision of the power line. Several
13 of 16
sampling points and the elevation differences are listed as shown in Table 2.

Figure 13. Diagram of Point Cloud Scanned by Laser 3D Scanner at K182.

Figure 13. Diagram of Point Cloud Scanned by Laser 3D Scanner at K182.

It can be seen from Table 2 that the power line measurement based on manual work adopted in
It can be
this paper seen fromthe
transforms Table 2 that the
direction of power line measurement
the power line from beingbased on manual work
approximately adopted
parallel in
to the
this paper transforms the direction of the power line from being approximately parallel
epipolar direction to being approximately perpendicular to the epipolar direction, so that the to the epipolar
direction
measuring to cursor
being approximately perpendicular
is accurately aligned to power
with the the epipolar direction,
line while so that
moving the the
along measuring
epipolarcursor
line.
is accurately aligned with the power line while moving along the
Manual measurement accuracy of the elevation reached ±0.326 m on the ground.epipolar line. Manual measurement
accuracy of the elevation reached ±0.326 m on the ground.
Table 2. Distribution of Elevation Difference of Sampling Points in the Power Line Profile (Unit: m).
Table 2. Distribution of Elevation Difference of Sampling Points in the Power Line Profile (Unit: m).
Elevation of 3D Point Cloud Gravity Center Elevation of Manual Measurement Elevation Difference
138.313
Elevation of 3D Point Cloud Gravity Center 138.720Measurement
Elevation of Manual Elevation0.407
Difference
135.709 135.644 −0.065
138.313 138.720 0.407
133.423 133.632 0.210
135.709 135.644 −0.065
131.511 131.414 −0.097
133.423 133.632 0.210
129.975
131.511 130.411
131.414 0.436
−0.097
128.813
129.975 128.804
130.411 −0.009
0.436
128.027
128.813 128.164
128.804 0.137
−0.009
128.027
128.632 128.164
128.053 0.137
−0.579
128.632
129.721 128.053
129.765 −0.579
0.044
129.721
131.185 129.765
130.857 0.044
−0.328
131.185
133.024
130.857
133.308
−0.328
0.284
133.024 133.308 0.284
135.237 135.683 0.446
135.237 135.683 0.446
137.826
137.826 137.376
137.376 −−0.450
0.450
140.790
140.790 141.163
141.163 0.373
0.373
Root Mean Square Error of the Elevation Difference: 0.326
Root Mean Square Error of the Elevation Difference: 0.326

Method 2: 20 distinct ground objects were selected within the power line corridor, whose 3D
Method
ground 2: 20 distinct
coordinates ground objects
were measured in the were selected
field by GPS within
RTK.theThepower line corridor,
accuracy whoseand
of planimetry 3D
ground coordinates
elevation werethe
both reached measured in the
centimeter field
level inby
theGPS RTK.[18],
ground The accuracy of planimetry
so were regarded and elevation
as ‘ground trues’.
both reached the centimeter level in the ground [18], so were regarded as ‘ground
The 3D coordinates obtained in the stereo measuring environment were compared with these true trues’. The 3D
coordinates obtained in the stereo measuring environment were compared with these
values one by one to evaluate the accuracy of the automatically extracted dense 3D point cloud. true values
one by one to
Differences evaluate
between thethe accuracy of
coordinates ofthe
all automatically extracted
the checking points densein
are listed 3DTable
point3.cloud. Differences
between the coordinates of all the checking points are listed in Table 3.
Table 3. Difference of Coordinates of the Checking Points (Unit: m).

Checking Point X Y Planimetry Elevation

1 0.020 −0.034 0.040 0.383
2 0.043 −0.014 0.045 0.126
3 0.043 0.007 0.044 0.103
4 −0.090 −0.015 0.091 0.278
5 −0.011 −0.076 0.077 0.156
ISPRS Int. J. Geo-Inf. 2017, 6, 14 14 of 16

Table 3. Difference of Coordinates of the Checking Points (Unit: m).

Checking Point X Y Planimetry Elevation

1 0.020 −0.034 0.040 0.383
2 0.043 −0.014 0.045 0.126
3 0.043 0.007 0.044 0.103
4 −0.090 −0.015 0.091 0.278
5 −0.011 −0.076 0.077 0.156
6 −0.017 0.036 0.039 −0.182
7 −0.129 −0.088 0.156 0.408
8 0.084 −0.003 0.084 −0.133
9 −0.006 −0.001 0.006 0.005
10 0.053 0.081 0.097 0.421
11 0.013 −0.180 0.180 0.177
12 0.017 −0.024 0.029 0.162
13 −0.005 −0.099 0.100 0.276
14 0.013 0.124 0.125 −0.578
15 −0.119 −0.006 0.119 0.269
16 −0.045 0.082 0.094 −0.391
17 −0.040 0.020 0.045 −0.234
18 −0.053 −0.059 0.079 0.356
19 0.086 −0.014 0.087 −0.196
20 −0.181 0.129 0.222 −0.442
Maximum Difference −0.181 −0.180 0.222 −0.578
Root Mean Square Error 0.071 0.075 0.103 0.302

It can be seen from Table 3 that the coordinates of distinct objects obtained by manual observation
in the stereo measuring environment are quite consistent with the coordinates measured in the field.
The maximum differences of the planimetry and elevation of the point cloud are 0.222 m and −0.578 m,
and the root mean square errors are ±0.103 m and ±0.302 m, respectively. Further analysis shows that
if the measurement error of ‘ground trues’ was deducted, the elevation measurement accuracy obtained
by the two methods is basically the same. Consequently, the results in Table 3 can be regarded as
evidence in evaluating the accuracy of our method for extracting power line and ground point clouds.
In addition, the dense image matching process is similar to that of manual stereo measures.
Using image matching to replace the manual stereo measure and using parabolic fitting during image
matching to further increase the matching accuracy, an accuracy of ±0.29 pixels can be achieved [19],
which significantly decreases the manual measurement error. Hence, compared with the manual stereo
measure, dense image matching has a higher accuracy.
It can be seen from Tables 2 and 3 that the elevation accuracies of power line and the ground
3D point clouds were ±0.326 m and ±0.302 m, respectively. Based on error propagation principles,
the relative elevation difference accuracy of power line and ground 3D point clouds was estimated
at ±0.444 m. Our proposed method achieved distance-measuring accuracy better than ±0.5 m;
such accuracy meets the requirements of power line inspection.

4. Conclusions
In this paper, a low-altitude UAV based power line inspection method for obstacles in power line
corridors was proposed. The images processed by the GNSS-supported aerial triangulation were the
input, and the obstacle distribution report was the output. We focused on solving three problems;
3D reconstruction of the ground surface of the power line corridor, measurement of the power line,
and automatic recognition of obstacles. According to the similarity measurement, which takes the
normalized CC and the shape of the coefficient curve into consideration, dense point clouds of canopies
and buildings with regular outlined can be extracted by SPMEC in rural areas with lush vegetation.
Even small building components such as the chimneys and solar water heaters can be clearly expressed.
ISPRS Int. J. Geo-Inf. 2017, 6, 14 15 of 16

A stereo image pair from inter-strips was used to build a measurable stereo image pairs, transforming
the power line from being approximately parallel to the direction to being approximately perpendicular
to it, which has effectively improved the measurement accuracy of the power lines. Experimental
results show that our proposed approach can automatically find and accurately locate the obstacles in
the power line corridors with measurement accuracies better than ±0.5 m, providing a new means
for quantitative inspection of obstacles to the safe operation of ultra-high voltage transmission lines.
However, there are still some issues that need to be addressed in order to improve the automation
of power line inspection, especially the measurement of automatic power lines. The bottleneck of
the automation is power line extraction from images, which is still a challenging task and will be
further studied.

Supplementary Materials: The following are available online at www.mdpi.com/2220-9964/6/1/14/s1.

Video S1: Power line inspection demo. This video demonstrates the reconstruction of three typical landscapes
within power line corridor using SPMEC: the first scene includes both the power line and ground surface,
second scene is of a rural settlement and the third scene is of a mountainous area.
Acknowledgments: This work has been supported by the National Natural Science Foundation of China
(Grant No. 41371432).
Author Contributions: Yong Zhang developed the algorithm and conducted the primary data analysis, as well
as crafting the manuscript. Xiuxiao Yuan advised on and contributed to the data analysis, as well as providing
remote sensing experience and significantly editing the manuscript. Yi Fang and Shiyu Chen carried out the
experiments and advised on research question generation and the reshaping of the manuscript.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Mu, C. Research on Extraction Method of Power Line Corridor Feature Based on Multi Remote Sensing Data.
Ph.D. Dissertation, Wuhan University, Wuhan, China, 2010.
2. Kuhnert, K.D.; Kuhnert, L. Light-weight sensor package for precision 3D measurement with micro UAVs
E.G. power-line monitoring. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, XL-1/W2, 235–240.
[CrossRef]
3. Ahmad, J.; Malik, A.S.; Xia, L.; Ashikin, N. Vegetation encroachment monitoring for transmission lines
right-of-ways: A survey. Electr. Power Syst. Res. 2013, 95, 339–352. [CrossRef]
4. Chang, A.; Wang, C. Mission planning of the flying robot for powerline inspection. Prog. Nat. Sci. 2009, 19,
1357–1363.
5. Mulero-Pázmány, M.; Negro, J.J.; Ferrer, M. A low cost way for assessing bird risk hazards in power lines:
Fixed-wing small unmanned aircraft systems. J. Unmanned Veh. Syst. 2014, 2, 5–15. [CrossRef]
6. Cerón, A.; Mondragón, I.F.; Prieto, F. Visual based navigation for power line inspection by using virtual
environments. Proc. SPIE 2015, 9406, 94060J:1–94060J:7.
7. González-Jorge, H.; Puente, I.; Roca, D.; Martínez-Sánchez, J.; Conde, B.; Arias, P. UAV potogrammetry
monitoring application to the monitoring of rubble mound breakwaters. J. Perform. Constr. Facil. 2016, 30,
04014194. [CrossRef]
8. Miralles, F.; Pouliot, N.; Montambault, S. State-of-the-art review of computer vision for the management of
power transmission lines. In Proceedings of the IEEE International Conference on Applied Robotics for the
Power Industry (CARPI), Foz do Iguassu, Brazil, 14–16 October 2014.
9. Hirschmüller, H. Accurate and efficient stereo processing by semi-global matching and mutual information.
In Proceedings of the IEEE Computer Society Conference on CVPR, San Diego, CA, USA, 20–25 June 2005.
10. Yasutaka, F.; Jean, P. Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell.
2010, 32, 1362–1376.
11. Bleyer, M.; Rhemann, C.; Rother, C. Patchmatch stereo—Stereo matching with slanted support windows.
In Proceedings of the British Machine Vision Conference, Dundee, UK, 29 August–2 September 2011.
12. Zheng, S.; Hu, H.; Huang, R.; Ji, Z. Realtime ranging of power transmission line. Geomat. Inf. Sci. Wuhan Univ.
2011, 36, 704–707.
13. Yuan, X.X. A novel method of systematic error compensation for a position and orientation system.
Prog. Nat. Sci. 2008, 18, 953–963. [CrossRef]
ISPRS Int. J. Geo-Inf. 2017, 6, 14 16 of 16

14. Shewchuk, J.R. Delaunay refinement algorithms for triangular mesh generation. Comput. Geom. 2002, 22,
21–74. [CrossRef]
15. Gribbon, K.T.; Bailey, D.G. A novel approach to real-time bilinear interpolation. In Proceedings of the Delta
IEEE International Workshop on Electronic Design, Test and Applications, Perth, Australia, 28–30 January
2004; pp. 126–131.
16. Felzenszwalb, P.F.; Huttenlocher, D.P. Efficient graph-based image segmentation. Int. J. Comput. Vis. 2004, 2,
167–181. [CrossRef]
17. Fischler, M.A.; Bolles, R.C. Random Sample Consensus: A paradigm for model fitting with application to
image analysis and automated cartography. Commun. ACM 1981, 24, 381–395. [CrossRef]
18. Lemmon, T.R.; Gerdan, G.P. The influence of the number of satellites on the accuracy of RTK GPS positions.
Aust. Surviv. 1999, 44, 64–70. [CrossRef]
19. Zhang, Z.X.; Zhang, J.Q. Digital Photogrammetry; Wuhan University Press: Wuhan, China, 1997.

© 2017 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC-BY) license (http://creativecommons.org/licenses/by/4.0/).