Sei sulla pagina 1di 10

Autonomous On-board Near Earth Object Detection

P. Rajan P. Burlina M. Chen D. Edell


Dept. of Computer Science Applied Physics Laboratory Applied Physics Laboratory Applied Physics Laboratory
Johns Hopkins University Johns Hopkins University Johns Hopkins University Johns Hopkins University
purnima@cs.jhu.edu

B. Jedynak N. Mehta A. Sinha G. Hager


Department of Mathematics Applied Physics Laboratory Dept. of Electrical Dept. of Computer Science
and Johns Hopkins University and Johns Hopkins University
Statistics Computer Engineering
Portland State University Johns Hopkins University

Abstract—Most large asteroid population discovery has been approaches that we are investigating include deploying the
accomplished to date by Earth-based telescopes. It is speculated algorithm on a BAE RAD750 with 8 MB of EEPROM and
that most of the smaller Near Earth Objects (NEOs) that are less 32 MB RAM in addition to Virtex 5 FPGAs. Understanding
than 100 meters in diameter, whose impact can create substantial
city-size damage, have not yet been discovered. Many asteroids that there are processing and memory constraints on-board,
cannot be detected with an Earth-based telescope given their size our algorithm design needs to not only meet the performance
and/or their location with respect to the Sun. We are investigating objectives of detecting and identifying asteroids, but its imple-
the feasibility of deploying asteroid detection algorithms on- mentation needs to be optimized for the respective platforms.
board a spacecraft, thereby minimizing the expense and need This paper describes an agile algorithm candidate that is being
to downlink large collection of images. Having autonomous on-
board image analysis algorithms enables the deployment of a investigated as well as our use of representative real and
spacecraft at approximately 0.7 AU heliocentric or Earth-Sun simulation imagery for testing it.
L1/L2 halo orbits, removing some of the challenges associated Notable prior work includes [5], describing a reference
with detecting asteroids with Earth-based telescopes. We describe
an image analysis algorithmic pipeline developed and targeted processing system for detection and identification of asteroids
for on-board asteroid detection and show that its performance named the Pan-STARRS Moving Object Processing System
is consistent with deployment on flight-qualified hardware. (MOPS). This pipeline identifies moving objects in our solar
system and links those detections within and between night ob-
I. I NTRODUCTION servations. It attributes detections to known objects, calculates
NASA has a congressional mandate to discover all Near- initial and differentially corrected orbits for linked detections,
Earth Objects (NEOs) at least 1 kilometer in diameter. For- recovers detections when they exist, and performs orbit iden-
tunately, 95% of the NEOs larger than 1 km that have been tification. Most proposed pipelines for Earth-based detection
discovered are likely not to impact Earth. Near-Earth Object include a step to combine images into a high signal to noise
search programs [1] are currently almost exclusively accom- ratio (SNR) static-sky image that is subtracted from the current
plished by Earth-based telescopes such as MIT’s LINEAR [2] master image to obtain a difference image containing only
project, the NEAT [3] program, the Catalina Sky Survey [4], transient sources. Examples include [8], where a shift-and-add
or Pan-STARRS [5]. An exception is JPL’s spacecraft-based technique is used to improve the SNR and then synthetically
WISE telescope brought out of hibernation to characterize creating long exposure images to facilitate the detection of
NEOs in the 3.4 and 4.6 micron infrared bands [6] (called trajectories. A related shift-and-add method using a median
NEOWISE). image rather than an average image is reported in [9]. A match
It is notable, however, that the NEO that impacted filter is used for asteroid detection and matching in [10]. In
Chelyabinsk, Russia on 15 February 2013 was only about 17 [11], [12], [13], tree based searches (including KD-trees) are
meters in diameter. It is estimated that only a relatively small used for efficient linking of successive asteroid detection and
fraction of those so called “city-killing” asteroids, particularly finding sets of observation points that can be fitted with an
objects less than 100 meters in diameter, have been discovered inherent motion model, through an exhaustive search for all
to date [7]. Because of their size, atmospheric effects and their possible linkages that satisfy the model constraints.
location with respect to the Sun, some of these NEOs cannot After careful evaluation, it was found that none of these
easily be detected with an Earth-based telescope. pipelines were suitable for an on-board system . We also
Our focus here is therefore on developing an algorithm evaluated alternative approaches for efficient line detection.
that can be hosted on-board a spacecraft. The two nominal The Hough transform, see [14], allows to search for 3-
978-1-4673-9558-8/15/$31.00 ©2015 IEEE
dimensional lines in parametric space. However, our exper- faint compared to the surrounding stars, the selection of the
iments, in accordance with [12], showed that the algorithm detection threshold impacts false alarm rate. Thresholding
is sensitive to the discretization of the parameters unless a yields binary detection images. The set of all binary images
very fine grid is used, but in this case the memory and is then used to generate an intersection image that contains
computational cost requirements are prohibitive. RANSAC above threshold pixels that occur in at least two images
(Random Sample Consensus), see [15], is a generic algorithm in the sequence. This is followed by logical differencing
for identifying parametric shapes. In our case, however, the whereby we produce a set of difference images by intersecting
randomization is not appropriate since the image is typically the corresponding binary image with the logical complement
dominated by noisy structures. In [16], the authors describe a of the common intersection image. This operation provides
framework for multiscale analysis, named beamlet analysis, in a list of candidate detections (movers) for each image in
which line segments play a role analogue to the role played the sequence. While the logical differencing results in good
by points in wavelets analysis. However, the resulting pipeline detections, additional artifacts such as crater-like formations
was not efficient enough and fairly complex. In [17], the (see Fig. 9) are seen as a result of some celestial bodies
authors use a generic discrete optimization algorithm: the being over-exposed. To mitigate this artifact, we find the
branch and bound algorithm for optimizing among sub-images connected components (using 8-neighborhood connectivity) in
and perform efficient object localization. Since line segments each difference image and filter out hollow objects based on
and sub-images are both identified by two pixels, we have connected component size and extent, and store the centroids
explored modifications of the algorithm in [17] adapted to of the selected components for the next stage of the pipeline.
line segments. However, if the average computation time and Trajectory Linking The list of centroids of moving objects
memory requirements are competitive, the worst case is not in obtained from image differencing defines a set of candidate
the range of performance compatible with the requirements of rectilinear trajectories. The goal is to find a subset of centroids
an on-board system. As a consequence we have not pursued that fit a linear model. The set of filtered centroids potentially
this approach. has a high number of noisy points (falsely detected movers),
When compared to prior studies, novel and salient con- and the cardinality of the set of all candidate trajectories
tributions of our work include: (1) a simple and efficient increases exponentially with the number of detections, thus
algorithmic pipeline that can be deployed on flight-qualified requiring subsequent pruning. Given a sequence of images,
hardware; (2) a simulation engine relying on principled optics we find all the possible trajectories connecting the detections
models allowing to generate realistic space-based imagery. between all pairs of images. There are k2 pairs of end images,


where k is the number of images in the sequence, k = 3 or


II. A PPROACH
k = 4 in the experiments, and there are O(k 2 n2 ) possible
The main components of our image processing pipeline trajectories, where n is the average number of detections per
addresses detection and linking of asteroids and takes as image. We arrange the images along a cuboid, where the
input a sequence of 3 or 4 time-lapse images acquired from z axis is the time of image acquisition. We then find the
space-based platforms, as described below (also detailed in points of intersection of each of these trajectories with all the
Algorithm 1). The pipeline is a bottom-up brute force search. remaining images. Here we assume that the velocity of the
The first 3 stages: Image Pre-Processing, Image Registra- asteroid is constant within the sequence. If there is at least one
tion and Image Logical Differencing allow for identifying detection in a 5x5 window around the point of intersection,
movers, effectively removing the background noise as well we increment the vote for that candidate trajectory. If the vote
as the astronomical objects that are not moving. The final exceeds a line confidence threshold, the trajectory is marked
stage: Trajectory Linking allows to explore systematically as valid. This arrangement allows for missing detections in
all the rectilinear trajectories connecting the movers along the some of the images in the sequence. This is pertinent as
sequence of images. the asteroids spin on their axis and their brightness can vary
Image Pre-Processing Median filtering is applied to reduce drastically from one image to the next. This is especially true
impulsive noise and artifacts that depend on the acquisition. for small asteroids that this mission is intended to discover.
Image Registration Image registration is used to bring the Unlike MOPS [5] and CSS[4], we do not set an upper limit
images into alignment with a common image of reference so on the velocity of the asteroid, and hence do not risk missing
that the stars in the background line up in all images. In fast movers. Finally, we reduce false detections using Support
the case of a triplet or a quadruplet image set, the second Vector Machine (SVM) based classification.
image is used as reference. We use a similarity transformation
(translation, rotation, scaling) and/or skew (full affine trans-
formation), depending on the image acquisition protocol, to III. E XPERIMENTS
align the images in the sequence. Our registration uses mutual
information [18] to estimate the transformation parameters. We detail the experiments performed using a range of
Image Logical Differencing A global thresholding is simulated space-based imagery generated using the JHU/APL
applied to the registered image for detecting asteroids and developed Renderer and Camera Emulator (RCE) as well as
suppressing background noise. As asteroids are typically very real imagery from the NEAT and Catalina Sky Survey dataset.
Fig. 1. (Left) An image simulated by RCE. (Right) 31 simulated MWIR images super-imposed in order to visualize the trajectory of the asteroid in a single
image. The true trajectory can be seen as a faint line towards the top center of the image.

Fig. 2. The trajectories found by the pipeline are shown in green. The true location of the asteroid is marked in red. (Left) Trajectory Detection on a simulated
triplet. (Right) Trajectory Detection on a quadruplet. Adding one more image to the triplet eliminates the false positives.
Algorithm 1 Detection and Linking Algorithm for a sequence When testing on a set of 2100 triplets or quadruplets or
of images quintuplets taken from simulated space-based imagery, we
1: Apply a median filter to each image in the sequence. characterize the algorithm with regard to detections at each
2: Align all the images to a reference image (e.g. the second image of the following three successive stages: the initial detection
for a triplet) using Mutual Information based registration.
3: Perform thresholding on the registered images using a pre-defined
of objects, the detection of moving objects and the detection of
threshold to generate binary detection images. trajectories. In Fig. 3, we plot the number of detected objects at
4: Compute the logical (soft) intersection image from the binary each step of the pipeline for each selected threshold value. The
detection images. A pixel has value one if and only if at least 2 right plot in Fig. 3 shows that the use of quadruplets allows for
detection images have value 1 at the same location. a single trajectory to be found irrespective of the number of
5: Compute the logical differencing between each binary detection
image and the intersection image. detections found at prior stages, echoing the results displayed
6: Find the connected components in the difference images. Replace in Fig. 2. The false positive rate was completely abated in this
each connected component by its center of gravity. case.
For each of the k2 pairs of images in the sequence, compute

7: Fig. 4 shows the True positive vs False positive curve
the set of all trajectories connecting the candidate moving object for asteroid detections at the final stage of the pipeline for
detections from each pair of end images.
8: For each trajectory do
simulated image sequences generated with integration time of
• Find the points of intersection of each trajectory with the
90 seconds and aperture size of 0.5 m. The various points
rest of the images. along the curves were obtained by varying the threshold in
• Check for detections within a 5x5 neighborhood of the point the step 3: of the Algorithm 1. The size of the asteroids in the
of intersection in each image. Increment the line vote by 1 simulated dataset ranges from 30 meters to 1000 meters. Fig. 5
if there are detections. shows the same information stratified by asteroid radius and
• Mark the trajectory as a valid line if the vote is higher than
a preset line threshold.
SNR(signal to noise ratio). The larger asteroids are always
detected. Among the smaller ones (<100 meters), some are
undetected. SNR for an image sequence is computed as the
A. Simulated Space-Based Imagery ratio of the median of the maximum pixel value on the
asteroid in each image in the sequence to the median of the
Using RCE, asteroids are modeled as spherical blackbody-
trimmed(upper 10%) mean noise level in each image. Note that
like emitters, with a cross-sectional area that approximates the
the SNR is a good indicator of the overall performance as all
sizes of actual asteroids and surface temperatures typical of
asteroids with sequence SNR larger than 1.3 were detected.
Sun-illuminated asteroids in an Earth-like orbit. Similar to the
Fig. 7 depicts the detection percentage as a function of SNR.
way stars are modeled, the radiation emitted is modeled using
The plot on the right of Fig. 7 shows how the SNR varies
Planck’s equation.
based on asteroid radius and distance.
2hc2 1
Bλ (T ) = ǫ 5 hc (1) Finally in Fig. 8 we show the True positive vs False positive
λ e λkB T − 1 curves for various stages of the pipeline. This figure portrays
where Bλ (T ) is the spectral radiance at a given wavelength λ how the false detections reduce as we move through the
and temperature T The value ǫ is the emissivity of the asteroid, pipeline.
which essentially converts the blackbody spectral radiance into
B. False alarm mitigation using Machine Learning
spectral irradiance. The constant, h is the Planck’s constant,
c is the speed of light, λ is wavelength, kB is the Boltzmann In order to reduce the number of false trajectories, we
constant, and T is the temperature. experimented using Support Vector Machines(SVM) that dif-
The asteroids are assumed to have a nominal temperature ferentiate between “good” vs. “bad” image sequences. The
of 200 K due to solar heating and emissivities in the range feature vector for classification was generated by taking a
from 0.9 to 0.98. The RCE uses stellar data available as part 5 × 5 window around each detection in the images to form
of the Two Micron All Sky Survey (2MASS) [19], a stellar a vector of length 100 for a quadruplet(75 for a triplet). The
survey that scanned the entire sky in three IR bands (centered asteroid ground truth was used to generate a 0/1 label for each
at 1.25 µm, 1.65 µm, and 2.17 µm, respectively). The 2MASS sequence. Tables I and II show the 10-fold cross-validation
catalog also incorporates data in two visible bands from results on 180 image sequences generated with integration
other surveys. An example of the simulated Mid-wavelength time of 90 seconds and aperture size of 0.5 m.
Infrared (MWIR) image and the ground truth trajectory derived Kernel Overall True positive False
from one set of simulated imagery is shown in Fig. 1. Fig. 2 accuracy(%) accuracy(%) positive
shows the final trajectories detected by our algorithm for one accuracy(%)
triplet and one quadruplet of the simulated MWIR dataset. linear 99.5 87.07 99.88
chi-squared 99.77 95.87 99.89
As is shown in Fig. 2, using trajectory verification on a rbf 99.25 100 99.23
greater number of images in the sequence allows us to quickly
TABLE I
disambiguate and reject false trajectories. In this case this trend SVM RESULTS 180 IMAGE QUADRUPLETS WITH LINE THRESHOLD 3.
is readily apparent when going from a triplet to a quadruplet
of images.
Kernel Overall True positive False
accuracy(%) accuracy(%) positive R EFERENCES
accuracy(%) [1] G. H. Stokes, J. B. Evans, and S. M. Larson, “Near-earth asteroid search
linear 99.87 99.41 100 programs,” Asteroids, vol. 3, pp. 45–54, 2002.
chi-squared 99.84 99.28 100 [2] J. B. Evans, F. C. Shelly, and G. H. Stokes, “Detection and discovery of
rbf 100 100 100 near-earth asteroids by the linear program,” Lincoln Laboratory Journal,
vol. 14, no. 2, pp. 199–215, 2003.
TABLE II
[3] NASA. (2015) The NEAT survey. [Online]. Available:
SVM RESULTS 180 IMAGE QUADRUPLETS WITH LINE THRESHOLD 4.
http://neat.jpl.nasa.gov/
[4] ——. (2014) The Catalina Sky Survey. [Online]. Available:
http://www.lpl.arizona.edu/css/
[5] L. Denneau, R. Jedicke, T. Grav, M. Granvik, J. Kubica, A. Milani,
C. Real Imagery P. Vereš, R. Wainscoat, D. Chang, F. Pierfederici et al., “The pan-starrs
moving object processing system,” Pan, vol. 125, no. 926, pp. 357–395,
It is important when developing an image processing 2013.
pipeline on simulated images to verify that the overall system [6] NASA. (2014) The NEOWISE survey. [Online]. Available:
http://neo.jpl.nasa.gov/programs/neowise.html
performs as expected on real imagery. [7] ——. (2015) A Near Earth Asteroid Census. [Online]. Available:
1) NEAT: Near Earth Asteroid Tracking (NEAT) [3] is http://tinyurl.com/bypjz4d
[8] M. Shao, B. Nemati, C. Zhai, S. G. Turyshev, J. Sandhu, G. Hallinan, and
an Earth-based program run by NASA from 1995-2007 to L. K. Harding, “Finding very small near-earth asteroids using synthetic
discover NEOs. Fig. 9 and Fig. 10 show the results at all tracking,” The Astrophysical Journal, vol. 782, no. 1, p. 1, 2014.
stages of the pipeline for one triplet of images of the 2002- [9] T. Yanagisawa, A. Nakajima, K.-i. Kadota, H. Kurosaki, T. Nakamura,
F. Yoshida, B. Dermawan, and Y. Sato, “Automatic detection algorithm
CY46 asteroid obtained from the NEAT system archive. for small moving objects,” Publications of the Astronomical Society of
2) CATALINA: The Catalina Sky Survey (CSS) [20], [21] is Japan, vol. 57, no. 2, pp. 399–408, 2005.
intended to discover NEOs, specifically potentially hazardous [10] P. S. Gural, J. A. Larsen, and A. E. Gleason, “Matched filter processing
for asteroid detection,” The Astronomical Journal, vol. 130, no. 4, p.
asteroids that pose risk to Earth. Fig. 10 shows the final result 1951, 2005.
for one set of Earth-based images from CSS. [11] J. Kubica, J. Masiero, A. W. Moore, R. Jedicke, and A. Connolly,
“Variable kd-tree algorithms for spatial pattern search and discovery,”
D. Implementation Robotics Institute, p. 253, 2005.
[12] J. Kubica, A. Moore, A. Connolly, and R. Jedicke, “A multiple tree
Prototype and test algorithms were coded in MATLAB. algorithm for the efficient association of asteroid observations,” in
The algorithms that show promise were then implemented Proceedings of the eleventh ACM SIGKDD international conference on
Knowledge discovery in data mining. ACM, 2005, pp. 138–146.
in C++ for benchmarking on a MCP750, a good proxy for [13] J. Kubica, L. Denneau, T. Grav, J. Heasley, R. Jedicke, J. Masiero,
the RAD750. The MCP750, with clock speed of 367 MHz, A. Milani, A. Moore, D. Tholen, and R. J. Wainscoat, “Efficient intra-
is completely dedicated to the algorithm and not servicing and inter-night linking of asteroid detections using kd-trees,” Icarus, vol.
189, no. 1, pp. 151–168, 2007.
spacecraft or instrument commands during benchmarking. Run [14] R. O. Duda and P. E. Hart, “Use of the hough transformation
time for Algorithm 1 on a quadruplet is between 1 and 10 to detect lines and curves in pictures,” Commun. ACM,
seconds. Preliminary tests indicate that 16 MB of RAM is used vol. 15, no. 1, pp. 11–15, Jan. 1972. [Online]. Available:
http://doi.acm.org/10.1145/361237.361242
and the current executable is 1.3 MB. Any pre-processing such [15] M. A. Fischler and R. C. Bolles, “Random sample
as step 1 would be implemented on an FPGA. While further consensus: A paradigm for model fitting with applications to
optimization is under way, current results indicate that this image analysis and automated cartography,” Commun. ACM,
vol. 24, no. 6, pp. 381–395, Jun. 1981. [Online]. Available:
algorithm is a viable candidate for a spacecraft processor. http://doi.acm.org/10.1145/358669.358692
[16] D. L. Donoho, X. Huo, I. Jermyn, P. Jones, G. Lerman, O. Levi, and
IV. CONCLUSION F. Natterer, “Beamlets and multiscale image analysis,” in in Multiscale
and Multiresolution Methods. Springer, 2001, pp. 149–196.
We have developed a pipeline for space-based asteroid de- [17] C. H. Lampert, M. Blaschko, and T. Hofmann, “Efficient subwindow
tection. Its performance and run time make it a good candidate search: A branch and bound framework for object localization,” Pattern
Analysis and Machine Intelligence, IEEE Transactions on, vol. 31,
for deployment on a RAD750. We demonstrated that the use no. 12, pp. 2129–2142, Dec 2009.
of quadruplets of image, instead of triplets is determinant [18] P. Viola and W. M. Wells III, “Alignment by maximization of mutual
in detecting faint asteroids. Current run time also supports information,” International journal of computer vision, vol. 24, no. 2,
pp. 137–154, 1997.
a mission concept that aims for image acquisition at a 10 [19] M. F. Skrutskie et al., “The two micron all sky survey (2mass),” The
minute cadence with plenty of time for additional processing. Astronomical Journal, vol. 131, no. 2, pp. 1163–1183, 2006.
Further experiments and refinements of the pipeline will allow [20] S. Larson, J. Brownlee, C. Hergenrother, and T. Spahr, “The catalina
sky survey for neos,” in Bulletin of the American Astronomical Society,
for better understanding the trade-offs of a space-based system vol. 30, 1998, p. 1037.
designed for asteroids that cannot otherwise be detected with [21] A. Drake, S. Djorgovski, A. Mahabal, E. Beshore, S. Larson, M. Gra-
an Earth-based telescope. ham, R. Williams, E. Christensen, M. Catelan, A. Boattini et al., “First
results from the catalina real-time transient survey,” The Astrophysical
Journal, vol. 696, no. 1, p. 870, 2009.
ACKNOWLEDGMENT
This work was supported by an Early Stage Innovations
grant from NASA’s Space Technology Research Grants Pro-
gram NNX14AB04G. The authors would like to thank E.
Christensen for providing the CSS data for this study.
Performance metrics for simulated triplets. Performance metrics for simulated quadruplets.
6 6
Threshold = 80 Threshold = 90
Threshold = 100 Threshold = 100
5 Threshold = 120 5 Threshold = 120
Log10(Number of detections)

Log (Number of detections)


Threshold = 130 Threshold = 130
4 Threshold = 140 4 Threshold = 140

3 3

2 2

10
1 1

0 0
A B C D A B C D
Stages of the pipeline Stages of the pipeline

Fig. 3. (Left) The number of detections at various stages of the pipeline for triplets of images. (Right) The number of detections at various stages of the
pipeline for quadruplets of images. Following are the stages of the pipeline : (A) Initial (B) Pre-Processing+Registration+Thresholding (C) Image Differencing
+ Connected Components + Filter (D) Trajectory Detection.

Fig. 4. True positive vs False positive curve at various threshold values for asteroid detections at the final stage of the pipeline for simulated image sequences
generated with integration time of 90 seconds and aperture size of 0.5 m. The plots showing the detections for the complete set of asteroids whose size ranges
from 30 m to 1000 m. (Left) 4 images per sequence with line threshold 3.(Right) 5 images per sequence with line threshold 4.
Fig. 5. True positive vs False positive curve at various threshold values for asteroid detections at the final stage of the pipeline for simulated image sequences
generated with integration time of 90 seconds and aperture size of 0.5 m. (Left) The curves are stratified based on the asteroid radius.(Right) The curves are
stratified based on the asteroid SNR.

Fig. 6. True positive vs False positive curve at various threshold values for asteroid detections at the final stage of the pipeline for simulated image sequences
generated with integration time of 90 seconds and aperture size of 0.5 m. The curves are stratified based on the asteroid distance. (Left) Asteroid of radius
50 m (Right) Asteroid of radius 30 m.
Fig. 7. (Left) Detection histogram for SN R ≤ 1.3 for the entire simulated image set of 2100 sequences. We detect all asteroids with SN R > 1.3 (Right)
Plot showing how SNR varies depending on asteroid size and distance.

Fig. 8. True positive vs False positive curve at various threshold values for asteroid detections at various stages of the pipeline for simulated image sequences
generated with integration time of 90 seconds and aperture size of 0.5 m.(Left) 4 images per sequence with line threshold 3.(Right) 5 images per sequence
with line threshold 4.
Fig. 9. Image Processing Pipeline results shown on 2002 NEAT data. (row 1): input image triplet (CY46) taken approx. 10 minutes apart. (row 2): Image
Registration and Thresholding. Left: Image-1 registered to Image-2. Middle: Image-2 after thresholding Right: Image-3 registered to Image-2. (row 3): Image
Differencing: Artifacts such as crater-like formations are seen in the difference images above. This is the result of some celestial bodies being over-exposed.)
(row 4): Image Differencing: Filtered centroids shown in each image of the sequence.)
Fig. 10. Results shown on real imagery from NEAT and CSS. Asteroid trajectory detected is shown in green, true location of the asteroid is in red. The
registered images are super-imposed to visualize the trajectory in a single image. (Left) Trajectory Detection for the NEAT CY46 Triplet. (Center) Trajectories
detected on a CSS triplet. (Right) Trajectory detected on a CSS quadruplet.

Potrebbero piacerti anche