Introduction To GeoEvent Processor - Module 4

NOTE: The ArcGIS GeoEvent Processor team updates the product tutorials frequently to reflect the latest version
of the software. Depending on

the product version you are using, there may be inconsistencies between your environment and illustrations or specific steps in the exercises. The
concepts outlined in the exercises, however, should be applicable across product versions.
Introduction to GeoEvent Processor Module 4
GEP 10.2.x / Rev B
Introduction to ArcGIS GeoEvent Processor Module 4

Introduction & Module 4 Overview .............................................................................................................. 3
Objectives for Module 4 ........................................................................................................................... 3
Getting Started.......................................................................................................................................... 3
Working with Processors .............................................................................................................................. 4
No Operation Processor............................................................................................................................ 5
Field Calculator Processor ......................................................................................................................... 6
Mathematic functions supported by Field Calculator ........................................................................ 10
String functions supported by Field Calculator................................................................................... 10
Field Calculator (Regular Expression) Processor ..................................................................................... 10
RegEx Example 1 .............................................................................................................................. 11
RegEx Example 2 .............................................................................................................................. 11
Field Enricher Processors ........................................................................................................................ 12
Configure a Field Enricher (Feature Service) Processor Feature Service ......................................... 16
Configure a Field Enricher (Feature Service) Processor Table ......................................................... 20
Configure a Field Enricher (File) Processor ......................................................................................... 20
Spatial enrichment of GeoEvents ....................................................................................................... 24
Field Reducer Processor.......................................................................................................................... 26
Field Mapper Processor .......................................................................................................................... 28
Congratulations! ......................................................................................................................................... 36
March 2014
Page 2 of 37
Introduction & Module 4 Overview

The Introduction to GeoEvent Processor tutorial the first in a series of tutorials developed to introduce
the many different capabilities of ArcGIS GeoEvent Processor for Server has been divided into five
separate modules. The scope and material covered in each module is described in Module 1.
The modules in this tutorial were designed to complement one another by presenting different functional
areas of the product and exploring the functionality through exercises. While exercises in later modules do
not generally require you to complete the exercises from the previous modules, you are encouraged to
work through the Introduction to GeoEvent Processor tutorial beginning with Module 1 and continuing
through Module 5. This is especially true for exercises in Module 5 in which readers can expect the
narrative and illustrations in an exercise to replace some of the step-by-step instruction presented in
earlier modules.
At the start of each module, and occasionally within a module, a GeoEvent Processor configuration will be
referenced which can be imported to update and/or create configurable items such as inputs, outputs,
GeoEvent Definitions, and/or GeoEvent Services. Carefully review the notes which identify what the
GeoEvent Processor configuration contains, as it may update or reset items you created and modified as
part of a previous exercise or product exploration you have completed on your own.
Module 4 presents the concept of event processing with exercises that illustrate how to configure and
integrate different types of processor into GeoEvent Services. This module covers some of the basic
processors used for field calculation and event enrichment as well as processors used to simplify events
by removing unwanted fields and mapping events containing different information to bridge differences
between event input and output.
Objectives for Module 4

Once you have completed the exercises in this tutorial you should be able to:
Describe the different types of processors included with GeoEvent Processor.
Explain how processors are configured and incorporated into GeoEvent Services.
Demonstrate basic processing capabilities such as field calculation and event enrichment.
Leverage processors to map GeoEvents from one GeoEvent Definition to another.
The exercises in Module 4 will introduce you to some of the basic processors which provide the
capabilities outlined above. In Module 5, you will work with some of the more advanced processors
including Incident Detector, Track Gap Detector, and GeoTagging.
Getting Started
If you are starting with this module, a GeoEvent Processor configuration file is included to help you
quickly get started with the exercises in this module. Importing this product configuration will create the
items listed below if items with the same names already exist, importing the product configuration will
update them and overwrite any changes you may have made in a previous module.
March 2014
Page 3 of 37
Module 4 GeoEvent Processor Quick Start Configuration contents:
A tcp-text-in input and a tcp-text-out output.
A Flights GeoEvent Service which contains a No Operation Processor.
A Flights-TcpTextIn GeoEvent Definition used to interpret simulated flight event data.
A registered instance of ArcGIS for Server named LOCALHOST_AGS.
If you worked through the previous modules, you will notice the Flights GeoEvent Service in the
configuration for this module has been modified to include a No Operation Processor. This processor will
be described more in the first exercise below. The registered data store imported with this configuration
assumes ArcGIS for Server is installed on the localhost and is reachable via the URL
http://localhost:6080/arcgis/.
Follow the steps below to import the GeoEvent Processor product configuration.
1. Open GeoEvent Processor Manager at https://localhost:6143/geoevent/manager/.
2. In GeoEvent Processor Manager, navigate to Site > GeoEvent Processor > Configuration Store.
3. Click Import Configuration and browse to Module4_QuickStart.xml file located in the
\configurations folder included with this tutorial.
4. Select the Module4_QuickStart.xml file and click Import to import the configuration.
The tcp-text-in input and tcp-text-out output were imported from the QuickStart in a stopped state. You
may find that stopping your input, output, and GeoEvent Service components before exporting a
GeoEvent Processor configuration is a good practice. Upon import, a started component has the
potential to begin processing data immediately and could possibly conflict with other started
components. You can start and stop components in GeoEvent Processor Manager.
5. Navigate to Services > Inputs and click
6. Navigate to Services > Outputs and click
to start the tcp-text-in input.

to start the tcp-text-out output.
The Flights GeoEvent Service should already be running. There is little a GeoEvent Service can do if its
inputs and outputs are not running.
Working with Processors

The exercises in the various modules of this tutorial have introduced you to different components of
GeoEvent Service. In earlier modules you used the GeoEvent Processor Manager to create inputs and
outputs and then incorporated these service elements to design and publish a GeoEvent Service. The
previous module introduced filters as another type of service element you can use when designing
GeoEvent Services.
Processors are another type of GeoEvent Service element. Processors includes several different types of
processors which you configure to build event processing capability into GeoEvent Services. Processors
are the components in a GeoEvent Service that enhance, restructure, and calculate new data values
March 2014
Page 4 of 37
associated with GeoEvents before they are routed to an output. Eight different types of processors are
included in the product:
Field Calculator
Field Reducer
GeoTagger
Field Enricher
Incident Detector
No Operation
Field Mapper
Track Gap Detector
The Incident Detector, Track Gap Detector, and GeoTagger processors will be covered in Module 5.
Exercises in this module will focus on:
Field Calculator processors used to create derivative data which can be either appended to an
event or used to overwrite data contained in an event. This processor has two types, one of
which allows Regular Expressions to be applied to event data.
Field Enricher processors used to retrieve data from either a feature service or a system text
file and append to enrich events. Processors which append data to events change an events
schema requiring GeoEvent Processor to create a new GeoEvent Definition.
Field Mapper processors used to map event data from one schema to another event schema.
This type of processor relies heavily on the GeoEvent Definitions which define the data types of
different fields within a GeoEvent.
Field Reducer processors used to reduce or remove unwanted fields from a GeoEvent.
No Operation processors a processor with no configurable properties which performs no

manipulation or changes to events it receives. This processor is primarily used as a place holder
when designing more complex GeoEvent Services.
No Operation Processor
The No Operation processor is the simplest processor to configure as it has no properties other than the
service elements name.
1. In GeoEvent Processor Manager, navigate to Services > GeoEvent Services and click to begin editing
the Flights GeoEvent Service.
2. Double-click the No Operation processor to open properties.
3. Click the Processor drop-down and review the available types of processors.
March 2014
Page 5 of 37
The type of processor used in a GeoEvent Service can be changed by editing the processors properties.
Processors do not generally alter event data unless they are specifically configured to do so. Unlike
filters which may discard events, processors generally perform the processing they are set to perform
and then pass the GeoEvent along to the next GeoEvent Service element.1
4. Leave GeoEvent Processor Manager open, editing the Flights GeoEvent Service for the next
exercise.
Field Calculator Processor

The Field Calculator Processor is probably the simplest processor to configure. However, the potential
functional scope of this processor is remarkable, as it provides access to a large number of mathematic
and string manipulation functions supported by the Java programming language.
The Field Calculator Processor allows the user to enter an expression which the processor will evaluate.
The calculation performed produces a result which the processor will either append to the event as a
new field or write into the original event field (overwriting the data originally received with the event).
This processor allows event data to be retrieved from fields and used in the expression being evaluated.
Follow the steps below to reconfigure the Flights GeoEvent Service to incorporate a Field Calculator
Processor.
1. In GeoEvent Processor Manager, navigate to Services > GeoEvent Services.
2. Double-click the Flights GeoEvent Service to start editing.
3. Double-click the No Operation processor and reconfigure it to be a Field Calculator Processor.
4. Type Field Calculator for the Name.
Some exceptions are the Track Gap Detector and Incident Detector processors which actually produce a new class of events
based on detections made by the processor. These will be discussed in Module 5.
March 2014
Page 6 of 37
5. Type Altitude / 3.28084 in the Expression property.

This is a mathematical conversion that will display altitude in meters instead of feet.
6. Click the Result Destination drop-down and choose New Field.
7. Type AltitudeInMeters for the New Field Name.
8. Click the New Field Type drop-down and choose Long.
9. Type Flights-TcpTextIn-FieldCalc in the Resulting GeoEvent Definition Name.
This incorporates the name of the GeoEvent Definition providing the event, into the name of the
GeoEvent Definition the processor will create. This will be discussed more below.
The Field Calculator Processor configured above, retrieves attribute values from the named field,
Altitude, and divides the value by the specified constant to convert from feet to meters. It then appends
the computed result as a long integer to the event in a new field named AltitudeInMeters.
Because the processor is modifying the events schema by creating a new event field, GeoEvent Processor
will need to create a new GeoEvent Definition. The name of the new GeoEvent Definition will be FlightsTcpTextIn-FieldCalc. Recognizing that the GeoEvent has been modified, and now has a new GeoEvent
Definition, will be important if you plan on sending the event data to an output which expects a certain
schema such as an Update a Feature output used in Module 2.
10. Click OK to save the processor.
March 2014
Page 7 of 37
11. Click Publish to publish the reconfigured Flights GeoEvent Service.

12. Open the GeoEvent Simulator and configure as follows:
Click Load File and browse to the FeatureFlights.csv file in the \simulations folder included
with this tutorial.
Leave the Time Field # property set to field 1 and click Load.
Check the Set value to Current Time checkbox to simulate real-time data.
Uncheck the Continuous Loop checkbox.
Set the rate to send 9 messages every 1000 milliseconds.
Click Click to Connect to establish your server connection.
Refer to the illustration below and the exercises in Module 1 for help configuring the GeoEvent
Simulator.
13. Open the TCP Console application introduced in Module 1.
14. In the GeoEvent Simulator, click Step
once to simulate a set of flight data events.
March 2014
Page 8 of 37
The event data should display in your TCP Console application. In particular, notice the name of the
GeoEvent Definition you assigned is being reported and that the new altitude fields have been
appended to the event.
15. In GeoEvent Processor Manager, navigate to Site > GeoEvent Processor > GeoEvent Definitions.
16. Notice a new GeoEvent Definition named Flights-TcpTextIn-FieldCalc was created.
This is a new managed GeoEvent Definition created by GeoEvent Processor.
GeoEvent Processor will create managed GeoEvent Definitions on-the-fly when a processor modifies an
events schema by adding or removing fields. Managed GeoEvent Definitions will be deleted if/when the
processor is reconfigured and the GeoEvent Service is republished.
If you were to edit the new GeoEvent Definition you would see the owner is listed as Flights:Field
Calculator. Each refers to the GeoEvent Service and the processor within the GeoEvent Service which is
relying on the GeoEvent Definition. You would also see the field names, for the new GeoEvent Definition
fields, have been adopted from the Flights-TcpTextIn. This was the GeoEvent Definition from the
GeoEvents which were originally sent to the Field Calculator processor.
You should not edit or otherwise change managed GeoEvent Definitions. Managed GeoEvent Definitions
are created and maintained by the system to support processing being performed by GeoEvent Services.
March 2014
Page 9 of 37
Mathematic functions supported by Field Calculator

The Field Calculator Processor supports a subset of the mathematic functions defined by the Java Math
package.2 You can incorporate functions such as round( ), toDegrees( ), or toRadians( ) into a Field
Calculator expression to invoke the function on values retrieved from an event.
For example, the expression above could be modified from Altitude / 3.28084 to round(Altitude /
3.28084) and the Field Calculator Processor configured to output its calculated value as a double rather
than as a long integer. This would be an option if you wanted to retain the floating point data type but
round the value to reduce the number of digits reported when the events are sent.
String functions supported by Field Calculator
The Field Calculator Processor also supports a subset of the string manipulation functions from the Java
String package.3 You can incorporate functions such as toLowerCase( ), toUpperCase( ), or valueOf( ) into
a Field Calculator expression to invoke the function on values retrieved from an event.
For example, the Field Calculator Processor will attempt to automatically convert data it retrieves from an
event to an appropriate type, in order to evaluate the processors expression and send the calculated
value as the data type specified. In the exercise above, the calculated value for the flights altitude, in
meters, was first computed as a double and then rounded to the nearest long in order to output the value
as a long integer as specified in the processors configuration.
In the next section you will explore a different type of processor a Field Calculator (Regular Expression)
Processor which allows a regular expression syntax to be included in the expression.
Field Calculator (Regular Expression) Processor

This exercise will introduce the Field Calculator (Regular Expression) Processor which allows regular
expressions, sometimes referred to as RegEx, to manipulate event data. There are no exercise steps in
this section. Instead, illustrations are provided to support the discussion.
A regular expression is a special text string for describing a search pattern. You are probably familiar
with wildcard notations such as *.txt to find all text files in a file manager. An equivalent expression
.*\.txt$ could be used to search a text listing of file names to identify the same files.
Regular expressions can be very powerful and very complex. For example, you could use the regular
expression \b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b to search a block of text for an email
address. Even if you are not a programmer, you can use regular expressions in many situations to find
and isolate information in a data stream. A simple example gr[ae]y would find both grey as well as
gray (regardless of how the word was spelled in the event data you receive). A regular expression is
essentially a search pattern used for searching text for data strings.
Refer to documentation on the Field Calculator beneath the Processors topic on the ArcGIS for Professionals site for a list of functions
supported by GeoEvent Processor.
Support for String functions is available in ArcGIS 10.2.2 GeoEvent Processor for Server.
March 2014
Page 10 of 37
A Field Calculator (Regular Expression) Processor applies a search pattern to a value usually a value
retrieved from an existing event field in order to produce a result which the processor will either
append to the event as a new field or write into the original event field (overwriting the data originally
received with the event).
RegEx Example 1
Suppose you were receiving events with an event field named Description. The data in the events was
sometimes longer than a 32 character limit in which some component of your system could tolerate.
You could use a Field Calculator (Regular Expression) Processor to grab the first 32 characters from the
Description attribute value in order to trim the value to a length acceptable by your system.
In the illustration above, the Pattern property uses a mechanism known as character repetition in RegEx.
The ^ anchors to the beginning of the string, the . is a wildcard matching any character, and the
{0,32} indicates that substring, with a minimum length of zero and a maximum length of 32, will be
returned as the matched pattern.
Notice the Field Name parameter encloses the expression in curl brackets. This is how this type of
processor expects named event fields to be expressed. The value from the named field will be
substituted in the expression in place of ${FIELD-NAME}.
RegEx Example 2
Suppose FeatureFlights.csv was used to simulate events and you wanted to split each FlightNumber into
a pair of values with the first containing the airline code and the second containing the numeric flight
identifier. You could accomplish this with a pair of Field Calculator (Regular Expression) Processors. In
the illustration below, a regular expression ^[A-Z]+ is used to match zero-or-more characters in the
range A-Z. The pattern is anchored to the beginning of the ${FlightNumber} field with a ^.
March 2014
Page 11 of 37
The second processor uses another regular expression [0-9]+$ to match zero-or-more digits at the end
of the ${FlightNumber} field.
Both processors illustrated above create new fields for the data extracted from the specified target
expression and each processor creates its own GeoEvent Definition for the event schema modified when
new data is appended to the event.
Field Enricher Processors

In this exercise you will modify the Flights GeoEvent Service and incorporate a different type of
processor known as a Field Enricher (Feature Service) Processor.
Note: You will need to have published the Pilots.mpk map package as a feature service prior to starting
this exercise. Recall the map packages needed to complete the exercises are found in the \data
folder provided with this tutorial.
Refer to the Publish a Feature Service to ArcGIS Server section found in Module 2 for help
publishing the necessary feature service.
March 2014
Page 12 of 37
A Field Enricher Processor can be used to retrieve data from an external table and add that data to the
event being processed, essentially enriching the event with additional data. This is accomplished by
joining the two data tables temporarily on-the-fly.
There are two types of Field Enricher Processors. One handles enrichment from a feature service, the
other from a system file. For this exercise you will be enriching events using data from the Pilots feature
service.
1. From the Windows Start menu, navigate to ArcGIS 10.2.x for Server and click Services Directory.
2. In the ArcGIS REST Services Directory, click IntroToGEP.
March 2014
Page 13 of 37
3. Click IntroToGEP/Pilots (FeatureServer) to open the web page for the Pilots feature service.
4. Notice the Pilots feature service has both a layer and a table:
pilotsFC (0)
(The layer at index 0)
pilotsTB (1)
(The layer at index 1)
In this exercise, you will use both the feature services layer and table to enrich GeoEvents.
5. Click pilotsFC (0) to review the properties.
Notice Geometry Type: esriGeometryPoint, this indicates the feature layer contains point features.
6. Scroll to review the Fields in the feature layer.

This section identifies the attribute fields (PilotName, PilotPhone, PilotGroup, PilotActive, etc) available
in the feature layer. The field names, alias names, and data types are specified for each attribute field.
March 2014
Page 14 of 37
7. Scroll to the bottom of the page and click Query.
8. On the Query: pilotsFC (ID: 0) page, configure your query with the following parameters:
Where:
1=1
(no spaces)
Out Fields:
(wildcard indicating all fields should be shown)
Format:
JSON
9. Scroll to the bottom of the page and click Query (GET).

The page should display a JSON representation of the features queried from the feature service. Scroll to
locate the section which identifies the features currently in the feature layer.
March 2014
Page 15 of 37
The JSON code should identify a total of 9 features only one has been expanded in the JSON illustrated
above.
If you do not have a JSON viewer plug-in installed in your browser, you might find it helpful to use your
browsers text search to locate the keyword features or geometry. Each feature in the features list
should identify the attribute values associated with that feature.
10. Refer to the illustration below of the FeatureFlights.csv file (opened in a text editor) that is
included in the \simulations folder included with this tutorial.
Notice the id of each Pilot feature in the JSON illustration above corresponds to the unique identifier of
one of the nine flight events in the FeatureFlights file.
Configure a Field Enricher (Feature Service) Processor Feature Service

You will now reconfigure the Flights GeoEvent Service and incorporate a Field Enricher (Feature Service)
Processor. The GeoEvent Service will enrich GeoEvents it receives with data from the feature layer.
2. Click Flights to begin editing the GeoEvent Service.
3. Double-click the existing Processor to open the properties.
4. Click the Processor drop-down and choose Field Enricher (Feature Service).
March 2014
Page 16 of 37
Optionally, you could double-click Processor in the menu to add a new processor to the GeoEvent
Service and then delete the existing processor.
When modifying a processor to a different type, be sure to update the name of the processor to reflect
the new type of processor being created. If you do not, it could be confusing if you had a processor
named Field Calculator, but it is actually a Field Enricher (Feature Service) Processor.
5. Configure the properties using the illustration below as a guide.
This processor above will contact the specified ArcGIS Server and look for an IntroToGEP server folder,
which contains a feature service named Pilots, with a layer named pilotsFC.
The processor will then join the feature layers table with the GeoEvents in order to retrieve
enrichment data associated with the current GeoEvent. The key fields for the join are the id field from
the feature layers table and the GeoEvent field tagged TRACK_ID. The attribute field PilotName from
the feature layer is the field from which enrichment data will be retrieved a comma separated list of
enrichment field names is allowed in the Field Enricher (Feature Service) Processor.
7. Click Publish to publish the Flights GeoEvent Service.
An illustration of the GeoEvent Service with the properties of the Field Enricher processor is shown on
below.
March 2014
Page 17 of 37
The tcp-text-in input and tcp-text-out outpu illustrated above were configured with the default ports
used in previous exercises: port 5565 for input and 5570 for output. Remember, no two GeoEvent
Service components can connect to the same port at the same time, so you may want to review the
current state of your configuration using GeoEvent Processor Managers Monitor page.
8. In GeoEvent Processor Manager, navigate to Services > Monitor and ensure the input, output, and
GeoEvent Service are STARTED.
If you are continuing from a previous exercise, the event counts for your input and output might differ
from the illustration above, the reason being, Reset Statistics was clicked to clear the statistics.
As a side note, modifying and republishing a GeoEvent Service resets its event count on the Monitor
page, however, the input and output event counts will not affected.
9. Open the GeoEvent Simulator and configure as follows:
Click Load File and browse to the FeatureFlights.csv file in the \simulations folder included
with this tutorial.
March 2014
Page 18 of 37
Leave the Time Field # property set to field 1 and click Load.
Check the Set value to Current Time checkbox to simulate real-time data.
Uncheck the Continuous Loop checkbox.
Set the rate to send 9 messages every 1000 milliseconds.
Click Click to Connect to establish your server connection.
This is the same configuration used in the Field Calculator Processor section above. You may want to
clear the previously sent events by clicking Clear Sent Events
and return the GeoEvent Simulator to
the beginning of the FeatureFlights simulation file by clicking
so the results you see match the
illustrations in the following exercises.
10. Open the TCP Console application.
11. In the GeoEvent Simulator, click Step
once to send a set of flight event data.
Notice each event displayed in the TCP Console application has been enriched with the pilots name.
Notice also the name of the GeoEvent Definition used to interpret the enriched events is displayed in
the TCP Console application. Processors which append data to events change an events schema,
requiring GeoEvent Processor to create a new managed GeoEvent Definition. The name of this GeoEvent
Definition was defined in the Field Enricher processor you created earlier.
If you were to edit the Flights-TcpTextIn-Enrichment GeoEvent Definition you would see PilotName listed
at the bottom of the event fields list. You would also see the name of the GeoEvent Service and
processor which rely on the GeoEvent Definition reflected as the Owner. Remember, you should not edit
managed GeoEvent Definitions; they are created and maintained to support processing performed by
specific GeoEvent Services.
When configuring the Field Enricher Processor above, you may have noticed the Field Tags property was
not specified. If you have one or more GeoEvent tags and want to apply those tags to specific fields that
are added to a GeoEvent through enrichment, you can specify those tags as a comma separated list in
the same order as you specified the enrichment fields.
March 2014
Page 19 of 37
Configure a Field Enricher (Feature Service) Processor Table

In the steps above, you used a Field Enricher (Feature Service) Processor to retrieve data from a feature
layer. This same processor can be used to retrieve data from a non-spatial table made available through
a feature service. The processor configuration looks very similar in fact the only change is which layer
you use from the feature service and the enrichment fields you want to use.
2. Click Flights to begin editing the GeoEvent Service.
3. Double-click the existing Field Enricher processor to access the properties.
4. Click the Layer drop-down and choose PilotsTB.
5. Type PilotName, PilotPhone in the Enrichment Fields property.
Notice the comma between the two values, you are specifying a comma separated list of values.
Referring to the steps 9 11 in the previous section, Configure a Field Enricher (Feature Service)
Processor Feature Service, use the GeoEvent Simulator to send another set of 9 flight events from the
FeatureFlights simulation file. In the TCP Console application, notice the flight events are now being
enriched with the pilots name and phone number.
If you were to edit the Flights-TcpTextIn-Enrichment GeoEvent Definition you would see both PilotName
and PilotPhone listed at the bottom of the event fields list. The GeoEvent Definition is being managed
for you. When the Field Enricher processor was edited and the GeoEvent Service republished, the old
GeoEvent Definition was deleted and a new one created; they have the same name because the
processor specifies the same GeoEvent Definition name be used.
Configure a Field Enricher (File) Processor
In the previous exercises, you saw how a Field Enricher (Feature Service) Processor can be used to enrich
incoming events by appending fields from a feature service. GeoEvent Processor provides another type
of processor which supports event enrichment using data from a file on a local system folder.
March 2014
Page 20 of 37
Using a file located in a system folder provides a lightweight way to enrich GeoEvents with data values
that are relatively static. In this exercise, you will enrich GeoEvents with similar information (pilot name,
phone number, etc.) as in the previous exercise this time retrieving the data from a file. The folder
containing the file must be registered with GeoEvent Processor as a data store.
1. Create the C:\GeoEvent\EnrichmentData directory if it does not already exist on your local
system.
2. In GeoEvent Processor Manager, navigate to Site > GeoEvent Processor > Data Stores.
3. Click Register Folder and specify the Name and Path to the local system folder as illustrated below.
4. Click Register to register the local system folder with GeoEvent Processor.
You should see a message indicating your folder was successfully registered.
You should see the folder listed under Registered Folders.
March 2014
Page 21 of 37
The steps above assume the system folder C:\GeoEvent\EnrichmentData exists and that the OS-level
user responsible for running the ArcGIS for Server and ArcGIS GeoEvent Processor for Server services,
and the user-id you have logged-in as, all have permissions sufficient to create and update system files
in the folder being registered. You can specify any system folder in which you are able to create files, but
the path you specify should be a fully-rooted path to an existing folder on your local machine UNC
pathnames are not supported.
It is recommended you keep the files being utilized for GeoEvent enrichment separate from other files
such as those that provide simulation data or those that are being written to by GeoEvent Processor
outputs.
You can click Validate All at any point to validate your registered data stores. Successful validation is
indicated by the green check mark next to a registered folder or ArcGIS Server connection.
5. Browse to the FlightPilotInformation.csv file in the \data\WorkingWithProcessors folder
included with this tutorial.
6. Copy and paste the FlightPilotInformation.csv file to the \EnrichmentData folder you just
created.
7. Review the illustrations below of the FlightPilotInformation.csv file (opened in a text editor).
The data in this file is in a comma separated format. The first line identifies the datas field names and
the second line specifies the data types for each field (String, Integer, Boolean, etc.)
Notice the id field for each pilot in the data maps to one of the several unique flight numbers in the
FeatureFlights event simulation file you have been using to simulate flight events in previous exercises.
Using the steps below, similar to those in the previous exercise, you will configure a Field Enricher (File)
Processor to enrich the GeoEvents from a file.
9. Click the Flights GeoEvent Service to begin editing.
10. Double-click the existing Field Enricher processor to open its properties.
11. Click the Processor drop-down and choose Field Enricher (File).
March 2014
Page 22 of 37
12. Change Name to Field Enricher File to reflect the type of processor being created.
13. Click the Registered Folder drop-down and choose Enrichment-Data-Folder. This is the registered
folder that contains the text file you will use for enrichment.
14. Type FlightPilotInformation.csv for the File Name to specify the enrichment data file.
15. Type id in the File Join Field.
Remember, the field names are hard-coded in the first line of the enrichment data file. The data
types for each field are on the second line of the file.
16. Type PilotName, PilotPhone in the Enrichment Fields property.
17. Type TRACK_ID in the GeoEvent Join Field property.
18. Type Flights-TcpTextIn-Enrichment for the Resulting GeoEvent Definition Name property. GeoEvent
Processor will use this name for the managed GeoEvent Definition created from the enriched
events.

20. Click Publish to publish the Flights GeoEvent Service
As you can see, a Field Enricher (File) Processor is very similar to a Field Enricher (Feature Service)
Processor. Both processors join an external table with the event being processed in order to retrieve
enrichment data associated with the current GeoEvent. The key fields for the join are the id field from
the system file and the event field tagged TRACK_ID. The appropriate PilotName and PilotPhone
attributes will be retrieved from the data file and appended to the event being processed.
March 2014
Page 23 of 37
Referring to the steps 9 11 in the Configure a Field Enricher (Feature Service) Processor Feature
Service section above, use the GeoEvent Simulator to send another set of 9 flight events from the
FeatureFlights simulation file. In the TCP Console application, notice the flight events are now being
enriched with the pilots name and phone number.
To confirm the enrichment is truly coming from the text file and not the feature service used in the
previous exercise, go ahead and move, rename, or delete the text file FlightPilotInformation.csv from
the system folder you registered as your data store.
Send another 9 flight events using the GeoEvent Simulator. Notice that when the enrichment file is
missing, the Field Enricher (File) Processor passes events it receives through without performing the
enrichment. If the text files values are updated, those updated values will be retrieved without needing
to stop and restart the GeoEvent Service.
Spatial enrichment of GeoEvents
In the exercises you just completed above, you worked with the Field Enricher (File) Processor as well as
the Field Enricher (Feature Service) Processor. These exercises focused on attribute enrichment of the
GeoEvents. In this exercise you will focus on enriching GeoEvents with spatial geometry. The only
requirement to accomplish this is that the geometry be represented as a JSON string value in the table
from which the enrichment data is being retrieved.
Representing geometry, especially polyline and polygon geometry, as a JSON string in order to spatially
enrich GeoEvents can be complicated. The JSON can be hard to read for those who do not have prior
experience working with JSON. For that reason, the discussion below introduces this concept without
providing specific step-by-step exercises. References to files included with your exercise materials are
intended to allow you to experiment with spatial enrichment if you choose to do so.
Appendix A includes a detailed discussion on how to construct the JSON string representations of
different geometries and how to use the GeoEvent Simulator to send the JSON strings to a GeoEvent
Service to simulate the receipt of different geometries.
March 2014
Page 24 of 37
Browse to the Weather.csv in the \simulations folder included with this tutorial. Open the file in a
text editor and review the simulation data.
Notice the simulation data contains a date and a track identifier, but no other data. If we had an
enrichment file with these same track identifiers, we could use a Field Enricher (File) Processor to enrich
simulated GeoEvents with geometries and information a web map could use to symbolize the geometry.
Browse to the WeatherFrontGeometry.csv enrichment file in the \data\WorkingWithProcessors
folder included with this tutorial. Open the file in a text editor and review the data.
Notice that, in addition to the track identifier FrontId, the data includes a short integer code we can use
to help symbolize a polyline geometry that represents a cold or warm front and a text string containing a
JSON coded representation of the polyline.
Given these two resources, configuring a GeoEvent Service with a Field Enricher (File) Processor is fairly
straight forward.
The Weather-TcpTextIn input in the illustration above is a variant of the Receive text from a TCP Socket
input you have been working with throughout this tutorial. Since the Weather.csv simulation data does
March 2014
Page 25 of 37
not contain a GeoEvent Definition, the input will need to be configured to create a GeoEvent Definition,
or even better (since the simulation data is so simple) use a GeoEvent Definition created specifically to
match the simulation data content.
A GeoEvent Processor configuration, Module4_WeatherEnrichment.xml, is included with this tutorial
which defines the GeoEvent Services and GeoEvent Definitions illustrated above.
Note: Importing this configuration will create the components above if they do not already exist. If
components with the same names already exist, importing the configuration will update them
and overwrite changes you may have made.
Module4_WeatherEnrichment.xml contents:
A Weather-TcpTextIn input and a tcp-text-out output.
A Weather GeoEvent Service which incorporates a Field Enricher Processor.
A Weather-TcpTextIn GeoEvent Definition used to interpret simulated weather event data.
A Weather-GeometryEnrichment managed GeoEvent Definition for the Field Enricher Processor.
An Enrichment-Data-Folder registered data store into which the WeatherFrontGeometry.csv

enrichment file should be copied before attempting to simulate and enrich event data.
Refer to the Get Started section above for information on importing a GeoEvent Processor configuration
file. Remember to stop any GeoEvent Service components with potential conflicts before starting any
components you import from the configuration file.
Field Reducer Processor

In the Field Calculator Processor section above, exercises demonstrated how values derived from
received event data could be added as new values to an event. In that exercise, the field FlightNumber
was split into its alphabetic prefix and numeric suffix with the result appended as two new event fields.
Suppose the events original data was no longer needed and you only wanted to keep the derived
values. To accomplish this, you would use a Field Reducer Processor.
A Field Reducer Processor allows unwanted fields to be removed from an event. Event simplification and
removal of redundant or obsolete values are just a couple of reasons why you might want to remove
fields from an event. The discussion below will focus on how to configure a Field Reducer Processor and
incorporate it into a GeoEvent Service.
Consider the GeoEvent Service illustrated below. A pair of Field Reducer (Regular Expression) Processors
are used to compute new fields for the AirlineCode and FlightId.
March 2014
Page 26 of 37
Suppose you wanted to remove the FlightNumber field once the values for AirlineCode and FlightId had
been added to the event. This probably is not a good idea since FlightNumber is serving as the TRACK_ID,
but for the purposes of this discussion we are only looking for a candidate field to remove.
Using a Field Reducer Processor you can easily remove fields from the RegEx-GetFlightId GeoEvent
Definition.
The Field Reducer illustrated above, examines event data it receives, identifies a field or fields specified in
a comma separated list of field names, and removes the identified field(s) from the event.
Removing fields from an event modifies the events schema, requiring GeoEvent Processor to create a
new managed GeoEvent Definition. The processor specifies the name of the new GeoEvent Definition
(which will be deleted if/when the Field Reducer processor is reconfigured and the GeoEvent Service is
republished).
March 2014
Page 27 of 37
Oftentimes, you may encounter event streams published by a provider that contains data you are not
interested in sending to your user community and stakeholder. Incorporating a Field Reducer Processor
into a GeoEvent Service is an easy way of eliminating that unnecessary event data before sending it to a
feature service.
Next, you will examine the Field Mapper Processor, which allows you to manage or normalize the fields in
a GeoEvent.
Field Mapper Processor

The Making Features Come Alive section in Module 2 introduced a dilemma frequently encountered by
users the schema of event data being received does not match the schema of a feature service layer.
Perhaps the events, adapted from JSON received over a GeoEvent Processor REST endpoint, include data
in hierarchical groups or lists a data model not supported by Esri feature services.
In the illustration below, notice the structure of the incoming sensor report includes data in separate
groups. Atmospheric data such as temperature and humidity is reported in a group named weather
while data on different gases is reported in a group named gases. Notice also that the sensors location
(latitude, longitude, and altitude) is reported in a third group.
The comma separated data used up to this point is incapable of modeling cardinal lists and groups. You
are more likely to encounter these when working with JSON or RSS data. The exercise in this section will
therefore introduce a new type of input, Watch a folder for new .json files, which allows a text file with
JSON formatted content to be copied into a system folder for GeoEvent Processor to retrieve. The input
will adapt the JSON to create new events which a GeoEvent Service can then filter and process. The
exercise will focus on how a Field Mapper Processor can be configured to both reduce and map event data
into a structure compatible with a feature service.
March 2014
Page 28 of 37
A GeoEvent Processor configuration file, Module4_FieldMapper.xml, is included with this tutorial to help
you quickly get started with the exercise below.
Note: Importing this configuration will create the components below if they do not already exist. If
components with the same names already exist, importing the configuration will update them
and overwrite changes you may have made.
Module4_FieldMapper.xml contents:
Two registered system folder data stores, GeoEvent_Input and GeoEvent_Output.
A Watch-Folder-for-JSON-Files input and a Write-to-JSON-File output.
A Sensor-TcpTextIn GeoEvent Definition used to interpret JSON data received by the input.
A Sensor-TcpTextOut GeoEvent Definition used to deconstruct GeoEvent data for the output.
A SensorMonitor GeoEvent Service that contains the input, output, and a No Operation
Processor.
The registered system folder data stores imported with this configuration assume that the following
folders exist on your local system named C:\GeoEvent\input and C:\GeoEvent\output.
Refer to the Get Started section above for information on importing a GeoEvent Processor configuration
file. Remember to stop any GeoEvent Service components with potential conflicts before starting any
components you import from the configuration file.
1. In GeoEvent Processor Manager, navigate to Services > Monitor and notice the Watch-Folder-forJSON-Files input and Write-to-JSON-File output are currently STOPPED.
You may find that saving your GeoEvent Processor configuration with components (inputs, outputs, and
GeoEvent Services) in the stopped state to be a best practice. A started component can potentially begin
processing data or conflict with other running components immediately after import. The
SensorMonitor GeoEvent Service imports in a started state, since there is little a GeoEvent Service can
do if its inputs and outputs are not running.
2. Click the Watch-Folder-for-JSON-Files input to begin editing.
March 2014
Page 29 of 37
In the illustration above, notice the following:
The input is watching the GeoEvent_Input registered data store (e.g. the system folder
C:\GeoEvent\input) for files whose name matches the regular expression pattern: .*\.json
The input has been configured to not create a GeoEvent Definition based on data it discovers.
It instead expects to use the Sensor-TcpTextIn GeoEvent Definition imported for this exercise.
The input specifies that a geometry should be created for each event received by retrieving the
longitude, latitude, and altitude from a group named location.
3. Click Cancel to close the properties dialog without making any changes.
4. In a Windows File Explorer, browse to C:\GeoEvent\input.
5. Move any *.json files you may have to a sub-folder or another system folder where the input
watching C:\GeoEvent\input will not find them.
6. In GeoEvent Processor Manager, navigate to Services > Inputs.
7. Click
to start the Watch-Folder-for-JSON-Files input.
The input will now read the content of any new *.json files dropped into the system folder being
monitored. If the Sensor-TcpTextIn GeoEvent Definition can interpret the data content, the input will
adapt the JSON to create new GeoEvents for the SensorMonitor GeoEvent Service to process.
March 2014
Page 30 of 37
8. In GeoEvent Processor Manager, navigate to Services > Outputs and notice the Write-to-JSON-File
output is currently STOPPED.
9. Click the Write-to-JSON-File output to begin editing.
In the illustration above, notice the following:
Files with a prefix SensorData and a json file extension will be written to the GeoEvent_Output
registered data store (e.g. the system folder C:\GeoEvent\output).
March 2014
Page 31 of 37
Files older than 4320 minutes (3 days) will be deleted. The output will check for old files as new
event data is written, rolling the current output file to create a new file daily.
The content of the output file will be formatted using square brackets to designate a comma
separated list of events. (The lists closing square bracket will be added to the file upon file
rollover or when the output is stopped and the lock on the file is released).
10. Click Cancel to close the dialog without making any changes.
11. Open Windows File Explorer, browse to C:\GeoEvent\output.
12. Move any *.json files you may have to a sub-folder or another system folder where you will not
confuse them with files written by the output illustrated above.
13. In GeoEvent Processor Manager, navigate to Services > Outputs.
14. Click
to start the Write-to-JSON-File output.
Notice a new file, SensorData2014_02_18.json, was created in the C:\GeoEvent\output system

folder. The date referenced in the file name should reflect your systems current date.
The output will now use the GeoEvent Definition to deconstruct events and write the event data out as
JSON to the target system file.
In this exercise you wrote a JSON representation of event data to a file. Next, you will reconfigure the
SensorMonitor GeoEvent Service to incorporate a Field Mapper Processor, which will alter the schema of
received events to match a desired output schema.
15. In GeoEvent Processor Manager, navigate to Site > GeoEvent Processor > GeoEvent Definitions.
16. Click to review both the Sensor-TcpTextIn and Sensor-TcpTextOut GeoEvent Definitions.
Note: Do not make any changes to the GeoEvent Definitions as you review them.
March 2014
Page 32 of 37
In the illustrations above, note the following:
The identified owner of the Sensor-TcpTextIn GeoEvent Definition is the adapter responsible for
interpreting inbound JSON received by an input. The Sensor-TcpTextOut GeoEvent Definitions
was created by a user logged-in to GeoEvent Processor Manager as arcgis.
Sensor-TcpTextIn has a hierarchical structure with some fields contained within groups, whereas
Sensor-TcpTextOut has a flat structure supported by feature services.
The two GeoEvent Definitions have different field names for their GEOMETRY field.
March 2014
Page 33 of 37
The received sessionId is handled as a short integer, while the SessionID broadcast out is specified
as a string.
One of the primary purposes of a Field Mapper Processor is to bridge the gap between two GeoEvent
Definitions which specify different names or different data types to represent a particular data value.
18. Click the SensorMonitor GeoEvent Service to begin editing.
19. Double-click the No Operation processor and reconfigure it to be a Field Mapper.
20. Type Field Mapper for the Name.

21. Configure the remaining properties using the illustration below as a guide.
This processor will map specified fields from the Sensor-TcpTextIn GeoEvent Definition to the SensorTcpTextOut GeoEvent Definition.
March 2014
Page 34 of 37
In general, a Field Mapper Processor will attempt to convert data from the received data type to the
target fields data type. For example, a natural integer such as 324 could be converted to a string
representation 324. If an attempted conversion fails such as trying to convert a string 324AB to an
integer, the Field Mapper Processor will write a null value into the target field.
22. Click Ok on to save the processor.
23. Click Publish to publish the changes to the GeoEvent Service.
The Watch-Folder-for-JSON-Files input, which should be started, will stream event data into GeoEvent
Processor from the data file and use the Sensor-TcpTextIn GeoEvent Definition to interpret the JSON
data from the file. The Write-to-JSON-File output will apply the Sensor-TcpTextOut GeoEvent Definition
to deconstruct GeoEvents to represent the data as JSON in an output text file.
The SensorMonitor GeoEvent Service was reconfigured to incorporate a Field Mapper Processor which is
actually doing quite a bit of processing. It will select fields of particular interest, remove fields from
received events that are not needed in the output, convert data types on-the-fly as needed to transfer
data from the inbound to the outbound data structure, and flatten the received hierarchical data structure
to one supported by a feature service.
Follow the steps below to copy a sample JSON file to the system folder being monitored by the input and
observe the data transformation managed by the GeoEvent Service.
24. In GeoEvent Processor Manager, navigate to Services > Monitor.
25. Browse to the SensorData.json file in the \data\WorkingWithProcessors folder included with
this tutorial.
26. Copy and paste the SensorData.json file to the C:\GeoEvent\input system folder.
March 2014
Page 35 of 37
27. From the Monitor page, observe the event counts for your input, output, and GeoEvent Service all
increment.
28. In Window File Explorer, browse to the SensorData*.json file in the C:\GeoEvent\output folder
and open it in a text editor.
Notice the data sent to the output file conforms to the field mapping being performed by the
GeoEvent Service.
Congratulations!
By completing the exercises in this module, you learned how to configure several different types of
processors and incorporated them into GeoEvent Services which you published using GeoEvent
Processor Manager.
You explored how a Field Calculator Processor can be used to calculate and derive new data values from
data received with an event. A Field Calculator Processor is extremely versatile mathematic
expressions, string functions, and regular expressions can all be used to compute data values which can
be added to events as new attributes or used to overwrite existing event data.
You also examined how a Field Enricher Processor can be used to enrich received events with data
either from a published feature service or system text file. The key to understanding event enrichment is
to recognize that a join is being performed by the Field Enricher Processor, enabling data to be copied
from a secondary table to enrich events being received and processed by a GeoEvent Service. Attribute
data is not the only data which can be used to enrich events. Events can be spatially enriched with a
geometry retrieved from a published source of enrichment data.
Finally, you considered a pair of processors used to transform an events schema. A Field Reducer
Processor removed unwanted fields from the events. A Field Mapper Processor can be used to reduce the
schema of a received event to map or match it to a target GeoEvent Definition. A Field Mapper Processor
can also be used to facilitate data conversion between data types as well as flatten a data structure so that
the event data can be used to update a feature service.
March 2014
Page 36 of 37
In Module 5, you will explore some additional processors available in GeoEvent Processor. Specifically,
you will take a look at the GeoTagger Processor which combines the concepts of spatial proximity and
event enrichment to add properties from a GeoFence to events as they are received. You will also explore
the Track Gap Detector Processor that monitors an event stream to detect gaps in reported tracks,
enabling notification when expected data is not received. And lastly, you will explore the Incident
Detector Processor that is able to detect and report on the duration of incidents unfolding within an area
of interest such as when and for how long an asset exceeds a specified speed limit or operates outside of
an assigned work area.
March 2014
Page 37 of 37

Introduction To GeoEvent Processor - Module 4

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Introduction To GeoEvent Processor - Module 4

Caricato da

Copyright:

Formati disponibili

NOTE: The ArcGIS GeoEvent Processor team updates the product tutorials frequently to reflect the latest version

of the software. Depending on

Introduction to GeoEvent Processor Module 4

GEP 10.2.x / Rev B

Introduction to ArcGIS GeoEvent Processor Module 4

Introduction to GeoEvent Processor Module 4

Introduction & Module 4 Overview

Objectives for Module 4

Describe the different types of processors included with GeoEvent Processor.

Leverage processors to map GeoEvents from one GeoEvent Definition to another.

Introduction to GeoEvent Processor Module 4

Module 4 GeoEvent Processor Quick Start Configuration contents:

A tcp-text-in input and a tcp-text-out output.

A Flights GeoEvent Service which contains a No Operation Processor.

A Flights-TcpTextIn GeoEvent Definition used to interpret simulated flight event data.

A registered instance of ArcGIS for Server named LOCALHOST_AGS.

to start the tcp-text-in input.

Working with Processors

Introduction to GeoEvent Processor Module 4

Track Gap Detector

No Operation processors a processor with no configurable properties which performs no

Introduction to GeoEvent Processor Module 4

Field Calculator Processor

Introduction to GeoEvent Processor Module 4

5. Type Altitude / 3.28084 in the Expression property.

Introduction to GeoEvent Processor Module 4

11. Click Publish to publish the reconfigured Flights GeoEvent Service.

Uncheck the Continuous Loop checkbox.

Set the rate to send 9 messages every 1000 milliseconds.

Click Click to Connect to establish your server connection.

13. Open the TCP Console application introduced in Module 1.

14. In the GeoEvent Simulator, click Step

once to simulate a set of flight data events.

Introduction to GeoEvent Processor Module 4

Introduction to GeoEvent Processor Module 4

Mathematic functions supported by Field Calculator

Field Calculator (Regular Expression) Processor

Introduction to GeoEvent Processor Module 4

Introduction to GeoEvent Processor Module 4

Field Enricher Processors

Introduction to GeoEvent Processor Module 4

2. In the ArcGIS REST Services Directory, click IntroToGEP.

Introduction to GeoEvent Processor Module 4

(The layer at index 0)

(The layer at index 1)

6. Scroll to review the Fields in the feature layer.

Introduction to GeoEvent Processor Module 4

7. Scroll to the bottom of the page and click Query.

(wildcard indicating all fields should be shown)

9. Scroll to the bottom of the page and click Query (GET).

Introduction to GeoEvent Processor Module 4

Configure a Field Enricher (Feature Service) Processor Feature Service

Introduction to GeoEvent Processor Module 4

Introduction to GeoEvent Processor Module 4

Introduction to GeoEvent Processor Module 4

Uncheck the Continuous Loop checkbox.

Set the rate to send 9 messages every 1000 milliseconds.

Click Click to Connect to establish your server connection.

once to send a set of flight event data.

Introduction to GeoEvent Processor Module 4

Configure a Field Enricher (Feature Service) Processor Table

Introduction to GeoEvent Processor Module 4

You should see the folder listed under Registered Folders.