Mapserver versus ArcIMS

A Comparison of ArcIMS to MapServer

Brock Anderson Refractions Research

MUM/EOGEO 2005

www.refractions.net

Mapserver versus ArcIMS

Refractions Research
Victoria, BC, Canada Spatial Systems Consulting Open Source Software PostGIS uDig / GeoTools
MUM/EOGEO 2005

www.refractions.net

Mapserver versus ArcIMS

The Client

BCs Ministry of Sustainable Resource Management (Information Management Branch)

Use mostly ESRI products in their mapping infrastructure. Were frustrated with ArcIMSs administrative neediness. Were looking for WMS alternatives to ArcIMS. We suggested MapServer.
MUM/EOGEO 2005

www.refractions.net

Mapserver versus ArcIMS

Test Plan
We prepared a detailed test plan to compare ArcIMS and MapServer as WMSs. The tests covered:
Ease of administration WMS 1.1.1 standard compliance Interoperability with other software Performance

MUM/EOGEO 2005

* Ease of administration - KEY TEST: time to add and reload services * Interoperability with other software - KEY TEST: supports ArcSDE 8.3 and 9.x? * WMS Standard compliance - KEY TEST: OGC WMS CITE * Performance (with ArcSDE) - KEY TEST: feature density - KEY TEST: feature complexity - KEY TEST: image output format - KEY TEST: concurrency - KEY TEST: reprojection - KEY TEST: throughput "under regular operating conditions"

www.refractions.net

Mapserver versus ArcIMS

Test Environment
Test Client Computer WMS Protocol Test Server

JMeter 2.01

ArcIMS 4.0.1, WMS Connector 1.0 MapServer 4.2.1

ArcSDE 8.3

MUM/EOGEO 2005

Jmeter is open source software designed to load test functional behavior and measure performance .

www.refractions.net

Mapserver versus ArcIMS

Test Preparations
Created synthetic data to be used for certain performance tests. Made a JMeter extension to simulate a diversity of GetMap requests. Created ArcIMS AXL files and MapServer map files.

MUM/EOGEO 2005

Synthetic data used for feature density and feature complexity test. JMeter extension used to generate WMS requests with random bboxes. Started with AXL files, and transformed them into .map files

www.refractions.net

Mapserver versus ArcIMS

Creating .map files from .axl files

ArcIMS .axl service file
<! This is part of an XSL transform that converts ArcIMS .axl files into MapServer .map files -->

<xsl:template match="ARCXML"> <xsl:text>MAP</xsl:text><xsl:text>&#xA;</xsl:text> <xsl:text>EXTENT </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@minx"/><xsl:text> </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@miny"/><xsl:text> </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@maxx"/><xsl:text> </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@maxy"/><xsl:text>&#xA;</xsl:text> <xsl:text>UNITS </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/MAPUNITS/@units"/><xsl:text>&#xA;</xsl:text> <xsl:text>WEB </xsl:text><xsl:text>&#xA;</xsl:text> <xsl:text>METADATA </xsl:text><xsl:text>&#xA;</xsl:text> <xsl:text>"wms_title" "[service_name_here]"</xsl:text><xsl:text>&#xA;</xsl:text> <xsl:text>"wms_srs" "[EPSG:????]"</xsl:text><xsl:text>&#xA;</xsl:text> <xsl:text>END #end metadata </xsl:text><xsl:text>&#xA;</xsl:text> <xsl:text>END #end web</xsl:text><xsl:text>&#xA;</xsl:text> <!-- process the layers --> <xsl:apply-templates select="./CONFIG/MAP/LAYER"/> <xsl:text>END #end map</xsl:text><xsl:text>&#xA;</xsl:text> </xsl:template>

MapServer .map service file

MUM/EOGEO 2005

Our client had an optimized .axl file containing their provincial basemap data. We used that .axl file to create a .map file from.

www.refractions.net

Mapserver versus ArcIMS

Testing Began, and

Early performance tests showed ArcIMS outperformed MapServer
Number of Concurrent Requests vs. Response Time
9000 8000
Average Response Time (ms)

8483

7000 6000 5000 4000 3000 2000 1000 0 1 2 4 8 Number of Concurrent Requests 702 775 2703 2793 3732 2502 1125 ArcIMS MapServer

MUM/EOGEO 2005

An initial test to convince ourselves that MapServer was comparable to ArcIMS We requested the same map each time: 4 layers (including points, lines and polygons) Each level of concurrency run for 10 minutes. Mention that 1) all graphs have at least 30 samples per point. 2) All tests done against ArcSDE.

www.refractions.net

Mapserver versus ArcIMS

MapServers Bottlenecks
Profiling revealed two main bottlenecks:
1 to 2 seconds of ArcSDE connection overhead per GetMap request Additional overhead extracting features from ArcSDE

MUM/EOGEO 2005

www.refractions.net

Mapserver versus ArcIMS

Minimizing the Connection Overhead

Persistent database connections would nearly eliminate ArcSDE connection overhead. As a CGI program, MapServer had no means to support persistent connections. Added FastCGI support. Thanks Frank! Updated the ArcSDE module to utilize persistent connections. Thanks Howard!
MUM/EOGEO 2005

Frank Warmerdam added FastCGI support to MapServer, and he created a connection pooling API Howard Butler updated the ArcSDE module to utilize connection pooling

www.refractions.net

10

Mapserver versus ArcIMS

Using MapServer with FastCGI

1. Compile MapServer with FastCGI support. 2. Configure your web server with a FastCGI module. 3. Update your map files:
LAYER PROCESSING CLOSE_CONNECTION=DEFER #all other layer settings here... END #LAYER

* Persistent connections are most useful for data sources with large connection overhead, such as ArcSDE.
MUM/EOGEO 2005

www.refractions.net

11

Mapserver versus ArcIMS

Other Enhancements
Recall, connection overhead was not the only slowdown. We also improved the MapServer code which pulled features from ArcSDE.
This involved experimenting with ESRIs ArcSDE C API. We discovered which operations were costly, and cut down on their use.

MUM/EOGEO 2005

*Performance is best for the default version of ArcSDE layers.

www.refractions.net

12

Mapserver versus ArcIMS

The Same Test Again

The earlier performance test was run again, this time with FastCGI and the other performance improvements.
Number of Concurrent Requests vs. Response Time
3000 2500
Average Response Time (ms)

2384 1867 MapServer 1067 1114 1104 861 934 916 ArcIMS

2000 1500 1000 500 0 1 2 4

Number of Concurrent Requests

MUM/EOGEO 2005

www.refractions.net

13

Mapserver versus ArcIMS

Performance Test Results

(Throughput)

MapServer now outperforms ArcIMS under MSRMs regular operating conditions.

Regular Traffic Over an Extended Period (Throughput)
4 3.5 3
Maps Per Second

2.5 2 1.5 1 0.5 0 1 2 3 4 5 6 7 8 Number of Concurrent Users ArcIMS MapServer

MUM/EOGEO 2005

The higher the better on this graph only Difference between concurrency test and this test: This test uses random requests for 10 min. This test doesnt hit the server with a sudden burst of requests. They are ramped up.

www.refractions.net

14

Mapserver versus ArcIMS

Performance Test Results (Image Format)

MapServer is slightly faster to return GIF and PNG images. ArcIMS is faster for JPG.
Image Format vs. Response Time
3000 2500 2000 1500 1000 500 0 JPG GIF Image Format PNG ArcIMS MapServer

MUM/EOGEO 2005

www.refractions.net

Average Response Time (ms)

15

Mapserver versus ArcIMS

Performance Test Results (Reprojection) (Reprojection)

MapServer reprojects faster than ArcIMS.
Reprojection vs. Response Time
8000 7000
Average Response Time (ms)

6000 5000 4000 3000 2000 1000 0 BC Albers (NAD83) Lat-Long (NAD 83) Output Projection UTM10 (NAD 83) ArcIMS MapServer

MUM/EOGEO 2005

www.refractions.net

16

Mapserver versus ArcIMS

Performance Test Results (Feature Complexity)

Feature complexity affects both servers almost equally (from ArcSDE).
Feature Complexity vs. Response Time
30000 25000 20000 MapServer (SDE) 15000 10000 5000 0 4 8 16 32 64 128 256 512 Number of Points in Each of 10,000 Circles ArcIMS (SDE) MapServer (.shp)

MUM/EOGEO 2005

www.refractions.net

Average Response Time (ms)

17

Mapserver versus ArcIMS

Performance Test Results (Feature Density)

MapServer extracts features from SDE slightly faster than ArcIMS does.
Feature Density vs. Response Time
60000 50000 40000 MapServer (SDE) 30000 20000 10000 0
00 00 00 00 00 00 00 00 00 00 00 80 00 48 12 72 24 84 36 60 96 20 40 00 0 0 0 0 0 0 0 0

Average Response Time (ms)

ArcIMS (SDE) MapServer (.shp)

10

12

MUM/EOGEO 2005

Number of Features

Actually, this test demonstrates that MapServer (extracts features from SDE) + (draws the map) faster

www.refractions.net

13

14

18

Mapserver versus ArcIMS

WMS Standard Compliance

MapServer passed all 83 WMS CITE tests. ArcIMS (w/ WMS connector) passed 71 of 83 WMS CITE tests.
ArcIMS failed 12 tests because:
Wrong MIME type for some responses. Wrong exception code in some exceptions.

MUM/EOGEO 2005

MIME type of responses is typically expected to be application/vnd.ogv.se_xml

www.refractions.net

19

Mapserver versus ArcIMS

Ease of Administration
MapServer administration benefits
No need to reload services (when service files change) MapServer restarts faster (as fast as the web server)

ArcIMS administration benefits

More granular control over log levels

MUM/EOGEO 2005

www.refractions.net

20

Mapserver versus ArcIMS

Interoperability and Support

Both WMSs support:
ArcSDE 8.3 and ArcSDE 9.0 Styled Layer Descriptor (SLD)

MUM/EOGEO 2005

I dont know how fully either server supports SLD, but our testing showed that both support these basic features: Selecting and styling based on a attribute value Selecting and styling based on spatial location (within a bounding box)

www.refractions.net

21

Mapserver versus ArcIMS

Conclusions
MapServer is easier to administer than ArcIMS. MapServer is more WMS standard compliant than ArcIMS. MapServer matches or surpasses ArcIMS in most performance tests. * (MSRM adopted MapServer for their COINPacific web mapping application)
MUM/EOGEO 2005

www.refractions.net

22

Mapserver versus ArcIMS

Questions?
Contact me:
Brock Anderson banders@refractions.net Refractions Research www.refractions.net (250) 383-3022

MUM/EOGEO 2005

www.refractions.net

23