http://github.

com/ksmpartners/ernie

Agenda

What is Ernie? Business case Project overview

BIRT Report Designer o Embedded API (demo) o Ernie server (demo) o Other modules
o

Performance

What is Ernie?
An asynchronous report generator that can be embedded in a Java/Scala application or exposed as a RESTful web service. Ernie generates reports based on definitions created with the BIRT Report Designer (supports JDBC, web services, Hive/Hadoop, and XML as data sources). Ernie provides CRUD management of report definitions and report output.

Business case: why Ernie?

Asynchronous over real-time reporting Reduced hardware cost

Less silicon investment required when real-time speed is not expected and report processing can be confined to a specific thread count / time window regardless of user load. Many real-time reporting systems constantly preprocess data to allow online reporting. Asynchronous report processing takes place only once a request is submitted.

Scalability -- vertical scaling will affect report processing speed, not maximum load.

Business case: why Ernie?

Report definitions are standalone entities containing data sources/queries, transformations, business logic, presentation markup -- all of which can be parameterized.

Developers/users need only manage two resources: definitions and output Reduced software cost: less UI to create and maintain (no web report viewer or designer necessary) Increased flexibility: through parameterization, a single report definition can serve many distinct purposes and generate output in different formats

Business case: why Ernie?

Open source BI

In 2010 study, open source technologies were found to power more than 50% of polled enterprises core infrastructure, and adoption focus is moving further up the stack (application servers, BI tools) Prediction that open source adoption will become / is becoming a strategic rather than tactical maneuver o That is, a choice made by business leaders rather than developers o Lower initial costs, more flexible support sourcing, more flexible integration o (Likely obvious to everyone here)

Business case: why Ernie?

Open source BI However: BI tooling is still >90% commercial BIRT was found by a Forrester study to be the clear leader among open source BI tools due to richness of reporting functionality But the study notes that the BIRT provides individual components (runtime engine, report designer) rather than a deployable package Enter Ernie

Technologies
Eclipse BIRT akka a toolkit and runtime for building highly concurrent,
distributed, and fault tolerant event-driven applications on the JVM

Lift web framework a mature scala web framework

Project overview
Main components:
Client-side Ernie engine akka ActorSystem Ernie embedded API (validation, error handling)

Ernie server

BIRT Report Designer

Embedded APIs
Idiomatic scala/java APIs for interacting with Ernie.

managing report definitions and their metadata

creating a report generation job based on a definition

interrogating the status and metadata of a job

managing job output

retrieving lists and catalogs of report generation jobs and definitions

Ernie server
Provides the same functionality as the embedded APIs in the form of a REST web service.

Complies with the IETF HTTP 1.1 semantics

Generates Swagger JSON specification and UI Supports SAML and HTTP Basic authentication/authorization

Also:
Ernie-gatling A lightweight stress testing suite written using Gatling. Scenarios written in Scala DSL and output generated using Highcharts. Ernie-model Serializable domain objects used for all information passing and persistence.

Ernie performance
Ernie-gatling code
Each simulated user: 1. 10% of the time, create a definition. 2. Get all available definitions 3. Repeat the following 5 times: a. 50% of the time, post a job and get the result b. 50% of the time, pause for 2 seconds then post a job 4. Repeat 3 times: get the output of any of the users jobs Over 2 minutes, start 10000 of these sessions

Ernie performance
Embedded API results overview Embedded API is able to process around 35000 requests for report
output retrieval and 10000 report generation jobs in a total of 3 minutes
Mean response time of less than 120 ms for over 200 requests per second.

Ernie performance
Output retrieval response time vs Global # requests/sec

See also

http://github.com/ksmpartners/ernie Eclipse BIRT homepage Actuate iServer/iHub for a commercial alternative Comparison of open source BI tools
http://www.innoventsolutions.com/open-source-reporting-review-birt-jasper-pentaho.html