Loki+Lire

A Framework to Create Web-Based

Multimedia Search Engines!
!
G. Becchi, M. Bertini, L. Cioni, A. Del Bimbo, A. Ferracani, D. Pezzatini
M.Lux

What is Loki?
A modular framework for the creation of web-based interfaces able to
handle different type of media

Client-side

Server-side

Presentation

Ingestion

Annotation

Transcoding

Collection management

Indexing

Tag/Content Based
Retrieval tools

Retrieval engine

Who can use Loki?

Researchers who need an interface to demo their own

multimedia annotation system
!

Researchers who need to create ground truth annotations

to create datasets or validate experiments (e.g. using
Amazon Mechanical Turk)

Lecturers and teachers who need an easily installable

tool for their teaching lab activity

Whats inside Loki

Solr / Lucene
Text indexing and search

Multimedia search UI

LIRE: Lucene Image Retrieval

Image component

Multimedia tools
Video component

Doc component

Audio component

Media transcoding
Media analysis pipelines

Storage
media, metadata, collections
and users management

Client side

Front-end development according to HTML5 standards

!

AngularJS for web application using the MVC design

pattern.
!

Media components developed as jQuery plugins using

properties to personalize their functionalities and
appearance.

er video
Video

o
ni ed
e

di

mili

Synchronized annotations / speech transcription

er video
Video

o
ni ed
e

di

mili

Frame-accurate annotation

er video
Video

o
ni ed
e

di

mili Frame-accurate annotations search

er video
Video

o
ni ed
e

di

mili

Similar frames proposed based on CBIR

Il componente per audio

Riproduzione audio

Audio

Visualizzazione forma donda per la navigazione

Aggiunta annotazioni

Ricerca annotazioniSynchronized

annotations / subtitles

17 di 19

Il componente per audio

Riproduzione audio

Audio

Visualizzazione forma donda per la navigazione

Aggiunta annotazioni
Ricerca annotazioni

Timecode accurate annotation and search

17 di 19

Image

magini

ea

Global image annotation

Image

magini

ea

ea

Local image annotation

er documenti
Document

umenti in
SVG

ut di pagina

pagine

Zoomable SVG page

er documenti
Document

umenti in
SVG

ut di pagina

pagine

Page thumbnail preview

er documenti
Document

umenti in
SVG

ut di pagina

pagine

Local textual annotations

Document

Textual search with page preview

I risultati
di
ricerca
Loki - a cross media search interface
La visualizzazione dei risultati della ricerca
Media-based results collection

I risultati
di
ricerca
Loki - a cross media search interface
La visualizzazione dei risultati della ricerca
Textual queries

I risultati
di
ricerca
Loki - a cross media search interface
La visualizzazione dei risultati della ricerca
Drag/drop for cross-media content based search

I risultati
di
ricerca
Loki - a cross media search interface
La visualizzazione dei risultati della ricerca
Tag-based facets for result filtering

I risultati
di
ricerca
Loki - a cross media search interface
La visualizzazione dei risultati della ricerca
Result clusters based on similarity to enhance diversity

Server side

Relational DB to manage media, metadata, users and

collections
!

Apache Solr to index textual documents, automatic/

manual annotations on media

!
!

Content based image retrieval using LIRE plugin for Solr

!

SOAP-based architecture to handle media processing

results
!

Ingestion and transcoding routines

LIRE: A Solr plugin for CBIR

Enable Solr for content based image retrieval by:
1) converting image features into indexed text based documents
2) hashing features so to allow inverted index search for sub-linear
retrieval

Lire Solr plugin supports global

image features like MPEG-7
ColorLayout, EdgeHistogram,
JCD, PHOG, and color histogram
in the opponent color space.

Loki and LIRE

Query by example

CBIR results

Loki and LIRE

Query by example

CBIR results

results cluster based

on visual similarity

Media transcoding and analysis

Transcode
for HTML5
Upload
Manager
Extract
keyframes

Video
content

Loki

Index keyframes
for CBIR

Media transcoding and analysis

Text
extraction

Indexing

Generate
SVG pages

Generate
JPEG thumbnails

Upload
Manager
Docs / PPT

Loki

Live demo

Why choose Loki?

Ready & easy to use web user interface for multimedia

search and management
!

Based on standards and leader open source tools

!

Easily extendable with new functionalities (using Solr

plugins) and media analysis pipelines.
Add your own processing.
!

Fully modular system. Choose the modules you

need!

Installation & Licenses

To deploy Loki you need:

Apache + MySQL for web interface and storage

Apache Tomcat for with Solr installation

LOKI Github repository

https://github.com/miccunifi/Loki
Apache License 2.0

LIRE Bitbucket repository

https://bitbucket.org/dermotte/liresolr
GNU General Public License (GPL) v2

Contacts
G. Becchi, M. Bertini, L. Cioni, A. Del Bimbo, A. Ferracani, D. Pezzatini
Media Integration and Communication Center
University of Florence, Italy.
[name.surname]@unifi.it

!
Mathias Lux
ITEC Klagenfurt
Klagenfurt University, Austria
mlux@itec.aau.at

Acknowledgment
Part of the research leading to these results has received funding from the EU 7th FP managed
by REA (FP7/2007-2013) under grant agreement no. 262428