Project LASSO for Open Source Log Management Dr.

Anton Chuvakin WRITTEN: 2007 DISCLAIMER: Security is a rapidly changing field of human endeavor. Threats we face literally change every day; moreover, many security professionals consider the rate of change to be accelerating. On top of that, to be able to stay in touch with such ever-changing reality, one has to evolve with the space as well. Thus, even though I hope that this document will be useful for to my readers, please keep in mind that is was possibly written years ago. Also, keep in mind that some of the URL might have gone 404, please Google around. Drowning in logs is all too common a sight nowadays when organizations are trying to struggle with a combination of operational, security and compliance requirement. A typical organization will have logs coming from a wide array of log sources such as server operating systems (Unix and Windows), desktops, mainframes, network gear such as routers and switches, web proxies, security gear such as network IDS, IPS or anti-virus tools, web, email, DNS server software as well as enterprise applications. Log Management and Intelligence is an approach to dealing with large volumes of computer-generated log messages (also know as audit records, audit trails, event logs, etc) which consists of log collection, centralized aggregation, long-term retention, log analysis (in real-time and in bulk after storage) as well as sharing the information with the relevant parties within the organization. Such analysis is usually performed for security, operational (such as system or network administration) or regulatory compliance reasons. Effectively analyzing large volumes of diverse logs faces many challenges such as huge log volumes (reaching hundreds of gigabytes of data per day for a large organization), log format diversity, undocumented proprietary log formats (that hinder analysis) as well as a presence of false log records in some types of logs (such as intrusion detection logs) Tools to handle the log collection and analysis are sometimes build by users, assembled from various open source components or acquired from commercial vendors, in the form of LMI or Log Management and Intelligence solutions. So far, the open source space have not been able to come up with a single tool to deal with a majority of log challenges, even though there are some promising contenders.

However, an open source community has been pretty effective in building pieces of a log management infrastructure. Syslog-NG, that enables log collection from Unix servers and network devices as well as serves as a better replacement for standard syslog daemons provided by the operating system vendors, is a primary example. Also, a huge number of simple scripts and small programs such as logwatch, logsentry, fwanalog were written by the open source community over the years to handle specific logs or with a particular slice of a log puzzle. At times it seems that it was easier for some to create their own script instead of looking for one online. However, a majority of these tools focused on Unix and Linux platforms and largely ignored Windows. One of the recent open source solutions that enable a critical part of log management is Project LASSO. Project Lasso is Windows-based open source software designed to collect Windows event logs, including custom application logs, and provide central collection and transport of Windows log data via TCP syslog to any syslog-NG compatible log receivers. Before Project LASSO, incorporation of Windows server and workstation logs in an overall log management process was extremely onerous. One had to use agents installed on every single Windows system to collect logs or be stuck with super-expensive proprietary solutions. And deploying agents on every system is one of the most hated pursuits in the whole domain of enterprise IT. Open source tools such as syslog-ng existed for years to simplify log management for Unix and Linux operating systems as well as network devices that support syslog (such as Cisco routers and firewalls), but Windows part of the world was largely excluded since binary Windows event logs are not syslog. Now, Project LASSO allows remote Windows log collection (it can also be deployed as an agent on each server, if needed) and then inclusion of such logs into a log management systems, such as the one by LogLogic or others. Overall, Project LASSO enables connecting the dots by enabling users to collect analyze Windows event logs with the same ease that they are used to in the Unix and Linux realm. After the data is collected by LASSO, one can use report and search to review and analyze logs across all the systems in the enterprise: Windows, Unix, network systems, application, etc. For example, one can run a query for the same user across all the systems he or she touched. Such capability is critical for compliance as well as incident response and forensics

ABOUT THE AUTHOR:

This is an updated author bio, added to the paper at the time of reposting in 2011. Dr. Anton Chuvakin (www.chuvakin.org) is a recognized security expert in the field of log management and PCI DSS compliance. Anton leads his security

consulting practice www.securitywarriorconsulting.com, focusing on logging, SIEM, security strategy and compliance for security vendors and Fortune 500 organizations. He is an author of books "Security Warrior" and "PCI Compliance" (www.pcicompliancebook.info) and a contributor to "Know Your Enemy II", "Information Security Management Handbook"; and now working on a book about system logs. Anton has published dozens of papers on log management, correlation, data analysis, PCI DSS, security management (see list www.infosecure.org). His blog www.securitywarrior.org is one of the most popular in the industry. In addition, Anton teaches classes (including his own SANS class on log management) and presents at many security conferences across the world; he recently addressed audiences in United States, UK, Singapore, Spain, Russia and other countries. He works on emerging security standards and serves on advisory boards of several security start-ups. Dr. Anton Chuvakin was formerly a Director of PCI Compliance Solutions at Qualys. Previously, Anton worked at LogLogic as a Chief Logging Evangelist, tasked with educating the world about the importance of logging for security, compliance and operations. Before LogLogic, Anton was employed by a security vendor in a strategic product management role. Anton earned his Ph.D. degree from Stony Brook University.