LogRhythm Balances Power, Simplicity

By P. J. Connolly  |  Posted 2010-09-13

LogRhythm Balances Power, Simplicity

Any serious IT compliance regime has to include processes for analyzing and interpreting the extensive, detail-packed log files produced by applications, servers and network equipment. This only sounds easy when you're not the one who has to go through these records on a regular basis; it's exponentially more difficult when you're trying to figure out what's going on inside your systems while it's still happening, and when you're faced with an in-progress crisis, the stakes can't get any higher.

The solution is to automate the chore of analysis and interpretation, but this requires a tool set that's highly scalable and capable of providing accurate results in a hurry. In this case, power is good, but ease of use is paramount.

LogRhythm in its namesake log management software provides a powerful and straightforward apparatus for collection and examination, but this comes at a cost: The published price for the company's LogRhythm LRX appliance is $25,000 for the 1U (1.75-inch) Windows-based system, and it increases for more complex installations. Customers also have the option of supplying their own hardware, if desired.

What one gets in return is a log parsing and management system based on Microsoft SQL Server that's designed to work with a wide range of operating systems and applications. If it generates a log file, the LogRhythm software can handle all but the most exotic cases without any special effort. When necessary, the company's engineers will work to develop an appropriate parser for the customer's needs.

A LogRhythm installation begins with a server (the Event Manager) running SQL Server 2005 and LogRhythm's ARM (Alarm and Response Manager) process; this machine manages the deployment's configuration and receives log entries that are considered noteworthy. In LogRhythm's jargon, these important or interesting logs are referred to as "events" and are used to generate alarms or other responses that are defined for the event.

But the Event Manager is just the brain of the operation. A LogRhythm deployment also relies on a SQL Server-based Log Manager, which runs the Mediator Server process. This collects log messages and, by applying predefined rules to the messages, determines whether they qualify as events to be forwarded to the Event Manager for further action. In a sizeable deployment, customers will find it necessary to run multiple Log Managers.

LogRhythm claims that the architecture is horizontally scalable to any conceivable degree, and the software can be deployed in a SAN (storage area network) environment or as a series of virtual machines. The software's own data integrity checks can verify that logs passed across trusted network boundaries or recovered from tape haven't been tampered with.

The other pieces of LogRhythm are the graphical .NET-based console for deployment management and interactive access to LogRhythm's stored data, which communicates with the Event and Log Managers via SQL Server protocols, and the System Monitor agents, which communicate with the Log Manager via a proprietary, encryptable application protocol. The monitor agents are typically installed on targeted systems, and a Log Manager system will usually also have a System Monitor installed. System Monitors provide file integrity checks as well, when these are enabled.

As noted above, the LogRhythm software doesn't just accept or collect logs from application, file and print servers; it also works with a variety of network security devices, such as Check Point firewalls, Cisco IDS (intrusion detection system) platforms and McAfee ePolicy Orchestrator, to provide a comprehensive view of what's going on in a network and when.

LogRhythm can process operating system and application logs from numerous Linux and Unix systems, as well as Windows event logs. It also handles standard syslog records and data sent with the NetFlow protocol. LogRhythm provides alarm or event notification to IT personnel via SMTP or SNMP, and includes a small truckload's worth of prepackaged reports intended to address the requirements of a variety of reporting schemes, including HIPAA (Health Insurance Portability and Accountability Act), PCI and Sarbanes-Oxley Act.

Testing the LogRhythm LRX2


Rubber, meet road

Bringing a LogRhythm appliance online begins with much the same process as any other Windows server. My lab testing used a single LogRhythm appliance-the 2U (3.5-inch) LRX2-that hosted both the Event Manager and Log Manager functions, running Version 5.1.1 of the software. The appliance arrives from the company with Windows Server 2003, SQL Server and the LogRhythm software in a preinstalled and unconfigured state; after supplying the machine with basic network details such as host name, domain membership and an IP address, I was ready to dive into the actual setup of the LogRhythm components.

For this release, that means tweaking a handful of config files with a text editor to inform the software of the networking basics (although the next major release of the LogRhythm components is expected to include a graphical setup utility that fills in those details at initial launch). At the end of this process, the system console pulls in a license file, and at that point you're ready to bring up the various LogRhythm services.

That is, unless you're me. In my first run with the LRX2, the company somehow provided me with a defective evaluation license that in effect ruined the installation. After spending the better part of a couple of days going back and forth with LogRhythm's support engineers, we agreed that the company would ship out a second unit and a usable license file.

On this second attempt, I was able to bring the system up without incident, and it was processing logs in a couple of hours. (LogRhythm has since reworked its procedures for evaluation units to ensure that future licenses are cut in a known good configuration.)

Mac OS X Server is the only brand-name server platform for which LogRhythm doesn't provide a system-specific log collection agent, but an agent for that OS is slated for Release 6 of the software. I found it relatively easy to configure the Xserve that I keep in the eWEEK Labs workroom to pass log entries to the LogRhythm host, which parsed them using a generic syslog filter for BSD, the basis for Mac OS X.

With a working system in hand and all needed services online, the LogRhythm appliance was then set to its primary task of log collection. This can be done by having logs pushed from the system in question, as I did with the Xserve, or pulled by a monitor agent, as one would do in an Active Directory domain. In the latter case, one configures the agent to run as a service, with the privileges necessary to collect logs from the Windows machines of the domain.

With the system collecting events, I found it fairly intuitive to use the console interface for interactive work with the data, in much the same way that one would when trying to analyze data to evaluate processes or to verify a sequence of actions. It's easy to set up canned searches for frequently used inquiries, and although it's probably a good idea for a user to have some background in data analysis to make the examination of log data more efficient, LogRhythm does an excellent job of insulating the user from the construction of SQL Server queries while still allowing valuable levels of detail.

There are a few things about LogRhythm that need to be addressed in future releases: A complete rework of the documentation would be a good place to start, since the company is already working on simplifying the setup process. I don't see the point in having hardware installation directions in the console's online help file, and the offline PDF documentation-currently a page-for-page translation of the WinHelp document-could stand to be broken up into discrete components that are usable away from the LogRhythm console.

But such quibbles aside, I have to admit that LogRhythm has successfully tackled the truly difficult parts of log analysis; it automates a great deal of the drudgework involved in report processing and allows IT personnel to focus on problem resolution. In short, it's not cheap, but it's money well spent.

Rocket Fuel