LinuxSecurity.com
Share your story
The central voice for Linux and Open Source security news
Home News Topics Advisories HOWTOs Features Newsletters About Register

Welcome!
Sign up!
EnGarde Community
Login
Polls
What is the most important Linux security technology?
 
Advisories
Community
Linux Events
Linux User Groups
Link to Us
Security Center
Book Reviews
Security Dictionary
Security Tips
SELinux
White Papers
Featured Blogs
All About Linux
DanWalsh LiveJournal
Securitydistro
Latest Newsletters
Linux Advisory Watch: August 29th, 2014
Linux Security Week: August 25th, 2014
Subscribe
LinuxSecurity Newsletters
E-mail:
Choose Lists:
About our Newsletters
RSS Feeds
Get the LinuxSecurity news you want faster with RSS
Powered By

  
Centralized File-Integrity With Samhain Part I Print E-mail
User Rating:      How can I rate this item?
Features This article will discuss installing and configuring a secure, centralized file-integrity program. Later articles in this series will discuss specific features, like deploying packaged clients to hosts on your network, creating customized reports, and other cool Samhain features.

Centralized monitoring is critical today with the large number of servers that are deployed in many organizations. Reading reports from individual servers can be quite cumbersome and time-consuming. Programs are needed that will provide accurate and secure information about the state and health of servers in a timely manner in a centralized location, whether with one server or multiple servers. One of the most critical needs is a program that will alert an administrator when files have been changed on a system whether intentionally or as the result of a security compromise.

Many programs exist that provide from very basic file-integrity to very specific file-integrity. For example, some file-integrity checks look for any file that has changed within a directory and others can ignore if a file has been accessed or grown in size but alert if the permissions have changed. The level of security needed for file-integrity can depend on a number of factors. How many servers are in an organization, the time an administrator has to spend looking at file-integrity reports, the level of training for the administrator, the cost of a good file-integrity program, the sensitivity of data stored on a server, etc.. There are some superb commercial applications that provide fine-grain control of file-integrity configurations. Of course, it can cost quite a bit of money for these programs but if the needs fit an organization this may be their only solution. There are some great free programs that can rival many of the commercial applications. However, some require extra configuration with other third-party software to have logs sent to a central location securely, for example. Some organizations don't have the administrators who have the time to create custom programs to add extra functionality to the software. There exists software that will run as a daemon on all servers to be monitored, including the centralized monitoring server, send reports over a secure channel, by default, provide access control to determine whether that hosts is allowed to send alerts to the daemon, deploy the client on remote hosts over SSH, runs on Windows, Solaris, Linux, AIX, and other platforms, with tons of other features that is available under the GPL license, Samhain.

Samhain (pronounced "Sowen". "sow" as in a female pig) is a file-integrity program that provides high-level configuration for determining what files and/or directories have been changed, deleted, or added. It can also keep a record of files that have the suid or sgid bit set, as well. For Linux, it can send an alert if a new kernel-module has been loaded. Samhain provides a secure, centralized logging facility by having the client(s), that perform the integrity checks, on servers, send encrypted reports to a remote server. Samhain provides a ton of extra features that are very well documented in the manual that comes along with the distribution. Of course, this series of articles will give you a real-world working example of how to deploy Samhain.

Installing Samhain

The central logging server will be called pango.example.com.

To install Samhain download the samhain tarball, verify the pgp signatures, and extract it.

Linux:

 
tar xzvf samhain-x.x.x.x.tar.gz

Solaris & AIX:

 
gzip -dc samhain-x.x.x.x.tar.gz | tar xvf -

#############################################################

SERVER INSTALL -- ONLY NEEDED FOR THE CENTRAL FILE-INTEGRITY SERVER
If you are using RPMS then you will need to download the mysql-devel package in order to compile Samhain with MySQL support

FILE INTEGRITY SERVER...IN THIS CASE, pango.example.com.

Samhain server install configured to log to a mysql database:

cd //samhain-x.x.x

./configure --enable-network=server --with-database=mysql --enable-xml-log --with-libs=-L/usr/lib/mysql/ -with-cflags=-I/usr/include/mysql

make

make install

NOTE: the "--with-libs=-L/usr/lib/mysql/" and "--with-cflags=-I/usr/include/mysql" are specified, explicitly, because on Red Hat the configure script doesn't properly locate the "libmysqlclient.a" library or the "mysql.h" header file.

NOTE:On AIX 4.3, the Makefile had to be modified and have the "-O3" optimizations flags removed. These appear to specific to gcc. If gcc is installed on the server then it may work with the modifying the Makefile. This only needs to be done if an optimize error occurs during the "make " process.

The "make install" will create a program called yule in /usr/local/bin. Yule is the server that listens on pango.example.com for incoming integrity reports. Its configuration file is /etc/yulerc, by default.

The "DatabaseSeverity" directive will have to be specified in the /etc/yulerc file on pango.example.com.


DatabaseSeverity=crit

DbHostname=pango.example.com

DbName=samhain

DbTable=log

DbUsername=samhain

DbPassword=password

Be sure that the yulerc file is configured to be readable and writeable only by root.

 
chmod 600 /etc/yulerc

The next article will explain how to setup the yulerc file so that the database password (actually the entire yulerc file) and the Samhain daemon can be virtually hidden.

Samhain currently supports MySQL and PostGRESQL. The nice thing about logging to a database is that you can then use PHP, Java, PERL or some other language to extract the data and view it over the web or create your own purdy reports. Samhain has a beta program called Beltrane that will do this work for you and provide an interface over the web to view the alerts and logs for clients.

END SERVER INSTALL

#############################################################

Client install: (See NOTE: for AIX install)

 
cd //samhain-x.x.x

./configure --enable-network=client --enable-xml-log
make

make install

This will add the samhain configuration file, samhainrc, to /etc and install the "samhain" executable in "/usr/local/sbin". That executable will be overwritten with the next step. Create a unique 16-digit number that will be added to pango.example.com to validate sending logs to the Samhain server daemon. All logs will be encrypted and sent to pango.example.com in the XML format, which will allow logging to the mysql server.

While in the root directory of the samhain source distribution run the command:

/usr/local/sbin/samhain_setpwd samhain pudding

eg. /usr/local/sbin/samhain_setpwd samhain pudding 1234567890123456

You have to choose a random 16-digit number to pass to the "samhain_setpwd" command. This will create a new executable for samhain in the current directory called, samhain.pudding. The "pudding" is really an arbitrary name, actually it is my sister's nickname. Move that one to the "/usr/local/sbin/" directory which will overwrite the default samhain executable created during the "make install" process.

mv -f samhain.pudding /usr/local/sbin/samhain

Copy the 16-digit number to a sheet of paper or open a console on pango.example.com. This number will be used to create the unique checksum to allow this server to authenticate and send logs to pango.example.com. On pango.example.com...run the command:

/usr/local/sbin/yule -P <16-digit number for the server>

For example, in the example above. Run the command:

/usr/local/sbin/yule -P 1234567890123456

The above command creates this checksum:

Client=HOSTNAME@FBE67F98C36DB5DF@30015639090F9CB064937DC58A0E70644B20EEA083AB7
E2BFF6C1D521D7675FD2DDC7BAEB745F59695B342028D548C72E7DFF135D1E9A05987EC
1D503E8FB1E248F035497924C2C1069B6615DDB35E2FA64D3608DFA3BDD53DD8D
D7B997A4B8BE0FB2C2BA2F50E0895B8015D795D7B5623FB924CEF3AC8E065FE6D810D971

append the output of the "yule -P" command to the end of the /etc/samhainrc file, under the [Clients] section and change "HOSTNAME" to the name of the server that the number was created on.

Configuring the client:

The /etc/samhainrc file is heavily documented and provides various levels of file-integrity. It can be configured to check files and directories, that always change, ignore access times, size, check for new files, deleted files, etc.. It can also check for new suid and sgid permissions, it will also send alerts if a file or directory has a uid or gid that doesn't exist on the system.

Below are some specific cofigurations added outside of what already exists in the samhainrc file. The default configuration file is very well documented with some examples. Below are some configurations that were added for my setup.

ChecksumTest=check

SetNiceLevel=19

SetIOLimit=500

SetFilecheckTime=1800

SetMailTime=86400
#Wait for this many messages to accumulate before sending an email
SetMailNum=10

#Prevent messages from being echoed to the console

SetConsole=/dev/null

SetLogServer=pango.example.com

#Time between sending TIMESTAMP messages

SetLoopTime=86400

Don't send duplicate reports for a modified file:

#Takes a yes/no argument
ReportOnlyOnce=yes

After configuring the Samhain /etc/samhainrc file, create the database. This may take some time, depending on what files and directories it has to create a checksums for and the suid files it has to check. Also, the priority specified in the configuration file can impact how fast the database is created.

samhain -t init

After the database is created and the steps have been followed above to create a unique key for the host the samhain client is on, start the samhain client

samhain -t check -D -e 120

This will run samhain as a daemon (-D) and send alerts every 120 seconds (-e 120) to pango.example.com. Use 120 seconds until you are comfortable with your setup then you can set the time between checks longer. Note that in the configuration file the "SetNiceLevel" is set to 19, which means the file check will run quite slow. This will keep the samhain daemon from using up a lot of CPU but it will cause the reports to be delayed when sent to pango.example.com. You may want to set the "SetNiceLevel" to something lower until you have Samhain configured correctly. Be sure to check the /var/log/yule_log file on pango.example.com for any problems with the hosts sending alerts. If Samhain, on pango.example.com, doesn't have a valid key for the hosts then it will not accept any alerts from it. A message will be echoed to /var/log/yule_log alerting that the host is invalid.

To start the server on pango.example.com, in this case:

yule -S

This will run the yule server in the background. Again, check the /var/log/yule_log file for an error messages with yule starting up or check it to be sure that the yule server started properly. If you started the server after the client, wait a few minutes to see the connection attempts by the clients:

tail -f /var/log/yule_log

When the connection attempts start and are successful, there will be a lot of information echoed to the /var/log/yule_log file. This information can be tuned up or down depending on how much information you want. There is also an html file in the /var/lib/samhain directory called "yule.html". This file is updated every 120 seconds, by default. It reads the /etc/samhainrc file for the hosts that are allowed to connect to the yule server and gives their status. It will display whether a client is connected or whether a policy has been sent from that client, etc.. You can create a symbolic link from that location to the web server directory on pango.example.com and view it anytime. Be sure that you read and understand the security risks involved with creating symbolic links in a web directory. For any monitoring server, I like to use SSH tunneling to access it. Generally, no services are available directly from the public interface. For example, the web server can be forced to listen on the local interface for incoming http requests in the httpd.conf file with the Listen directive set to:

127.0.0.1:80

In order for anyone to connect to the server they will have to be on pango.example.com from the console or they can create an SSH tunnel to access the web server.

Conlusion

It is hoped that this article provides a good introduction with some of the capabilities of Samhain, how to install it, and set up a centralized file-integrity server. Samhain comes with a detailed manual in the source distribution. The next series of articles will explain other features of Samhain especially how to deploy Samhain, quickly, to multiple servers, hiding Samhain, using the yule server for a remote syslog server, how to setup Beltrane, and sample php and servlet scripts for extracting data from the Samhain log file or database to create customized web-based reports.

Special thanks

First and foremost, thanks to the Linuxsecurity.com team for their continued support with publishing my articles. Especially, Dave Wreski since I never send him an article on time because I always catch something that isn't right about it at the last minute. Thanks for your patients Dave. ;-)

Some LinuxSecurity list members provided a lot of help with proofreading this article and giving suggestions and comments: Emily Ratliff (many thanks!!), Bernard Hoffmann, Matt Hemingway, and Andy Grimm.

Thanks to Bone, Chris, Cris, Barium Spring Home for Children ("The Foundation of Duane's Path to Liberation"), Charla, Chrissy, Mr. David, Bob, Donna, CFCC, Pfeiffer University, Leslie, Adam, STG, NCDC, Lauren, Jason, and mutsman for their continued support for all that I do.

About the Author

Duane Dunston is a Computer Security Analyst at STG Inc. for the National Climatic Data Center in Asheville, NC. He received his B.A. and M.S. degrees from Pfeiffer University and he has his GSEC certification from SANS. He writes poetry, just started photography, and hangs out at Old Europe Cafe, Early Girl's eatery (tell'em Duane sent you), and still wakes up every morning ready to go to work. If anybody works at NBC who reads this, tell Ann Curry he says, "Hello", again.

Comments
LogsWritten by Joe on 2008-02-02 16:34:40
I have succeeded in the installation and configuring of samhain and yule. Yule is logging any CRIT level messages to a mySQL DB. The problem I have is, how do I make use of any of the logs in SQL. Almost every field appears to be null. And the fields that are not null are not helpful in the slightest. 
 
Thanks
Written by Vik on 2008-09-10 14:43:34
hey! 
 
I am a beginner in Linux and its applications. So please have patience with me. 
I installed Samhain for a standalone system and then customized it according to my requirements. Further the database got initialized and then later on at a different time interval I compared the database. 
Where does the output go? Does it go in a file? Or do I have retrieve it from the database and if yes then how? 
If you can share more information on this then that would be great. 

Only registered users can write comments.
Please login or register.

Powered by AkoComment!

 
< Prev   Next >
    
Partner

 

Latest Features
Peter Smith Releases Linux Network Security Online
Securing a Linux Web Server
Password guessing with Medusa 2.0
Password guessing as an attack vector
Squid and Digest Authentication
Squid and Basic Authentication
Demystifying the Chinese Hacking Industry: Earning 6 Million a Night
Free Online security course (LearnSIA) - A Call for Help
What You Need to Know About Linux Rootkits
Review: A Practical Guide to Fedora and Red Hat Enterprise Linux - Fifth Edition
Yesterday's Edition
How Cops and Hackers Could Abuse California’s New Phone Kill-Switch Law
Why Russian hackers are beating us
DQ Breach? HQ Says No, But Would it Know?
Partner Sponsor

Community | HOWTOs | Blogs | Features | Book Reviews | Networking
 Security Projects |  Latest News |  Newsletters |  SELinux |  Privacy |  Home
 Hardening |   About Us |   Advertise |   Legal Notice |   RSS |   Guardian Digital
(c)Copyright 2014 Guardian Digital, Inc. All rights reserved.