An interview with Brian Gemberling, creator of the PullthePlug.com project. Brian invites everyone to find security vulnerabilities on his open systems. After visiting PullthePlug.com, I thought that our readers would be interested in finding out more about the project. This email address is being protected from spambots. You need JavaScript enabled to view it., the project leader, has offered multiple Linux, BSD, and CISCO systems to the public for exploration. A listing of available systems with IP address and login information can be found here. Brain invites everyone to use his systems to explore the possibilities of current technology. We decided to interview him because of the number of security concerns that he needed to address. We hope that by reading this interview you have a better understanding of how systems are compromised and how to prevent future incidents.

LinuxSecurity:When and how did you gain interest in security? How did you gain your security knowledge? How did 'PullthePlug.com' begin?

Gemberling:I really got interested in computer/network-based security in my last two and a half years of college. I enjoyed working with networks and Unix immensely and I was always looking to learn more. Needless to say once I got interested in network/Unix security I was no longer looking to learn more, I was learning more. I realized that there was what seemed to be an endless wealth of knowledge one these subjects. There was one catch, however, there was very little in the way of "structured" subject matter. At that time you could not just walk into Borders and pick up a book on differing types of computer related security. At first I was disappointed about this. In time I realized going to a bookstore and buying a book on security does not make one a security expert (not that I consider myself one). I learned security in what most people would consider the hard way. I basically read text files, tried things out on my own, installed BSD and Linux countless times, broke BSD and Linux countless times (At that time I seemed to excel at this), ran tcpdump to understand attacks, tried (and failed) to exploit systems on my college network, and lastly tried (and succeeded) to exploit systems on my college network. The last two steps brought me to some conclusions. One I was lucky I never got caught trying to exploit those systems before I knew what I was doing. Two, I was lucky I never gained access to those systems before I knew what I was doing. Third, I was extremely lucky that once I knew enough to exploit those systems I had the head of the Computer Science program to back me up and allow me to secure the systems I exploited. Fourth, it would have been nice to have somewhere to learn, without the risks of getting caught. Once I graduated from college I got a job at a large tier-1 ISP doing security work. A lot of my co-workers where interested in learning more about security. At the time I had about four machines and a cable modem with one static IP address. I set up the four machines and just basically did port forwarding to my machines. You could login to each machine by hitting a different port. This is really how the idea for pulltheplug.com came about. Obviously the implementation is a bit different now.

LinuxSecurity:Could you explain to our readers what the 'pulltheplug.com' project is about?

Gemberling:I can sum up the goal of pulltheplug.com in one word: learning. That's the entire purpose of pulltheplug.com. I get a chance to learn, others get a chance to learn, etc. I think and hope that it is a win-win situation for everyone that comes and uses my network. The best part about pulltheplug.com in my opinion is no holds barred learning. That's the best there is. As I state on my site, just about anything goes except Denial of Service attacks and anything else that is more or less useless and stupid.

LinuxSecurity:How far do you intend to take this project? What do you gain from doing this?

Gemberling:I intend to continue with pulltheplug.com until it no longer is fun or interesting. Hopefully that will not happen anytime soon. Some friends and I have a few utilities we would like to write, so that will be a next step. As for what I gain from running pulltheplug.com, I get to learn more. Nothing more, nothing less. I do not get any money from pulltheplug.com, in fact line costs and hardware costs come out of my own pocket.

LinuxSecurity:Before putting each system online, what steps do you follow to protect yourself and your liability? Are there known vulnerabilities on your systems left open intentionally? .. or have you secured these systems completely to your knowledge?

Gemberling:Before I put each system online I do the following. Maybe your readers will find these steps useful for their systems.

1.) Install a basic stripped down system. I install only what I need and nothing more. The worst thing you can do is install a system and not know what is on it. I rarely install any daemons from the system distribution. I feel better downloading the source, checking it out, and compiling/installing it myself.

2.) Turn off all unneeded services. If I am ever uncertain if I will need a service, this is usually a good indication I don't need it. Plus if I do need it I can always turn it back on later.

3.) If it is a Linux based system I download the latest and greatest stable kernel release. Then I go to Openwall https://www.openwall.com/ and get Solar Designer's security patches. This patch adds multiple security features to the Linux kernel. I apply the patch and recompile the kernel with only the drivers and features I need and nothing more. Again this is just basic security practice, save applying the Openwall patch, that everyone should do with a system that needs basic to advanced security. If it is not Linux I just recompile the kernel with only the drivers and features I need.

4.) I make sure every package that I have installed is up to date and "secure".

Those are the basic steps I use to implement a new machine into my network. Until null0.pulltheplug.com came up I had never left any holes open on my boxes. I took a different approach on null0, I left a hole open, but there was no publicly available exploit out.

LinuxSecurity:How many of your systems have been compromised, and can you explain step-by-step how each system was compromised?

Gemberling:I've had three of my systems compromised, at least to my knowledge only three have been compromised. The first machines compromised was bassd. It is a FreeBSD box and more or less the box got rooted because I was lazy and slow to apply the patch. The exploit that was used was the /proc vulnerability in the BSD kernel. I saw the post about it on bugtraq and I went outside to run. I got back about 45 minutes later and I saw the exploit on bugtraq. Well needless to say it was too late. About 2 minutes later I got an email with the title, Gotcha! Null0 was the second box to be compromised. This was for lack of a better term, really cool. I put up Null0 and did not allow any local access to the box. I said there was a known hole on the box but there was no public exploit out for it. A fellow figured out it was gdm and within about a week had written his own exploit and compromised the box. The last box that was compromised was roothat. Another fellow gained root access on this box via the recent Linux kernel bug. When he/she compromised the box it was not a known issue and therefore the box was not upgraded to a new kernel, because quite frankly the new kernel was not available. All of these exploits are now public and the code is readily available at PacketStorm Security

LinuxSecurity:What is the average amount of time a person will spend trying to compromise your system and how closely do you monitor their actions?

Gemberling:Typically someone will come on and try every exploit they can find. This takes about 20 minutes or so. Originally I monitored everyone's actions on the servers quite closely. This has become more difficult recently because of the sheer volume of users on the boxes. In just the first twelve days of June there were over 2,000 separate logins on roothat.

LinuxSecurity:What are some of the most common methods people use to find vulnerabilities on your systems?

Gemberling: Searching for SUID binaries and trying prewritten exploits.

LinuxSecurity:What are some of the most creative ways people have tried to compromise your security? How many known attempts have been made?

Gemberling:The most creative way was creating their own exploits. I couldn't even begin to come up with a number of how many attempts have been made.

LinuxSecurity:What do you feel is the most common vulnerability in Linux systems worldwide? What can administrators do to prevent this?

Gemberling:Improper configuration and not keeping up to date with packages/patches. A lot of administrators seem to think just because you put up a Linux box you can walk away from it and it will be secure. This is simply not true. In order to be a good administrator and keep your site secure you have to pay attention. A good administrator will keep up to date with patches, keep up with mailing list such as bugtraq, and just watch their system. The worse thing any administrator can do is just setup a box, put it in the corner and walk away.

LinuxSecurity:What are some of the major pitfalls Linux Administrators fall into? How has the pull the plug project helped you as an administrator?

Gemberling:I'm not an administrator of systems by trade so this question is probably not best answered by me. I honestly do very little administration of the boxes on my network other than the initial setup/configuration and making sure they are running. I could say that this has taught me that a properly setup Unix based system actually takes very little time to administer but I don't think that is true. I don't view what I do as administering my machines, so I've never really seen my actions in that capacity.

LinuxSecurity:What system on your network would you consider to be 'most secure' and why? Can you explain how that ties in to your entire network infrastructure?

Gemberling:I feel is my most secure box is Generic.labs.pulltheplug.com. The reason? OpenBSD, what more is there to say. I know this is a Linux publication but if you are really serious about security, OpenBSD is worth your while to look into. It doesn't have all the latest and greatest features (one of the many reasons it is so secure out of the box) but it is a very capable server.

LinuxSecurity:Have you considered putting up a windows NT/2000 system? Do you feel that open-source software is more secure? (support your answer with why).

Gemberling:I've considered putting up an NT machine, however, I don't have a copy of NT and the licensing is quite expensive. As for open-source being more secure, that's a tricky question. Open-source obviously has strengths over closed-source software for usability and portability. One would think that since a piece of software is open-source and available to all it would be more secure, but this is not always true. Basically it comes down to who finds the security flaws in the software first. Closed-source software has an advantage in this case due to its lack of availability of the source code. This is obviously "security through obscurity" which in my opinion is not the proper approach, but none the less it is a frequent approach. The strength in open-source software is the potential number of programmers that could possibly audit the code in question. Overall I feel open-source is more secure yes, however, making any piece of software secure is a tremendous task. Security depends on the programmer. This is basically the problem; humans are imperfect and therefore so are their end products. Imperfect products obviously are not secure.

LinuxSecurity:How do you feel about the mass-media's portrayal of 'hacking'?

Gemberling:Honestly I don't really pay much attention to mass media, I tend to find current events depressing. From what I have seen it's pretty much typical of what one would expect from the media, exploitation. The media knows the public fears what they do not understand and they wield that fear very well. The situation worsens due to the fact that "Hacking", computers, networks, the Internet, software, and the mix, as a hole is so very complex. I would love for a reporter to give the real story; the one about how if there never were hackers there would not be such technologies as TCP/IP, HTTP, Streaming Video, HTML, packet switched networks, etc. If it were not for hackers we would all probably still be on circuit switched networks using gopher, WAIS, and ANSI for color. It would be nice to see someone point out that "hackers" typically are visionaries with good intentions, not the underground sociopathic individuals they are portrayed to be in the media.

LinuxSecurity:I would like to thank Brian for having this interview. We look forward to seeing future projects! If anyone has any questions reguarding this interview, please feel free to drop us an This email address is being protected from spambots. You need JavaScript enabled to view it.. As always, if you have any ideas for other interviews, or any suggestions, please let us know. We want to serve you!