LinuxSecurity.com: Please explain again for our readers what Pull the Plug is about. What is the concept? How does it work? Who can participate?

PullthePlug.org: The concept of PullThePlug has always been to provide an arena for like minded individuals to discuss, train, and learn about computer security and associated technologies.

The primary focus of PullThePlug as a community is to deliver information and resources on computer security to a wide range of audiences. Some services we currently offer are war-game machines (vortex, semtex, catalyst, blackhole), mailing lists, IRC channels, and live lectures (http://www.pulltheplug.org/about/suntzu/) and repository/web hosting for research efforts (http://www.pulltheplug.org/about/rcs/)..

As a result of PullThePlug being community driven (by the community for the community), anybody can participate in some way or another. More often then not, new talents are seen when participating in our wargames or contributing to mailing lists, and people are also free to join the IRC and discuss any topic of interest, or provide ideas or services which help in furthering the community driven learning experience.

LinuxSecurity.com: Daniel, how did you get involved with Pull the Plug? What is your current role with the site?

PullthePlug.org: I first became interested in PullThePlug in 2001 when a co-worker showed it to me. Eager to learn about network security, I visited the site frequently, reading documentation, and playing war-games. Near the end of that year the organization was running short on resources and the servers being used to run the war-games were shut down. By 2003 I became really involved in the project when I helped create the first new war-game since the last ones were shut down (vortex.labs.pulltheplug.org). A friend, Kurtis Meyers, and I donated a server to run the new war-game and Andrew G. Administered it. The initial founder of PullThePlug, Brian Gemberling, was happy to rack our server and provide the necessary bandwidth. The new war-game led to a large increase in traffic and more interest than Brian could manage by himself, so Brian gave me the responsibility of handling the day to day management of PullThePlugs resources. Since then we have continued to increase our traffic and interest quite a bit. A management team has been created to organize PullThePlug. This group includes Andrew Griffiths, Samy Al Bahra, Daniel Hudson, and myself. Together we make all of the decisions and work allocations related to PullThePlug.

LinuxSecurity.com: What happened to Brian Gemberling (founder)? Is he still involved with the project?

PullthePlug.org: Brian keeps himself busy with his newly made business PullThePlug Technologies LLC (http://www.ptptech.com), located in Aushburn, Virginia. His business offers secure collocation, rack space, and a variety of Internet services with an emphasis on security. Initially PTPTECH only offered services to private parties, but on June 13 his services became available to the public. He provides bandwidth and rack space forour servers. Brian is no longer involved in the everyday operation of PullThePlug, However, he still donates bandwidth, rack space, and time.

LinuxSecurity.com: How has the project changed since our original interview? (June 26th 2000) How much has it grown? How many people are now involved, and how many hosts do you currently maintain?

PullthePlug.org:

• The management of PullThePlug has changed hands from Brian to a four person management team created from outstanding community members. Other people who are not a member of the management team still take part in many of the administrative services such as managing the IRC chat rooms and the war-games.
  
  
• We moved from PullThePlug.com to PullThePlug.org since PullThePlug is on it's way to becoming a non-profit. PullThePlug.com is now part of PullThePlug Technologies owned and operated by the founder of PullThePlug, Brian.
  
  
• The staff has changed a lot. Many of the old crew wanted PullThePlug to remain private, while others wanted to grow and acquire/provide new resources to the public. Many people left as PullThePlug got too big for their tastes.
  
  
• The old war-games are gone and a whole new breed of them are up, including vortex, semtex, catalyst and blackhole.
  
  
  • Vortex resembles mainsource which is a level based wargame focusing on learning security concepts such as buffer overflows, format strings and some encryption stuff.
    
    
  • Blackhole is also level based and focuses on remote exploitation of overflows, format strings etc.
    
    
  • Semtex is much more "Down to earth" it doesn't focus on vulnerabilities - instead - it's purpose is to allow players to hone their network programming skills.
    
    
  • Catalyst is for those looking to play around with binaries and hone their "binary analysis" skills. Technology has changed a lot in 4 years and we try to keep up with all the "latest and greatest".
    
    
• We've also pioneered new things like Live Tutorials (http://www.pulltheplug.org/about/suntzu/). Basically, people can choose a topic to 'lecture' on and choose a medium such as irc, silc, voip or even teleconferences and physical meetings. Listeners can login to suntzu and see what's being explained real time. Allowing for the observer to actually see with his/her own eyes what's being discussed.
  
  
• We also have a Development machine which provides SVN/CVS services to various projects. Some of the projects we host include kerneled (http://www.kerneled.org, home to many popular FreeBSD ports and various software patches), which includes quite a subset of software and other various private projects.

Our size: Currently we have 4 "master" (physical) machines and over 8 virtual servers. We also host ruxcon.org.au (http://www.ruxcon.org.au) which is a security conference held in Australia.

How many People are involved now? 4 people in management team about 8 total people just helping out Including wargame administrators like "aton" - who runs semtex.labs.pulltheplug.org and Ken Davies who helps us out whenever our servers go down by going to the datacenter and fixing stuff. Both of which have been with us for quite sometime.

We receive 250 300 visitors to our site per day on average. As well as an average of 80-90 people on our IRCD and over 60 people on our mailing list.

LinuxSecurity.com: How often are your systems compromised? What have you learned from the process? How has it benefited your skill set personally?

PullthePlug.org: Oddly enough PullThePlug does not receive an excessive amount of hacking attempts, but we have experienced several Denial of Service (DoS) attacks against the wargame machines, and other services we provide (such as the live lectures). The management team have always been swift in their response to these incidents. There has been no known successful compromise of the PullThePlug network. We believe our war-games provide a unique challenge to the security community, and thus much more challenging than a simple dotslash.

As a learning curve, we have realised the benefits in network monitoring, securing systems, patch management, and other such day-to-day administrative activities. This has taught most of the staff how to look for and identify interesting event patters (most of the data on the PullThePlug network is logged and managed remotely), in addition we use complex filters on the upstream router to block out traffic to hosts which we deem sensitive. We utilize virtual servers extensively as well. This creates an environment that minimizes possible exposure to the rest of the systems and also segregates "trouble" machines, effectively cutting off any chance of total compromise. We also use grsecurity kernel patches (http://www.grsecurity.com). Not only from a security perspective, but administering the network has always provided a unique challenge to staff and as such is constantly teaching us new things.

LinuxSecurity.com: Although everyone who attempts to compromise a machine uses different techniques, have you noticed any common patters (methods) that are used across the board? Please describe the anatomy of a typical attack.

PullthePlug.org: We simply don't leave machines open to attack - instead we close off the machines and leave 'conduits' for attacking, which are levels.

An attacker must then work their way up through the levels with increasing difficulty. This provides a unique challenge that turns out to be very rewarding in the end.

Another benefit from doing this method is that if people are unfamilar with some aspect, they'll need to learn it before progressing, which encourages people and exposes them to new stuff.

One interesting effect of the level based wargames we provide is that people are constantly suprising us with new and innovative ways to approach certain levels. With semtex, one user submitted an solution developed with Microsoft Excel, while another user has reverse engineered linux binaries (on catalyst) under the Windows platform.

Typically the most common approach to the wargames is the most obvious, and people will compromise the levels via standard stack smashing techniques, format strings, heap exploitation and so on. Though I state the "standard" techniques are used, there is no definative approach people are taking. This is primarily due to the challenges being different to all other wargames we have seen - they all provide the opportunity for exploitation, but there is always that slight twist to make it all the more interesting, challenging and rewarding.

As for the PullThePlug network being attacked, we often see portscan attempts followed by brute forcing - occasionally an exploit against a service we don't provide (which we usually consider to be worm traffic). If we were to class the most common attack scenarious, it would probably be due to worm traffic, and involves probes against particular port's (to determine whether or not a service is provided), followed by multiple malicious payloads sent to that service.

LinuxSecurity.com: After being involved in this project, what have you learned to be the single most important step in keeping a linux/unix system secure?

PullthePlug.org: The single most important step is trusting in the abilities of the people who are protecting your assets. If you cannot trust them. Then you cannot Trust the security of your systems and networks.

For small environments where they don't have the funds available to do a any serious security stuff, their most important step would be ensuring machines are kept up to date, along with anti-virus signatures, and perhaps some basic end-user training.

In larger environments, you'll need to have skilled administrators who know their field inside out, who will keep abreast of security issues, will look at and examine methods of improving the security of the systems, and hopefully designing away various security issues.

In huge environments, you'll generally have duty seperation, and teams of people handling various facets, such as people who write policies, the people who implement them, the people response for monitoring the security of systems, and so fourth. In this case, its nessesarcy that people work together on achieving the required level of security.

Problems will generally be approached by doing a risk analysis and attemtping to remove or mitigate high risk / high impact and working down. To solve the problems though, you'll need to have the appropriately skilled people with the backing of the company.

To bring this back to pulltheplug, a lot of the stuff we do involves minimizing exposure while trying to make the appropriate systems accessible by people. A example of minimising exposure would be seperating various services we provide from people's shell accounts, and only providing the files needed to make that service work as expected.

LinuxSecurity.com: Pull the plug is a slightly different concept from a honeynet. While the goals are similar, are the results different? Explain the advantages of operating openly as opposed to covertly like the administrator of a honeynet would.

PullthePlug.org: Pulltheplug is pretty much completely different from a honeynet. We aim to help people understand applied security concepts, rather than setting up boxes for random people to compromise.

We do get the joy of observing some of the more interesting exploit's against challenges when people wish to tell us about them, but there is a significant difference between that and a honeypot. We differ not only in terms of goals, but also strategy. The games are not setup to observe peoples actions, and are not setup as bait to understand new exploit strategies.

All levels are generally left un-moderated, which allows participants to choose whether or not to share information with the rest of the community (this could be an exploit technique, or idea's for new challenges etc).

Because participants have this freedom, it also builds a strong level of trust within the community, and provides people with a safe zone to experiment and broaden their ideas without penalty.

Community members and new comers alike - see our community as a place to share ideas without the ego's that plague many other communities. Some say we are the next best thing before being a totally private community.

LinuxSecurity.com: For those readers interested in system monitoring, what open source tools would you recommend? Would you mind providing the names, a short description, and the URLs to several of your favorite host and network monitoring tools?

PullthePlug.org: These are tools we recomend overall.

• grsecurity http://www.grsecurity.net - grsecurity is a kernel patch which provides a comphrensive approach to increasing the security of a system. grsecurity provides detection, prevention, and containment, which is useful on a couple of the systems Pulltheplug runs.
  
  
• openwall kernel patch http://www.openwall.com - The openwall kernel patch allows us to provide an increased level of security that isn't as extreme as grsecurity. This is used to allow people to learn such things as bypassing non-executable stacks for example.
  
  
• syslog-ng http://www.balabit.com/products/syslog_ng/ - Secure replacement to syslog. We utilize syslog-ng to monitor our network and facilitate remote storage of logs.
  
  
• stunnel http://www.stunnel.org/ - We utilize stunnel to provide secure encrypted means of transporting logs and other streamed data across the network and internet.
  
  
• Linux VServer Project http://linux-vserver.org/ - Linux Virtual Servers provides the means for complete segregation of server processes allowing us to minimize exposure in the event of a successful attack. They also allow us to extend the value of our limited resources by running several modularized Linux Distributions under the same linux kernel.
  
  
• TrustedBSD Security Extensions http://www.trustedbsd.org/ - "The TrustedBSD project provides a set of trusted operating system extensions to the FreeBSD operating system, targeting the Common Criteria for Information Technology Security Evaluation (CC)." We utilize many of the extensions on our development hosting server.

LinuxSecurity.com: With so many systems to watch, I'm sure managing logs could be quite difficult. How do you cope with this? What automated tools are you using? How long are the logs retained? How can you apply what you've learned about log management in this project to a business/enterprise/university network?

PullthePlug.org: By utilizing features in syslog-ng we are able to log to both flat files and a mysql database. Currently our central log server has space for well over 2 years of logs. log files are seperated by year, month and day. Monthly our SQL tables are rolled over for quick searching later. In addition to default logging facilities, we also log usage of various system calls.

Virtual Servers are logged over the network via syslog-ng using pipes and stunnel. Stunnel is configured to use keys for verification of hosts and encryption of traffic. Swatch is used to generate alerts that match various regexp patterns found in logs.

Many of the procedures we use on pulltheplug's network are used on many other networks both private and public.

LinuxSecurity.com: What vision do you have for the future of pull the plug? If money wasn't an issue, how would you evolve the network? What immediate goals (1 year) do you have for pull the plug? What long term goals do you have?

PullthePlug.org:

Long Term:

• provide more challenges for people, and get more members of the community involved with setting up and running games.
  
  
• provide capture-the-flag (CTF) type games for people so they can practice more offensive skills.
  
  
• setup a relatively large network and try to emulate certain environments so that people can try to compromise them, and additionally, allow people to secure those networks, so we can have a game of cat and mouse amongst some people.
  
  
• Participate more activley in security conferences - perhaps doing suntzu tutorials.
  
  
• Create and maintain a compile farm comprised of various platforms and Operating Systems for developers.
  
  
• Eventually file for non-profit status with the IRS. So that we are able to recieve tax deductable donations.

  Short Term:

• Create and Present more Suntzu tutorials. Get more people involved in sharing what they know.
  
  
• Provide more resources for development projects. Such as the current hosting on karissa.pulltheplug.org.
  
  
• Upgrade current wargame machines - so that we may provide more levels and new types of games.

LinuxSecurity.com: Any final words for the readers at LinuxSecurity.com?

PullthePlug.org: For those of you who are looking for a security challenge, looking to learn something or enjoy technical chat visit us.

( http://www.pulltheplug.org ) If your interested in contributing in some way please dont hesitate to drop us an email. people@pulltheplug.org

We'd like to thank Benjamin Thomas and LinuxSecurity.com for this opportunity. Along with everyone here at pulltheplug dot org who keeps the community chugging along! (Mercy, Andrewg, Samy, Aton, Steven, Nemo... and the rest)

Special Thanks:

Ken Davies - Thanks for being our remote hands.
David King - Thanks for your input.

Only registered users can write comments.
Please login or register.

Powered by AkoComment!