LinuxSecurity.com:  How did you get started with security? What were some of the bigger challenges you faced as you tried to be one step ahead of your black-hat adversaries?

Marcus Ranum:  I got started when I was working in sales support for Digitial Equipment Corp, and my boss made me take over management of one of our corporate Internet gateways. It wasn't really a firewall, in those days, but over the course of a couple years I evolved it into a pretty tough firewall which later became a commercial product (the DEC SEAL). My interest was always in building large-scale distributed systems and doing network management. Security was just something that snagged me, sucked me in, dragged me under, and has never let me go.

To me the biggest challenge has always been keeping up with what the bad guys are doing while keeping my hands clean. There are a lot of security folks who play on both sides of the fence; their excuse is that they need to do that to learn what the enemy is doing. In reality, I think that's just a pose they adopt that lets them have the benefits of being a hacker without the downside of getting in trouble if they get caught. I've tried all my career to be living proof that you can be a security professional without having a background as a "black hat" or "gray hat" hacker but it still boils my blood that I get 2 or 3 e-mails a week from hotmail.com addresses asking for hints how to break through firewalls, etc. It's very disappointing, especially now that some conferences (SANS, Interop, etc.) are teaching "how to hack" classes and promoting hacking as something that's fun and cool.

LinuxSecurity.com:  Can we start with having you explain what an intrusion detection system actually is, and a mention of the various types? What is the difference between misuse detection and anomaly detection? Host-based and network-based?

Marcus Ranum:  An intrusion detection system is a security system designed to detect unauthorized accesses (or suspicious activity) within a system or a network. Host-based intrusion detection systems tend to focus on what's happening within the host itself. Network-based intrusion detection systems generally operate at an IP level, trying to infer attacks against the network from traffic and its contents. The host-based approach tends to focus on logs, application states, and kernel information for its data sources, while the network-based approach tends to focus on packets. Of course, there is always some crossover: some network-based systems look for host problems, and some host-based intrusiond detection systems latch the bottom of the host's IP stack and look at packets.

Anomaly detection and misuse detection are the two primary approaches for analyzing the data the intrusion detection system deals with. In the misuse detection approach, the intrusion detection system has a knowledge base of "signatures" that represent known attack patterns. The system matches events against the signature database, as it sees them. This is a very predictable approach - if your knowledge base is good, and your pattern matching ability is good, then you will reliably detect known problems. On the other hand, if you don't know what a particular attack looks like, you can't detect it. So misuse detection systems may miss a new attack; they're much like antiviral software in that regard. Just like antiviral programs, misuse detection systems are easy to deploy, and very effective. Anomaly detection systems are based on statistical analysis - they try to determine what looks "unusual" based on the types of events seen in the network beforehand. Anomaly detection approaches often emphasize training, neural networks, statistical margins of error, etc. This approach is less predictable than misuse detection, since it depends on learning what's going on - and the system makes "inferences" about the significance of events. One of the biggest problems with statistical methods is that they can tell you if something is "unusual" but they can't tell you what it means. That's left as an exercise for the user. For that reason alone, many network administrators find misuse detection to be more valuable: it tells you what things mean because it's got that knowledge base to compare against.

In real life, most intrusion detection systems use both anomaly detection and misuse detection, since neither approach is perfect in itself - you can get pretty good coverage by trying a bit of everything. I've been predicting for a long time that eventually the host-based versus network-based paradigm will break down as well: there are some things that each type of system is especially good at, and an intrusion detection system designer would have to be a fool to ignore that.

LinuxSecurity.com:  The phrase "defense in depth" describes a method of providing multiple layers of security to a system in an effort to reduce the risk of compromise should any one of those layers become subverted. My experiences have been that many of those not familiar with this concept, or have been deluded by marketspeak, think especially that an IDS is a panacea. Once it's implemented, it may be the case that a guy that previously worked in the tech support group is the one responsible for monitoring the GUI-based point-and-click front-end and calls someone when he thinks there's a problem. Can you explain what role an IDS should play in an organization?

Marcus Ranum:  I hate the picture you portray: of an organization that is deluded by marketing and which doesn't take security seriously enough that their staff have the time and wherewithal to understand and pay attention to what's going on in the network. I know that there are a lot of sites out there like that, but it's awfully depressing to contemplate.

Where would intrusion detection fit in an organization? Well, if they're concerned about security and want to do things right, they'll be monitoring traffic for unusual signs on the interior of their firewall, and on critical interior segments. Mission critical machines will also be running intrusion detection. Typically, all of the intrusion detection systems will report to a central console that is staffed by someone who can make the first approximation of what's significant and who will have the ability to call in the troops if something is clearly amiss. This means that the organization will have to have an incident response plan, a call tree for emergencies, and sufficient training to know whether or not a particular attack represents, as Captain Nemo says, "an accident or an incident."

LinuxSecurity.com:  What are the hurdles that must currently be overcome with today's IDS boxes? What are some of the new features coming in the next revisions of IDS's?

Marcus Ranum:  The main hurdles have to do with reducing the number of false positives (false alarms) while successfully handling greater and greater data rates. Some of our customers are running sustained throughputs of 500+mb/sec - it's hard to collect that kind of traffic and examine it closely. So one of the new features I think most IDS will have is tight coupling between network-oriented traffic capture and host-based security analysis.

LinuxSecurity.com: How and why did you design the NFR operating system as a derivative of OpenBSD, the BSD distribution reportedly developed explicitly with security as a focus?

Marcus Ranum: We really wanted to move away from operating systems entirely, but it's difficult. To us the choice of OpenBSD was one of convenience - one of my staff has a good relationship with some key members of the OpenBSD team so we just naturally gravitated in that direction. But in today's world, you can't assume that things will stay the same: we use virtually none of OpenBSD in our appliance - only the bootstrap loader, some device drivers, and the kernel/file system. We wanted to be able to swap operating systems any time. We think we could replace the operating system in a matter of a month or so, if we had to.

LinuxSecurity.com:  I understand you have a few sound reasons why your product doesn't run using Linux as a base. Can you explain why that is, and why you're only now revisiting Linux? Will we have IDS administrative console access soon on Linux?

Marcus Ranum:  The main reason is that we don't really care about operating systems; since our product comes with its own operating system built into it, we're very happy in the operating system department. There's no reason to switch or support another operating system other than if we wanted to be trendy. Since our users have no access to operating system features, they can't tell if it's BSD or Linux or whatever - they shouldn't care. Of course, you always run into die-hards who are disgruntled when you're not supporting whatever it is they particularly like. From our perspective, that's like complaining that your car's microcontrollers aren't running Linux. Who cares what they're running?

We've got command line tools for accessing the IDA from UNIX or Windows machine; they work fine under Linux. We keep exploring the idea of writing an X-Window based GUI, but our experience is that UNIX users are happy with command line, and Windows users are happy with GUI only. So we think things are pretty well covered in that department.

LinuxSecurity.com: What do you think about the concept of having the IDS box interact directly with the firewall, and provide the ability to block off the offending address as it's happening? Obviously this could cause a denial of service in and of itself, but is this type of proactive measure being worked on?

Marcus Ranum:  That kind of thing has always made me nervous, but our customers ask for it all the time. I still believe that passive approaches are the only ones that are truly viable (interfacing with a router/firewall is "passive" while sending reset packets and network unreachables, etc, is "active"). The trend I believe we'll see in the future is centralizing of information into places where a human can react/manage the process as an intelligent participant. That's going to be an interesting and active area of research in the future, I predict.

LinuxSecurity.com:  What do you think of the idea of incorporating the intrusion detection software directly into the network router, instead of a dedicated device for this purpose, such as NFR?

Marcus Ranum:  I think it's a neat idea in principle but it doesn't and won't work very well in practice. The reasons are a little subtle, but let me explain a few of them: To do intrusion detection "right" at a network layer, you have to do TCP stream reassembly. The fact that many of the commercial IDS out there don't do it now is an embarrassing little secret they'd like to keep, but it's really important since the bad guys know lots of ways around IDS that don't reassemble streams. But stream reassembly is hard (which is why some of the products out there don't do it) and it's also CPU and memory intensive. Router makers are extremely sensitive to things that are going to require extra RAM or a bigger CPU, so they've got a problem if you want to push IDS into that platform. My guess is that the IDS capabilities will be minimal (mostly for marketing purposes) or you'll see something that is really an IDS that also happens to know how to route - and it'll be a lot slower than a "real" router or switch. Let me give you an example: if you're trying to capture and reassemble an 80% saturated FDDI, you're looking at about 17,000 packets/sec - they won't all come in sequence so you're going to have to buffer packets. You can eat 100MB of RAM in no time at all and I don't see a lot of routers with that kind of spare real estate. Another problem with the idea of putting an IDS into a router or switch is log retention. IDS like to keep records of what they saw; hard disks are the only reliable way to do that. I don't see a lot of routers with ultra-wide SCSI disks in them. If I did, I'd be building IDS on them! :)

LinuxSecurity.com: I understand you no longer make the source code available to your product. Wouldn't it be possible to release the source code under an Academic Source License, as well as provide a binary-only commercial evaluation copy for those who are interested in purchasing it?

Marcus Ranum: We actually have (under limited circumstances) made source available for researchers in the community, gratis. The whole issue is painful for me because I come from an "open source" background (as a friend says, "I was 'open source' when 'open source wasn't cool.') but I've had too much intellectual property ripped off. There's already one product on the market that I believe is highly derivative of NFR - some chump's going to make a lot of money on that product, without so much as a "thank you" to the real innovators. I know a lot of people think I'm "anti open source" (which is silly, since I was posting source code before most of them had even heard of the Internet) but really I've had some nasty experiences with rip-offs that has soured me on the concept. There are a couple of guys who made tens of millions of dollars by ripping off the firewall toolkit (another of my 'open source' products) - after a while it gets irritating.

Another problem I had with making source code available was that we had a lot of people who refused to read the README files, and ate a ton of our time with questions that we'd already expended considerable effort to answer. That made it hard for us to get work done! To top it off, people would ignore the README that explained that our commercial product was a whole different (and much more well-rounded!) solution, and they'd compare our source code "do it yourself" toolkit against our competitors' commercial offerings. I hate to say it, but being 'open source' hurt us really badly in a number of ways. So we're careful now.

LinuxSecurity.com:  I read your new column in the last issue of ;login, the USENIX and SAGE magazine, where you talked about how protocols and technologies are becoming more and more complex, yet seem to have increasingly less security, as a tradeoff for ease-of-use. Set-top boxes, xDSL, and cable modems are all adding to this fertile environment for attackers. What do you think our future is like? How can we design secure systems, yet make them easy enough for the uninitiated to use?

Marcus Ranum:  I don't think so. I think things are going to get infinitely worse before they get better (if they ever do) There's just too much shovelware out there. I saw a great example a few months ago: a web-cam that's a completely "hands off" appliance. Turns out it runs Linux inside, and a web server and a little FTP server. The web server has buffer overruns and the FTP server allows FTP bounce attacks. Of course the guys who built it were trying to make a camera, not a secure system, but I wonder how many of those are sitting out on people's DMZ networks...

LinuxSecurity.com: It's been said that the only way we are going to resolve the distributed denial of service is for upstream providers to perform source address IP spoof protection, as well as diligent, educated administrators to do their part in preventing their systems from becoming compromised. In the bandwidth consumption attacks such as smurf, it's often difficult to determine legitimate from illegitimate traffic. What role do you see intrusion detection systems playing in the prevention and reporting of on-going attacks?

I believe that one of the important roles IDS will fill is by increasing the level of accountability on the Internet. We've already had customers catch hackers that were using their sites as jump-off points. Consistently, the hackers were surprised to find out that someone was watching who did what, and when. Well, it's unfortunate, but I think the days of anonymous Internet use are numbered.

LinuxSecurity.com: Many of today's commercial firewall products include hooks to perform virus scanning, LDAP authentication and other "conveniences" that certainly seem to be more than a firewall should be responsible for. This ultimately resulted in a buffer overflow, of all things, on a firewall recently reported to bugtraq. Do you think these add-ons provide value-add to the consumer, or are an inexcusable attempt by the firewall vendors to create "the only security device you'll ever need"? What do you think the eventual polymorphism of firewalls will be in the next few years?

Marcus Ranum: I think that was one of my old products. :( The hole was added after I'd left over managing the product, though. I tend to be extremely fussy about having anyone else's code on my products, for exactly that reason. I don't believe security can be built piecemeal through acquisition: it has to be consistently designed, by someone who is familiar with the system and who is interested in making the most secure product, not the most feature-rich.

Firewalls are an interesting problem. There are some fundamental problems with the entire firewall concept which, fortunately, the bad guys have not taken advantage. I believe that within the next few years, they will. Then I'm not sure what happens to the idea of firewalls. It bugs the hell out of me since I don't have a solution, either. It's one reason I got out of the firewall business back in '95 when I did.

Marcus, I'd like to thank you for having this interview. We all very much look forward to seeing the projects you're working for the future, and hope to see you at the next computer security conference!