Earlier this month, a public server of the Apache Software Foundation (ASF) was illegally accessed by unknown crackers. The intrusion into this server, which handles the public mail lists, web services, and the source code repositories of all ASF projects was quickly discovered, and the server immediately taken offline. . . .
Earlier this month, a public server of the Apache Software Foundation (ASF) was illegally accessed by unknown crackers. The intrusion into this server, which handles the public mail lists, web services, and the source code repositories of all ASF projects was quickly discovered, and the server immediately taken offline. Security specialists and administrators determined the extent of the intrusion, repaired the damage, and brought the server back into public service.

The public server that was affected by the incident serves as a source code repository as well as the main distribution server for binary release of ASF software. There is no evidence that any source or binary code was affected by the intrusion, and the integrity of all binary versions of ASF software has been explicitly verified. This includes the industry-leading Apache web server.

Specifically: on May 17th, an Apache developer with a sourceforge.net account logged into a shell account at SourceForge, and then logged from there into his account at apache.org. The ssh client at SourceForge had been compromised to log outgoing names and passwords, so the cracker was thus able get a shell on apache.org. After unsuccessfully attempting to get elevated privileges using an old installation of Bugzilla on apache.org, the cracker used a weakness in the ssh daemon (OpenSSH 2.2) to gain root privileges. Once root, s/he replaced our ssh client and server with versions designed to log names and passwords. When they did this replacement, the nightly automated security audits caught the change, as well as a few other trojaned executables the cracker had left behind. Once we discovered the compromise, we shut down ssh entirely, and through the serial console performed an exhaustive audit of the system. Once a fresh copy of the operating system was installed, backdoors removed, and passwords zeroed out, ssh and commit access was re-enabled. After this, an exhaustive audit of all Apache source code and binary distributions was performed.

The ASF is working closely with other organizations as the investigation continues, specifically examining the link to other intrusion(s), such as that at SourceForge (https://sourceforge.net/) [ and php.net (https://www.php.net/search.php ]

Through an extra verification step available to the ASF, the integrity of all source code repositories is being individually verified by developers. This is possible because ASF source code is distributed under an open-source license, and the source code is publicly and freely available. Therefore, the ASF repositories are being compared against the thousands of copies that have been distributed around the globe. While it was quickly determined that the source code repositories on the ASF server were untouched by the intruders, this extra verification step provides additional assurance that no damage was done.

As of Tuesday, May 29, most of the repository has been checked, and as expected, no problems have been found. A list of verified modules will be maintained, and is available here:

Because of the possible link of the ASF server intrusion to other computer security incidents, the investigation is ongoing. When complete, the ASF will offer a complete and public report.

The Apache Software Foundation strongly condemns this illegal intrusion, and is evaluating all options, including prosecution of the individual(s) responsible to the fullest extent of the law. Anyone with pertinent information relating to this or other related events should contact root@apache.org. Anyone from the media with further interest should contact press@apache.org.

Thanks.

Brian Behlendorf
President, Apache Software Foundation