A group of computer science PhD students have won Best Student Paper honors at the 2008 IEEE Symposium on Security and Privacy for their paper entitled "Cloaker: Hardware Supported Rootkit Concealment". The paper, authored by PhD students Francis David, Ellick Chan, and Jeffrey Carlyle and professor Roy Campbell, reveals the existence of a new class of malware that is immune to detection by current techniques.
According to the group, rootkits, which are used by malicious attackers who desire to run software on a compromised machine without being detected, have become stealthier over the years as a consequence of the ongoing struggle between attackers and system defenders. In the design of the Cloaker rootkit, the researchers explored how hardware features could be coerced by an attacker to hide malware. The results of their work allow manufacturers to mitigate these attacks before attackers can exploit them in real world devices.
A primary goal for the group was to not alter any part of the host operating system (OS) or programs in order to hide Cloaker from existing rootkit detection techniques. Additionally, Cloaker does not leave any detectable trace in the filesystem and is thus invisible to typical intrusion detection tools that scan filesystems. The result is an extraordinarily stealthy rootkit.
"In essence, Cloaker is a malicious and hidden micro-OS environment that coexists with the existing OS on the device," said the group.
The team built their proof-of-concept on the ARM platform, which powers 90% of mobile handsets shipped today. They focused on the platform because of the vast proliferation of mobile devices compared to PCs and an increasing deployment of ARM-based phones worldwide. They, however, cautioned that similar vulnerabilities may exist on PCs as well.
"We hope that our work motivates future computer system designers to carefully evaluate security gaps at the boundary between computer architecture and system software" said the group. While a possible countermeasure is presented in their paper, the group stressed that significant additional research is required to devise comprehensive defenses against such malware.
The group's research is funded by Motorola and DoCoMo. More information and the complete paper can be found at Systems Software Research Group Security Projects.