Identifying Native Applications with High Assurance

Almohri, Hussain M. J. and Yao, Danfeng and Kafura, Denis (2011) Identifying Native Applications with High Assurance. Technical Report TR-11-20, Computer Science, Virginia Tech.

Abstract

The work described in this paper investigates the problem of identifying and deterring stealthy malicious processes on a host. We point out the lack of strong application iden- tication in main stream operating systems. We solve the application identication problem by proposing a novel iden- tication model in which user-level applications are required to present identication proofs at run time to be authenti- cated by the kernel using an embedded secret key. The se- cret key of an application is registered with a trusted kernel using a key registrar and is used to uniquely authenticate and authorize the application. We present a protocol for secure authentication of applications. Additionally, we de- velop a system call monitoring architecture that uses our model to verify the identity of applications when making critical system calls. Our system call monitoring can be integrated with existing policy specication frameworks to enforce application-level access rights. We implement and evaluate a prototype of our monitoring architecture in Linux as device drivers with nearly no modication of the ker- nel. The results from our extensive performance evaluation shows that our prototype incurs low overhead, indicating the feasibility of our model.