Changes to the Java security model
The upcoming security changes in Oracle Java address three long-standing issues with the Java security model.
The most significant change is how signed applets are handled. In the past Oracle has suggested that all websites switch to signed applets, advice that contradicts recommendations by security experts, because signing an applet would also confer privileges to escape the sandbox. In fact, signed applets are the original method of escaping the Java sandbox, and have been abused by both attackers and security auditors for the last decade. Metasploit has a module specifically for this purpose.
Oracle is changing this model so that signing an applet no longer confers sandbox escape privileges. This is a good thing for security.
The second change has to do with whether unsigned applets are allowed to run. In recent versions of Java, unsigned applets required additional steps on behalf of the user to run. This change will make that even more cumbersome and push developers to always sign their applets, something many were loath to do with the existing security model. This change also allows the whitelisting of specific web sites and central management of Java security policies, something that has been a significant problem for enterprises so far.
The last change relates to certificate validation. In order to verify that an applet has a valid signature, Java needs to walk the certificate chain, making sure that it ends in a trusted root. This works fine until a certificate has been revoked due to a compromise.
In the current security model, Java will not verify the revocation status of each certificate, making it easy for an attacker with a stolen certificate to continue using it after the attack has been discovered. This change will force revocation checks using published CRL and the Online Certificate Status Protocol (OCSP), incurring a performance hit, but preventing this type of attack in the future.
Taken as a whole, this is good thing for Java, but these changes don’t solve the underlying problem with the Java sandbox itself. Until Oracle implements process-level sandboxing, such as that used by Adobe Reader and Google Chrome, a malicious applet with a valid signature can still abuse JRE security flaws to escape the sandbox and compromise the system. Obtaining a code signing certificate has not been a barrier for malware in the past and there is little chance it will become one in the future.
The introduction of OCSP validation has an interesting side effect as well – an attacker can sign an applet with a custom certificate pointing to a OCSP server they host. While this won’t allow them to spoof the validity of the certificate, it will tell them whether the applet was loaded, even if it fails the check. This type of information leak is present in most OCSP-enabled applications, and has been used in other file formats to track user actions (PDF, etc).
Author: HD Moore, Chief Research Officer at Rapid7.