Java was developed by Sun Microsystems, who also developed a browser plug-in to run Java code on a web page, the Java Runtime Environment (JRE).
Microsoft developed their own Java plug-in, called the Microsoft Virtual Machine (MSVM) and based on JRE. MSVM came installed on all Windows machines. Sun objected to this, and after a legal battle, Microsoft removed MSVM from XP in Service Pack 2 (SP2). New computers now come with JRE, and support for MSVM will end in 2007.
Security
Both JRE and (the now obsolete) MSVM run Java programs (applets) in a sandbox, an isolated part of the computer where they can 'play' without doing any harm by altering system settings, altering or adding files etc.
Building a Better Sandbox java.sun.com
What are the risks?
Security vulnerabilities in the sandbox
A security flaw in the Java sandbox can allow malicious code to access the system outside the sandbox.
A vulnerability was found in MSVM in 2002 which allowed malicious Java code to download a Trojan horse, and Microsoft issued a patch to fix it.
JS.Fortnight viruslist.com
Sun's JRE has also had vulnerabilities:
Beware of new dangerous Java code flaw PC Flank Security News
Sun Stamps on Java Bug vnunet.com
Sun's IE Java plug-in could bug you pcmag.com
Users of pre-XP SP2 Windows systems will need to make sure they have removed MSVM, because any future security vulnerabilities will not be fixed. Anybody wishing to continue using Java will obviously have to install Sun's JRE. A removal tool for MSVM is available here.
Microsoft Java Virtual Machine Support microsoft.com
How will I benefit by upgrading to the Java Virtual Machine from Sun Microsystems? java.com
Users of XP SP2 and users of previous versions of Windows who have switched to Sun's JRE need to make sure that they are running the latest version. Unfortunately, Sun's JRE doesn't auto-update, nor does it notify users that a new version is available. Users may therefore be unaware that they have an insecure application on their computers, which may be leaving them vulnerable to infection by malware. Even if users do download the latest version, older versions remain installed unless intentionally removed, and may again leave the user vulnerable to infection.
Potential Vulnerability with Sun Java auto update broadbandreports.com
CERTs warn about java bug being exploited sans.org
Sun Acknowledges Security Hole in Patch Process Security Fix
To browse safely with Java, download the latest version when updates are announced and always remove older versions from Control Panel > Add/Remove.
Download the latest version here.
Spyware via Java
Although Java runs in a sandbox, a Java applet can request permission to access the system outside the sandbox, for example to read and write files. The user must give permission for the Java applet to access the system. Such applets from legitimate sites will have a security certificate. Giving such permission to untrusted web sites, or where no security certificate is provided is not a good idea, as this could lead to the installation of spyware.
Any browser with the JRE plug-in is vulnerable to an attempt to install spyware by a malicious Java applet attempting to get the user to allow system access, but just such an attempt to install spyware was targeted at the Firefox browser because it does not support ActiveX, the preferred vector of such installations in Internet Explorer.
Please be aware of the possibility of seeing such an attempt to install malware. Do not give Java applets permission to run outside the sandbox unless it is a trusted web site with a valid security certificate. Use an anti-virus program and keep it up to date because Java applets which attempt such an installation will be detected as a Trojan horse.
Spyware via Firefox? It's true Ed Bott's Windows Expertise
What should I do?
The Java plug-in is not an essential addition to any browser. It can be disabled or removed if you think the security risks outweigh the advantages of having it. If you have the Java plug-in, check that it is up to date to avoid security vulnerabilities, and only allow trusted sites to run applets outside the sandbox.
Users of Internet Explorer can add known malicious sites to the Restricted Zone, a black list of sites will then be blocked from running Java (and other active content). Please see the Blocking Bad Sites page to learn how to do this.
Users of Firefox can use the No-Script add-on to block JavaScript and Java except for trusted sites, which are added to a white list.
An example of active content using Java: