NETWORK TESTING LABS REVIEW
Endpoint Security Performance
Performance Report Card
Grade scale is A through F, with F = Failing and A = Perfect
The first column shows the report card category and its weighting.
|
|
McAfee Internet Security 2009
|
McAfee Total Protection 2009
|
Norton Internet Security 2009
|
Norton 360 V2.0
|
Kaspersky Internet Security 2009
|
Trend Micro Internet Security 2009
|
Trend Micro Internet Security Pro V.2
|
Microsoft Windows Live OneCare 2.0
|
|
CPU (40%)
|
A –
|
A –
|
B
|
C
|
B
|
C
|
C
|
C
|
|
Memory (20%)
|
B
|
B
|
C
|
C
|
C
|
C
|
D
|
D
|
|
Hard Disk (20%)
|
A
|
A
|
A
|
C
|
B
|
C
|
C
|
C
|
|
Network (10%)
|
A
|
A
|
B
|
D
|
C
|
C
|
B
|
B
|
|
Video (10%)
|
A
|
A
|
C
|
C
|
C
|
C
|
D
|
D
|
|
Overall score
|
A –
|
A –
|
B –
|
C –
|
C
|
C
|
D +
|
D +
|
Test bed and methodology
We evaluated each endpoint security product’s computing resource usage in a variety of situations and on a variety of platforms.
For these tests, our lab's computing environment consisted of Windows XP and Vista running on three machines. A Dell Latitude D505, with 512 MB RAM, a 1.50 GHz Intel CPU and 30 Gb hard drive ran Windows XP, while Vista ran on both an HP Pavilion DV9000 with Intel Core2 T5300 1.73 GHz, 2,046 MB RAM and 160 Gb hard drive as well as a Dell Optiplex 755 with Intel Core 2 Quad Core 3.16GHz, 4 Gb RAM and 320 GB hard drive.
On all these platforms, we ensured that unnecessary Windows Services, such as Microsoft Indexing, were disabled. We ensured that all platforms used the same video resolution, and we also ensured that each computer had a 100 Mb/sec network connection. We used a 512 kb/sec Frame Relay link to connect to the Internet.
In the performance testing, on each of the three platforms, we measured installation time, system startup time, system shutdown time, time to hibernate, resumption from hibernation time, time to launch Internet Explorer in order to load Download.com’s initial Web page, time to open a PowerPoint file, time to open a PDF file, time to open an MP3 file, time to convert WAV files to MP3 format, process count, memory usage at idle, memory usage during hard disk scanning, time to send 250 email notes, time to receive 250 email notes, time to fully scan the hard disk for malware, resource consumption at idle and during a full scan, as reported by PC Mark Vantage, and video frames per second achieved while a security product was idle and while it was scanning the hard disk.
The following table provides more detail on the battery of tests we employed.
|
No.
|
Test
|
Description
|
|
1
|
System startup
|
Script-measured time until the CPU utilization falls below 10% for 5 seconds
|
|
2
|
System shutdown
|
Stopwatch-measured time from selecting the shutdown option until the computer's power indicator dims
|
|
3
|
Time to hibernate
|
Sleep duration plus HiberWrite duration
|
|
4
|
Resume from hibernation
|
Wake duration plus HiberRead duration
|
|
5
|
HD full scan time (min:sec)
|
Time to to perform a full scan of the computer’s hard disk
|
|
6
|
PC Mark score at idle
|
Score obtained by running PC Mark Vantage with the computer idling (NOTE: PC Mark only runs on Vista)
|
|
7
|
PC Mark score during scan
|
Score obtained by running PC Mark Vantage while a full scan of the computer's hard disk was underway (NOTE: PC Mark only runs on Vista)
|
|
8
|
Frames/sec during game
|
As reported by the FRAPS utility -- game running, but no other activity
|
|
9
|
Frames/sec gaming & scanning
|
As reported by the FRAPS utility -- game running and full scan of the computer's hard disk underway
|
|
10
|
Launch I E (Download.com)
|
Script-measured time from Internet Explorer's launching to the moment I E finishes displaying the entire Web page
|
|
11
|
Convert WAV to MP3
|
Script-measured time Apple's iTunes V6 software took to convert 710 MB of WAV files to 192kbps MP3 format
|
|
12
|
Open PDF file
|
Script-measured time to open a 3.7 MB Acrobat PDF file
|
|
13
|
Open MP3 file
|
Script-measured time to open a 1.2 MB MP3 file in Media Player
|
|
14
|
Open PowerPoint file
|
Script-measured time to open a 2.1 MB PowerPoint presentation file
|
|
15
|
Send 250 email notes
|
Script-measured time for Outlook script to send 250 50KB messages
|
|
16
|
Receive 250 email notes
|
Script-measured time for Outlook script to receive the 250 50KB messages
|
|
17
|
Memory usage at idle
|
Memory - Peak Working Set Maximum amount of working set memory used by the processes
|
|
18
|
Memory usage during scan
|
Memory - Peak Working Set Maximum amount of working set memory used by the processes
|
|
19
|
Process count
|
From Task Manager
|
|
20
|
Installation time
|
Elapsed stopwatch-measured time between the appearance of the installation software's first window and clicking the "finish" button at the end – including file copying, online registration and, if performed by the installation software, initial system scan for malware
|
Table 6. Battery of Tests
NOTE: Our time measurement scripts were based on the Win32::OLE Perl library
The exact product versions we tested were as follows:
|
|
Version
|
|
McAfee Internet Security 2009
|
9.0
|
|
McAfee Total Protection 2009
|
9.0
|
|
Norton Internet Security 2009
|
16.0.0.125
|
|
Norton 360 V2
|
2.4.0.4
|
|
Kaspersky Internet Security 2009
|
8.0.0.454(a)
|
|
Trend Micro Internet Security 2009
|
Pgm: 17.0.1179
Engine: 8.900.1001
|
|
Trend Micro Internet Security Pro 2009
|
Pgm: 17.0.1224
Engine: 8.900.1001
|
|
Microsoft Windows Live OneCare 2.0
|
2.5.2900.15
|
Table 7. Product Versions
