When it comes to keeping bad stuff from entering computers, hardware protection devices work better than software solutions. “The software side of things still needs a lot of work,” NSS Testing Labs CTO Bob Walder told TechNewsWorld.
That is part of the findings from a rigorous battery of tests done on leading intrusion detection and prevention (IPS) products conducted by the NSS Group in Great Britain. Tests included evaluations of the first generation of 2 Gbps plus IPS devices.
Testing Gaining Momentum
The NSS Group, which has been testing computer products since 1991 and security-based products exclusively since 1999, recently completed round two of its comprehensive third-party evaluation of intrusion detection and prevention products.
The test, which included nine participants, produced four failures and five NSS Approved awards. The security companies receiving passing grades are BroadWeb, Fortinet, SecureSoft, Top Layer Networks and V-Secure.
Last year, the first round of testing included five products, a list of the only players in the IPS field. Almost immediately, the market began to grow, and nine vendors signed up for Edition Two.
Ten vendors have signed up for the wired category in the third round of testing so far. Seven vendors will be tested in the new Multi-Gigabit category.
Test Specs
“It is interesting to note that between publishing Edition One and Edition Two, the analyst groups who were previously so sure that IDS was dead and IPS stillborn have now come around to our way of thinking,” Walder, the author of the report, said from his offices in France.
The NSS IPS Group Test evaluates the performance, reliability, security effectiveness and usability of Network IPS products. The test consists of seven sections within three primary areas: performance and reliability; security accuracy; and usability.
“While the so-called ‘deep inspection firewalls’ are not ready for prime-time deployments, security administrators need to make the best use of the technology that is available, and for now that means a combination of firewalls, in-line intrusion prevention devices, and intrusion detection systems,” Walder said. “They are likely to be in use for quite some time to come.”
The test suite contains more than 800 individual tests, many of which are run multiple times, to provide the most thorough and complete evaluation anywhere of IPS products available today.
This current round saw the introduction of a new rate-based IPS methodology to complement the existing content-based IPS methodology used in the first round of testing. This has allowed NSS to more accurately test rate-based/attack mitigation products.
Winners, Losers
The NSS testing program has become largely accepted in the Internet security field as a sort of Good Housekeeping Seal of Approval.
As more vendors are submitting their security products for evaluation, NSS is focusing on maintaining the validity and meaningfulness of the results.
Not every product submitted for testing receives an NSS Approved award. Pushing the products under test to their limits in a heavily utilized network produced some interesting results. It also posed problems for some vendors.
Testing standards are very high, as evidenced by the fact that nearly half of the products submitted for this year’s test failed at some point.
“We believe that our IPS test methodologies will become the de facto standard for testing in-line intrusion prevention/attack mitigation devices, and the NSS Approved logo an essential item on the list of requirements when purchasing these products,” Walder added.
Meeting a Need
IPS testing is a growing market because of the number of vendors entering the field in response to the rise in IPS threats, said Michael Paquette, vice president of marketing and product management at Top Layer Networks.
“Security is becoming mainstream news. Now everybody is getting concerned about computer security,” Walder said.
Paquette said he is very pleased with the benefits obtained from the NSS testing program. Having a respected third-party validation of your claims adds credibility and marketability to the products.
Rigorous testing also provides product engineers with knowledgeable feedback to help improve the product, he added.
“The testing provides a meaningful mix of laboratory environment and real world conditions. The testing is done on a real Web server using a copy of the environment,” Paquette said. “NSS does a very good job with both environments,” Paquette told TechNewsWorld.
Trends Not Discovered
Security product providers aren’t reporting clear trends yet, Walder said, noting that it is a bit too early for that. People are beginning to learn that IPS is the needed next security device.
Walder said lots of companies are starting to rush products to the market. This is partly in response to consumers and enterprise realizing that firewalls are not the sole means of reliable protection against intrusion attacks.
“IPS products are very different from vendor to vendor,” Walder said. “There are choices between hardware and software solutions. And the worst aspect of them all is management software.”
Costs, Directions
The testing process is reasonably expensive. Paquette said his company spent around US$25,000 for the two-week participation.
“But the real value is the exposure you gain to lots of expensive test equipment and the real-world responses,” he said.
Walder said he expects IPS vendors to sharpen their products’ focus as a result of the testing program.
“It’s all about reducing the burden on administration. It’s about being able to spot the top five items from hundreds of traffic incidents that come in on a given day,” Walder said.
He said vendors are beginning to take a multi-gigabyte approach with low bandwidth for smaller use. In this way, IPS security it developing into a market for everyone.
“The concept of the Zero Day Detection is the Holy Grail of IPS protection. But that is not fully achievable,” Walder said about the ability to spot and stop an attack so quickly that it can no longer spread while vendors develop new signatures and patches.
Loading ...
