Answers to your Questions
Who needs to test firewalls and other network
security devices?
Service providers, network operators and data center operators
generally test network security equipment within their test
labs during equipment evaluation and benchmarking, network
design validation, acceptance testing and regression testing.
For example, firewalls must be tested to ensure that they
meet not only functional and security criteria, but also to
characterize their performance and scalability in the proposed
configuration. Performance testing verifies that a firewall
can cope with the volume and application mix of expected traffic,
both now and in the future, and that application performance
remains acceptable when the network is subjected to Denial
of Service attacks. A firewall that is configured poorly or
selected purely on price or reputation can easily become the
network bottleneck!
Equipment manufacturers and system integrators measure device
performance during system / QA testing to ensure that product
specifications are met, to check reliability by stressing
the device with high loads and "corner case" traffic,
and to verify stability under real-world conditions likely
to be encountered by customers. Sales and marketing departments
also rely on third-party test equipment for customer demonstrations
or within proof-of-concept labs to verify that a customer's
existing and future network requirements can be met with a
proposed device, topology and configuration.
Why aren't vendor specifications
adequate?
Firewall and VPN vendors generally provide a huge feature
list but very little performance data. Typically, a firewall
data sheet will list the maximum TCP session rate, the maximum
number of concurrent TCP sessions, and the unencrypted and
IPsec encrypted traffic throughput. This poses several problems
for the customer:
- Not all measurements are created equal. For example, some
vendors measure sustainable TCP session rate using complete
TCP sessions, with 3-way handshakes for both the opening
and closing of connections. Other vendors offer inflated
results by only opening sessions, or worse, by simply blasting
the device with TCP SYN connection request packets and ignoring
acknowledgements. Comparing dissimilar measurements is meaningless.
- Raw TCP statistics are no longer useful. Today's network
security devices are application-aware, meaning that they
perform packet inspection, filtering, rate-limiting and
perhaps intrusion detection based on the layer-7 packet
contents to enable features such as URL and content filtering,
Network and Port Address Translation (NAT), and protocol
anomaly/conformance checking. Each feature can (and does)
have a huge impact on performance, making it necessary to
test application-layer performance using real, stateful
application traffic such as HTTP, FTP, NFS and SMTP - both
with and without DOS attacks during the test.
- Each network is different. Different topology, device
configuration, filter sets, traffic volume, applications
and services. Performance should be characterized in the
target configuration using both a breadth and mix of application
protocols to determine real-world performance and scalability.
During network design and equipment selection, independent
test equipment can help you meet or reduce your equipment
budget by benchmarking the performance of devices from different
vendors to ensure that you can satisfy your expected traffic
needs with fewer devices or at a lower cost.
Our firewall has Common
Criteria and ICSA certification. Why isn't that enough?
The National Information Assurance Partnership (NIAP) NIST/NSA
Common Criteria
Evaluation and Validation Scheme (CCEVS) is a government
/ industry initiative for evaluating and validating products
for "accreditation" for conformance to the Common
Criteria for IT Security Evaluation (ISO Standard 15408).
Common Criteria certificates are issued for IT products such
as firewalls, IDS/IPS and anti-virus systems, and even for
operating systems and switches. Certified products and their
test results are published on a public Validated Products
List (VPL), used by buyers (especially in government/defense)
during product selection to ensure that the products meet
basic requirements for security.
The International Computer Security Association (ICSA) Labs
Modular Certification Criteria (version 4.0), developed in
conjunction with the Firewall Product Developers Consortium
(FWPD), also offers certification against a standard yet evolving
set of criteria to ensure that products claiming to have firewall
capabilities meet an industry-accepted standard.
Neither CCEVS nor ICSA certification is an endorsement of
the "goodness" of an IT product -- this is where
third-party performance test equipment is needed. Performance
test gear such as the Agilent NetworkTester can verify the
layer 4-7 functionality of an application-aware firewall or
network security device under stress; gauge its suitability
for a particular network design; measure its scalability for
a target network, with a given mix of applications and expected
traffic load; characterize its ultimate performance limits;
and quantify its ability to filter stateful application traffic
and successfully block or limit Denial of Service attacks.
Why is Application Intelligence necessary,
and how does it impact performance?
Application intelligence is a general term covering any kind
of layer 7 processing including URL/content filtering, rate-limiting,
packet inspection, anomaly checking, virus/spam filtering,
DOS attack mitigation, and intrusion detection/prevention.
Let's study the FTP protocol (active mode) as an example.
Within an FTP session, the FTP "port" command can
be used (on the control connection) to convey a client TCP
port number to the server to use for the data connection for
the actual file transfer. Without application intelligence,
the client firewall would not know which TCP port number was
being used for the data connection, so the client firewall
administrator would need to either block all FTP active mode
traffic (undesirable), or would need to allow all incoming
traffic from FTP data port 20 to any destination port - opening
up a huge vulnerability to attacks that "spoof"
FTP packets. In contrast, application-aware firewalls inspect
the content of the FTP control connection packets, read the
negotiated data port number, and open up a small, temporary
"hole" for the data connection for the duration
of the FTP session. Similar issues exist for many protocols
such as RTSP, DNS and VoIP. This is just one of several reasons
for application intelligence.
Compared to a traditional "TCP/IP" firewall, a
typical application-aware device:
- Inspects each packet more "deeply" (looking
beyond the TCP headers into the application payload);
- Keeps session state information about layer 5-7 protocols;
- Correlates traffic across multiple connections to detect
attack patterns;
- Compares layer-7 headers and content against blocked URLs,
blacklisted user lists, virus signature databases, "bad"
words and password-cracking commands.
These capabilities are process-intensive and have been shown
to degrade performance by over 50%.
For this reason, it is vital to measure the application performance
of your network security devices, in your target configuration,
with real application traffic.
How can DOS attacks degrade
firewall performance by as much as 50-95%?
Whether or not the firewall is application-aware or includes
DOS attack detection and prevention capabilities, DOS attacks
can impact performance.
A simple TCP SYN flood attack spoofs incoming TCP connection
requests from multiple source addresses. To the firewall,
these look like "regular" connection requests. The
firewall must process each request and, if it matches its
filter rules, it must create a new entry for the session in
its state table. It will hold each entry until the half-open
connection times out or until the firewall runs out of resources
(the table fills). Even though the DOS attack bandwidth may
be quite low, the firewall may waste most of its processing
power dealing with the attack, reducing the performance available
for processing legitimate packets. If the firewall's state
table fills, connections will be dropped or refused, effectively
reducing useful performance to zero.
A clever firewall with DOS attack mitigation capabilities
may be able to correlate the attack packets and prevent the
state table from overflowing, or perform rate limiting on
new connections, but the correlation calculations themselves
will sap firewall performance.
For these reasons, it is vital to measure the application
performance of your network security devices both with and
without simulated DOS attack traffic.
How can I test Network
and Port Address Translation (NAT/PAT), and why is it especially
important for application-aware firewalls?
Network and Port Address Translation (NAT/PAT, or simply NAT)
serves two purposes:
- It enables a private network of many hosts to share one
(or a few) public network IP addresses. It does this by
swapping the public IP source or destination address with
the private IP source/destination addresses within the IP
packet header of each packet that traverses the firewall
or gateway. The firewall maintains a table of active sessions
for this purpose. Usually, it will also translate TCP and
UDP port numbers to prevent possible "collisions"
between two private network hosts attempting to use the
same port numbers to communicate with the same remote host.
- It may help to hide or "cloak" hosts in the
private network, although some writers argue that this is
no more effective than a correctly configured stateful firewall
without NAT/PAT, and that network administrators should
close security holes rather than relying on "cloaking".
Application protocols such as DNS, FTP, H.323 (for VoIP)
and some instant messaging, P2P and gaming applications have
difficulty traversing NAT/PAT firewalls. Each of these protocols
carries IP addresses, TCP and/or UDP ports within the payload
of the application-layer packets.
For example, the DNS service translates hostnames into IP
addresses (or vice versa for d.c.b.a.IN-ADDR.ARPA and reverse
lookups). If the IP address returned or submitted is one that
must be translated, it requires the firewall to decode and
look within the DNS message. Similarly, the FTP "port"
and "pasv" commands communicate TCP port numbers
for the data connection, which may need to be translated.
Therefore to correctly perform address translation, a firewall
or gateway must be "application-aware".
The translation of addresses within the IP, TCP/UDP and application
layers, and the "tracking" of the state of applications,
reduces the firewall's available processing performance. For
this reason, firewalls and NAT gateway devices must be tested
using stateful application traffic - between real clients
and real servers. The device performance must be measured
for each application and for the expected mix of applications.
Another consideration closely related to NAT is DHCP. Most
NAT firewalls include a DHCP server to allocate addresses
automatically to hosts in the private network. Following a
temporary power outage or connection failure, the firewall
may receive many DHCP requests at the same time. For large
private or service provider networks, it's important to measure
the firewall's DHCP performance.
What capabilities should
I look for in network security performance test equipment?
- Client and server emulation integrated into a single system
with a single application and single user interface. This
allows you to configure and control the whole test system
- both clients and servers - from within a single Test Plan.
It also allows you to correlate client and server measurements
in real time to determine the cause of performance bottlenecks.
Common parameters, such as port numbers, URL lists, VLAN
IDs and address ranges, can be represented with a single
named attribute across both client and server test interfaces.
- A broad range of application protocol emulations, including
protocols such as NFS, SAMBA, RTSP, DNS and instant messaging,
to comprehensively test the functionality and performance
of application-aware firewalls.
- The ability to mix multiple protocols on a single port,
and the ability to easily create multiple client profiles
to stress the firewall with a realistic mix of application
traffic and user behaviors.
- The ability to easily emulate and measure stateful application
traffic over "access" protocols such as 802.1x,
DHCP, PPPoE, IPsec and IPsecv6. It is not sufficient to
test these network access protocols in isolation; their
performance must be tested in conjunction with real applications
such as HTTP, FTP and SMTP. The access protocols should
be fully integrated into the GUI so that these tests can
be rapidly configured without the need for scripting and
without configuring and running multiple applications.
- A powerful, flexible user interface with many "transaction
variability" features, so that you can rapidly create
and run complex tests without the need for scripting:
- Named attributes and named lists let you represent
common parameters (such as IP addresses, email blacklists
and blocked URLs) with global variables. It is useful
to be able to change the value of these parameters from
a single location.
- List functions allow you to cycle through or randomize
IP address ranges and named lists.
- A string expression editor lets you create "dynamic"
strings that change from transaction to transaction…
for example, to create a varying Subject field to simulate
SPAM, or to rapidly create a DNS lookup table.
- The ability to attach files, such as email viruses,
multimedia play lists and FTP attachments, is useful
for testing the device with real content.
- Real-time control of named attributes and lists allows
you to change protocol and test execution parameters
even while the test is running - for example, to determine
the impact of a larger file attachment, or the difference
between HTTP 1.0 and 1.1 performance, or to change the
mix of traffic from different applications or different
client profiles. Without this capability, you must first
stop the test before making changes, allowing connections
to time out and the firewall's state table and CPU utilization
to decrease.
How can I perform password-cracking
tests? I have a firewall or intrusion detection/prevention
device that can be configured to detect and block such attacks.
Here are a few ideas:
- Configure the DUT to detect and block repeated password
attempts. Configure the test equipment to emulate a telnet
client and a telnet server. Configure a client profile that
repeatedly attempts to login to a remote Unix server (behind
the firewall) by cycling through a password dictionary file
or choosing a password at random, or using a string expression
editor to create unique username/password attempts.
- Configure the DUT to detect and block attempts to list
or access the /etc/password file by enabling the DUT's NFS
or SAMBA file sharing control, or by detecting and blocking
the string "/etc/password" within protocol messages.
Configure the test equipment to emulate multiple NFS, SAMBA,
telnet, FTP and/or HTTP clients. Some clients should be
configured to generate "permitted" requests, while
others should generate requests.
|
Network Services 
