Application
layer firewall technology for Enterprise Networks
Alpesh
J. Patel
East
Carolina University
ICTN
6865 – Fall 2008
Abstract
Firewall is an essential part of network security. The latest type
of firewall is called application layer or proxy-based firewall. Application
layer firewall is different from the other types of firewalls because it uses
layer seven of the OSI model to filter traffic.
There are many disadvantages of
application layer firewall. One of them is that every protocol must have a
proxy in order for the firewall to be completely effective. Another
disadvantage is that it generates a large reduction of performance since the
application services have to go through a proxy.
There are many solutions to overcome
these and other disadvantages of application layer firewall. One of them is to
have a hybrid firewall that would filter at the application layer like an
application layer firewall and filter at the network layer like a stateful
firewall.
Application layer firewall is a
great step toward better firewalls. An organization has to decide how to
overcome its disadvantages and decide where security and performance meets well
for them.
Introduction
“A firewall is a device or set of devices configured to
permit, deny, encrypt, or proxy all computer traffic between different security
domains based upon a set of rules and other criteria.” (Firewall, 2008) A
firewall is an essential tool in a network to filter unwanted traffic to and
from the internal network. It is essential that an organization has a firewall
and that it is configured correctly to the organization’s goals. For example,
you would not block HTTP port 80 from outside the network unless you want to
deny all incoming Internet traffic. “A
firewall is simply defined as a collection of components placed between two
networks to protect a private network from unauthorized intrusion.” (Al-Tawail
and Al-Kaitham, 1999) To protect your private network from the public, you will
have to have at least one firewall in the demilitarized zone (DMZ). “Firewalls
cannot protect against an insider attack. Firewalls provide little protection
against computer viruses.” (Avolio, 1999) To resolve the problem of an insider
attack, an organization must have reliable policies in place to reduce the
chances of an insider attack. Many insider attacks occur after an employee is
terminated. One policy is to remove access for that employee immediately.
Another policy is to have restricted physical access to the firewall. To
resolve the problem with computer viruses, an organization must keep virus
definitions up-to-date on all systems. Firewall is not the only answer to all
security attacks. In fact, there is no one device that is the answer to all
security attacks.
“Firewall
originally meant a wall to confine a fire in a building. The technology emerged
in the 1980s when routers were first used to separate networks.” (Firewall,
2008) Firewall technology really became popular for network security after
several attacks to existing networks. The Morris Worm was the first large scale
attack on Internet security. The community was not prepared for the attack at
all. (Firewall, 2008) The history of firewall shows the importance of firewall
use in an organization. In today’s standard, there is no legitimate
organization that doesn’t have a firewall in its network. For example, US Dept
of Justice and US Marine Corps have a more secure network involving numerous
firewalls and intrusion detection devices in their network compared to public
sector companies because of national security. “The first commercial firewall
was from Digital Equipment Corporation in 1991, and was based on the DEC
corporate firewall.” (Avolio, 1999) The first commercial firewall was great for
the limited services needed to monitor at the time. Today, we have many services that need
monitoring including access to the World Wide Web, video conferencing, e-mail,
music, etc. Firewall technology has to adapt to these growing needs. “Firewall
technology has improved substantially since it was introduced in the early
1990s, beginning with simple packet-filtering firewalls and advancing to more
sophisticated firewalls capable of examining multiple layers of network
activity and content.” (Wack, 2002) Since the first commercial firewall,
firewall technology has evolved to meet new challenges. Today, a firewall must
be able to examine different layers of the OSI model to block exploitation from
attackers.
There are three types of firewall technologies. The first
type of firewall technology is called packet filters. Packet filters act by inspecting the ‘packets’.
This type of packet filtering doesn’t look to see whether a packet is part of
an existing stream of traffic. (Firewall, 2008) Packet filter firewall
technology uses only network layer of the OSI model. Since it can’t distinguish
between types of traffic, it is not conducive for use in enterprise networks
for any other OSI layers. Packet
filtering basically looks to see if the packet is alright to forward. (Avolio,
1999) It is simple in its design. “Packet filtering is known for its speed and
reliability. Makes them ideal for placement at the outermost boundary.” (Wack,
2002) Packet filtering can be used in the outermost boundary to block certain
traffics. The more firewalls in the network, the harder it will be for
attackers.
The second type of firewall technology is called
‘stateful’ firewall also known as circuit level firewalls. Stateful firewall
maintains records of all connections passing through the firewall and is able
to determine whether a packet is either the start of a new connection, a part of
an existing connection, or an invalid packet. This type of firewall can help
prevent certain Denial-of-service attacks. (Firewall, 2008) Stateful firewall
technology is still used today because it can provide another barrier toward
attacks. Stateful inspection firewall maintains a record of connections and
ports. They make access control decisions based on connections. (Wack, 2002)
Stateful firewall addresses security vulnerabilities in the TCP/IP protocol
suite. It doesn’t look at the application layer services for possible
vulnerabilities.
The third type of firewall technology is called
application layer firewall. It is also known as proxy firewall, application
gateway firewall, and gateway firewall. The key benefit of application layer
filtering is that it can ‘understand’ certain applications and protocols (such
as File Transfer Protocol), and it can detect an unwanted protocol or an abused
protocol. (Firewall, 2008) This type of firewall operates on the application
layer of the OSI model. It can block packets by protocols and applications. It is
a method that none of the other firewall technologies can perform. “The effectiveness
of a firewall is increased substantially in preventing access by Internet
hackers by application-layer processing capabilities.” (Al-Tawail and
Al-Kaitham, 1999) In today’s standards, Internet hackers are evolving to newer
measures to bypass firewalls in an organization. An organization must research
into new firewall technologies. Since
application layer firewall makes it decision based on applications, the
client/server model won’t apply. (Avolio, 1999)
Application
layer firewall uses proxies to block certain applications. Application layer
firewall fully terminate TCP and SSL protocols and secure based on positive security,
negative security, real-time dynamic application learning, and profiling.
Proxies fully control the TCP and SSL handshakes to the clients and to the
servers. (NetContinuum, 2006) Application layer firewall looks at protocols and
uses proxies to look at application packets to determine if it is acceptable. The
auditor should ensure that the security on the operating system is secure
before evaluating the security offered by the application level firewall.
(Naidu, 2007) Since firewalls are prone to insider and computer virus attacks,
keeping systems secured is vital. Application layer firewall operates more
closely to the operating systems than other firewalls. The effectiveness of
this firewall is tied with the security of the operating systems.
Each
individual application-proxy interfaces directly with the firewall access
control rule set to determine authenticity. This results in a finer level of
access-control for each application and connection. (Wack, 2002) The
application proxies are bind closely with the firewall rule set to determine
access, thus application layer firewall can have a certain rule set, the
application proxies can exercise the rule set, and in addition the proxies can
use additional authentication for the end user.
Advantages and Disadvantages of
Application based firewall
There are numerous advantages of using an application
layer firewall. For example, access to network layer through application layer
in the OSI model and not requiring a network layer route between the inside and
outside interfaces of the firewall are the primary advantages of an application
proxy firewall. Access to the higher layers of the OSI model allows the
firewall to make decisions with more knowledge than other firewall types.
(Adeptech, 2003) This is a very good advantage of the application layer
firewall since this is the only type of firewall technology that accesses
multiple layers of the OSI model. This advantage needs to be investigated in every
organization. Application proxies prevent specific protocol vulnerabilities
because it rewrites an entire packet before going to the next OSI layer.
(Adeptech, 2003) This advantage is only performed by application layer
firewalls since it is bind closely with the applications.
“The
advantages of proxy servers are user-level authentication, logging, and
accounting.” (Opplinger, 1997) Since application layer firewalls are bind
closely with application proxies, application layer firewalls can have dual
security measures. First, it can use its own rules to determine if the packet
is deemed good to traverse. Second, it has user-level authentication for the
application that the packet is trying to access. If either of these security
measures fails, the packet will not traverse to the next stage. This duality is
only available with application layer firewall.
Providing
security services on the application layers is a very flexible way to handle individual
security requirements. (Opplinger, 1997) Certain requirements are only for
certain applications. For example, an instant messaging application has some
unique requirements from other applications. The method of providing security
to these unique applications is application layer firewall. It is the only type
of firewall that can provide this security measure.
Application
layer firewalls include proxy services. Proxy services enforce high-level
protocols. (Cisco Systems, 2002) Application layer firewall can understand protocols
such as FTP since they’re proxy services for FTP. Since they can understand
such protocols, they can block certain incoming or outgoing FTP traffic and the
proxy service can regulate FTP traffic by user authentication. “Proxy services
maintain information about the communications passing through the firewall.”
(Cisco Systems, 2002) This is an advantage that can only be provided by
application layer firewalls. It relies on proxy services for logging all types
of communications for numerous applications. “Proxy services can be used to
deny access to certain network services.” (Cisco Systems, 2002) These network
services can be numerous protocols. Proxy services are capable of manipulating
packet data. (Cisco Systems, 2002) Proxy firewalls have the ability to provide
granular policy controls based on user authentication. (Messaging News, 2006) A
proxy service can rewrite an entire packet to meet its goal. Proxy services are
very good in providing firewall policy control on an application level. They do
not allow direct communications between external servers and internal computers.
(Cisco Systems, 2002) “Network Address Translation occurs by default with proxy
firewalls.” (Zeltser, L., et.al. , 2005) Proxy services have built-in Network
Address Translation to prevent attackers from using proxy services to attack
certain protocols if they somehow bypass the application layer firewall. Proxies
provide users with the appearance that they are communicating directly with
external servers. (Cisco Systems, 2002) No other type of firewall technology
provides application proxies that provide transparency so that the end user
will not see a difference while communicating with external networks. Proxy
services generates audit records, allowing administrators to monitor attempts
to violate the firewall’s security policies. (Cisco Systems, 2002) This gives
administrators a granular view of the attacker’s attempts by examining the
proxy services’ audit records. Only application layer firewalls can give such a
granular view.
Only the application layer firewall can inspect and
filter on defined application requests.
With application layer firewall, an organization can enforce their
firewall policies through the defined application proxies. (Fernandez E., et
al., 2003) “It protects against possible
implementation faults in the protocol stacks of the internal systems.”
(Fernandez E., et al., 2003) Because application layer firewalls uses
application proxies, it doesn’t rely on the protocol for filtering, thus, it
doesn’t have the faults of protocols.
Application layer firewall is considered to be the most secured and
advanced firewall. (Phatak V., n.d.) There are many firewalls in the market
that use application proxies. For example, “Pipex Security Firewalls and
InterGate Firewall are two firewall products that use application proxies.”
(Fernandez E., et al., 2003) There
are several disadvantages of the application based firewall technology. An
organization must carefully review the advantages and disadvantages to conclude
if this type of firewall is feasible. “The disadvantage of application proxy
firewalls is speed. Since the proxies decode and analyze the entire packet,
this requires much more overhead than other firewall types.” (Adeptech Systems,
2002) An organization must determine the value of performance versus the value
of security. Since application layer firewall is the only the type of firewall
that looks at the entire packet, it is evident that an organization must carefully
review this security measure compared with the value of the performance.
Since application layer firewalls rely on application
proxies, there must be an application proxy for every application that an
organization uses to fully use the features of the application layer firewalls.
It is a major disadvantage if an organization that uses a non-standard
application doesn’t have an application proxy for that particular application.
(Oppliger, R., 1997)
There are disadvantages related to proxy services. These
disadvantages need to be carefully reviewed since application layer firewalls
use proxy services. One disadvantage is that “proxy services require you to
replace the native network stack on the firewall server.” (Cisco Systems, 2002)
This might not be a great disadvantage since application layer firewalls do not
have the flaws of protocols that other firewalls have. Another one is that
“because proxy servers listen on the same port as network servers, you cannot
run network servers on the firewall server.” (Cisco Systems, 2002) Since
application layer firewall require a large processing time, it is not viable to
use network servers on the firewall server. Proxy services are vulnerable to
operating-system bugs. (Cisco Systems, 2002) Other firewall types are
vulnerable to protocol flaws because they use the protocols in the OSI model.
Application layer firewall strongly uses operating systems and applications,
thus flaws in the operating system or the application in question will strongly
affect the application layer firewall.
“Application
layer firewalls overlook network packet information that is contained in lower
layers.” (Cisco Systems, 2002) Application layer firewalls operate on the
application layer of the OSI model. Any flaws in the lower layers of the OSI
model is ignored when using application layer firewall, but only the
application layer firewall is considered to have the strongest security due to
the detail inspection of the packet. “Proxy-based firewalls may require change
in applications and/or the user’s interaction with the system.” (Fernandez, E.
et al 2003) Application layer firewalls are complex to install than other
firewall types since it is more secured and it is the only one that uses
application proxies.
Yet another disadvantage of application layer firewall
comes with dealing with Virtual Private Network (VPN). “Virtual Private Network
(VPNs) may not function through a proxy firewall. VPN packet authentication
will fail if the IP address of the sender is modified during the transmission.”
(Zeltser, L. 2005) The IP address of the sender can be modified during
transmission if the application layer firewall uses network address
translation. “A proxy firewall as a bottleneck could become a single point of
failure for network services.” (Agilent Technologies, 2002) This is a crucial
disadvantage because using only a single proxy firewall to monitor and block
traffic could result in a single point of failure from either the attackers or
from internal attacks.
Resolutions to the disadvantages of
Application layer firewall
Despite all these disadvantages of an application layer
firewall, there are several resolutions that an organization needs to evaluate
if application layer firewalls are to be used in their network. One of the
resolutions is to combine packet inspection and application layer firewall
technologies to include all layers of the OSI model. One company has performed
this resolution. “By enforcing deep packet inspection in concert with
Application Proxy firewall technology, WatchGuard Firebox X appliances inspect
all seven layers of the OSI model, thus providing unmatched, holistic network
security.” (WatchGuard Technologies, 2008)
The complexity issue of application layer firewall can be
resolved with a well designed network. “Proxy-based firewalls may require
change in applications and/or the user’s interaction with the system. This is
not necessary, however, in a well-designed system.” (Fernandez, E. et al, 2003)
In fact, in a well designed network, users might have very little or no
difference in interaction with the system. The disadvantage of Virtual Private
Network (VPN) not functioning through a proxy firewall can be resolved. The
problem arises because the VPN doesn’t end at the firewall. If the VPN endpoint
is the firewall, then it is not a problem. (Zeltser, L. 2005) This resolution
also requires a well designed network such that the complexity of the firewall
and the VPN problems are eliminated.
The disadvantage that if an application level proxy is
not available, the functionality of the application level firewall will not be
fully utilized can be addressed by generic proxy toolkits. Most organizations
don’t use non-standard ports such that there is no proxy available. Even if
this is not the case, the vendor for the application will quickly try to
provide a proxy to maintain customer satisfaction. Generic proxy toolkits give
you the ability to use the functionality of an application level firewall in a
smaller scale. A popular example of a generic proxy toolkit is called SOCKS.
“SOCKS provides most of the benefits of an Application-Level Proxy like
authentication and Listener/Initiator.” (Agilent Technologies, 2002) Since it
is a generic toolkit, it doesn’t have the specific application level policies.
Some organizations are trying to resolve the problem of
the rush of new applications that don’t have proxies or the smallest change in
an application resulting in a requirement of a new proxy by investing in new
technologies. Palo Alto has invested in a new type of technology that solves
this problem. “Palo Alto says it solves this dilemma with a signature-based
system that allows for matching network traffic against a database of more than
550 applications.” (Baumstein, A. 2008) The only problem with this type of
technology is that the database must be kept up-to-date with the latest
applications.
In order to resolve the problem with a firewall being the
single point of failure, obviously, there should be a distributed-based
firewall system in the network. In a distributed solution, policy is still
centrally defined. Enforcement takes place on each endpoint. Distributed
firewalls are no longer a single chokepoint. Throughput is no longer limited by
the speed of the firewall. (Bellovin, S. 1999) With a distributed firewall
solution, the importance of network design is critical. The question that each
organization should ask is which type of firewall and where do we need to
install it on our network.
To address the disadvantages of application layer
firewalls, many vendors have devised a hybrid firewall that has application
layer firewall features and packet inspection firewall features. “The main
result of recent advances is that it is now common to see many
application-proxy gateway firewalls with basic packet filter functionality.
Hybridization of firewall platforms makes the pre-purchase product evaluation
phase of a firewall project important.” (Wack, J. 2002) So, it is very
important to know which firewall to use, its advantages and disadvantages, and
where to use it on your network.
Conclusion
Application layer firewall is a new type of firewall
technology that is only going to improve over time. It is the only type of
firewall that inspects packets at the application layer, therefore, it is not a
question of using this firewall; but it is a question of where to use this
firewall. It has many advantages and disadvantages, but the current advances in
hybrid firewall technologies have made many of the disadvantages meaningless.
With careful network design and the use of hybrid firewalls, an organization
can have firewalls with robust network security that can inspect packets at any
OSI layer. A fine example is “WatchGuard Firebox X appliances.”(WatchGuard
Technologies, 2008) Use application layer firewalls or add the benefits of
stateful inspection and decide where to use application layer firewalls in your
network. An organization’s best measure for this type of firewall is where to
draw the line between security and performance.
References
Adeptech Systems (2003). Considerations in Developing Firewall Selection Criteria. [Electronic Version] Retrieved on 8/29/2008 from: http://seclists.org/firewall-wizards/2003/May/att-0044/FirewallWhitepaper_v_05_pdf
Agilent Technologies (2002, October). Network Tester Firewall Primer [Electronic Version]. Retrieved on 8/29/2008 from: http://advanced.comms.agilent.com/networktester/docs/whitepapers/pdfs/NetworkTesterFirewallPrimer.pdf
Al-Tawail, K. and Al-Kaitham I. (1999). “Evaluation and Testing of Internet Firewalls”. International Journal of Network Management, 9, 135-149. *
Avolio, Frederic (1999, June). Firewalls and Internet Security. The Internet Protocol Journal, 2(2). Retrieved on 9/11/2008 from: http://www.cisco.com/web/about/ac123/ac147/ac174/ac200/about_cisco_ipj_archive_article09186a00800c85ae.html *
Baumstein, A.
(2008, March). From the Labs: Palo Alto’s
Firewall Appliance Information Week, March 24, 2008. Retrieved on 8/26/2008
from: http://www.informationweek.com/news/hardware/reviews/showArticle.jhtml?articleID=206904763
Bellovin, S.
(1999, December). distributed firewalls
Retrieved on 8/29/2008 from: http://www.usenix.org/publications/login/1999-11/features/firewalls.html
Cisco Systems (2002). Evolution of the Firewall Industry. Retrieved on 8/26/2008 from: http://www.cisco.com/univercd/cc/td/doc/product/iaabu/centri4/user/scf4ch3.htm
Fernandez, E., Petrie-Larrondo, M., Seliya, N., Delessy-Gassant, N., Schumacher, M. (2003). A Pattern Language for Firewalls [Electronic Version] Retrieved on 8/26/2008 from: http://jerry.cs.uiuc.edu/~plop/plop2003/Papers/Fernandez-firewalls.pdf
Firewall. (2008, August 26). In Wikipedia, The Free Encyclopedia. Retrieved on 8/26/2008 from: http://en.wikipedia.org/w/index.php?title=Firewall&oldid=240685574
Messaging News (2006, January). “Existing Network Security Devices”. Messaging News, 1
Naidu, Krishni (2007, April). “waFirewall Checklist”. Retrieved on 8/26/2008 from: http://www.sans.org/score/checklists/FirewallChecklist.doc
NetContinuum (2008). Proxy vs. Non-Proxy Web Application Firewalls. [Electronic Version] Retrieved on 8/26/2008 from: http://www.securitytechnet.com/resource/rsc-center2/vendor-wp/Netcontinuum/ProxyvsNon-ProxyWAFs.pdf
Oppliger, Rolf (1997, May). “Internet Security: Firewalls and Beyond” Communications of the ACM, 40 (5), 92-102. *
Phatak, V. (“n.d.”). Network Firewalls. Retrieved on 8/29/2008 from: http://www.phatak.com/firewalls.php
Wack, John (2002, January). Guidelines on Firewalls and Firewall Policy [Electronic Version] Retrieved on 8/26/2008 from: http://csrc.nist.gov/publications/nistbul/01-02.pdf
WatchGuard Technologies, Inc.; WatchGuard Wins Trifecta of Top Honors. (2008, January). Computers, Networks & Communications, 234. Retrieved on 9/25/2008, from Sciences Module database. (Document ID: 1529937961).
Zeltser, L., Kent, K., Northcutt, S., Ritchey, R., Winters, S. (2005, April). Inside Network Perimeter Security: Proxy Firewalls (2nd ed., chap. 4). Retrieved on 8/29/2008 from: http://www.informit.com/articles/printerfriendly.aspx?p=376257