There are many open source security tools available to system and network administrators, and to security professionals. I have listed a few of the most popular tools that could be used to make your machine more secure from outside attack. Basically they could be classified into 4 categories ie Network Reconnaissance, Intrusion Detection, Vulnerablity Scanners and OS Hardening Tools.
It is a common belief that a professional hacker (cracker?) before he atempts an attack on a network will try and make a blue print of the network (like say finding out which all ports in a machine are open.Whether he could telnet into the machine and so on). Network Reconnaissance tools provide this information, such as the ports open, services running on a host and the type of OS used.
Nmap (Network mapper) is an open source utility for network exploration or security audit. It was designed for fast scan of large networks but works equally well against a single host. Nmap uses raw IP packets to determine what hosts are available on the network, what services (ports) they offer, what OS'es (and versions) they run, what type of packet filters/firewalls are in use and many other characteristics. This is the first tool used by Bad Guys (for evil purposes ) and good guys (for finding out shortcommings) on a network.
Ethereal is a Packet Sniffer used for capturing traffic or open already captured files to scan for possible problems. For example, you can capture traffic between your firewall and router to see if any intrusions are taking place. Unfortunately, if a cracker happens to be an insider, he could use this tool to sniff out passwords and sensitive details that are transmitted across the network.
IDS tools should form the second line of defence on your network. IDSs are used to monitor a network or individual computers. There are two types of IDSs: network-based, which monitor a network or segment of network, and host-based, which monitor a particular system. IDSs can alert the administrator when attacks take place, resources are misused or during other suspicious activities.
This lightweight network-based IDSs created by Martin Roesch is a general purpose sniffer for various versions of linux, unix and windows. It detects a wide variety of suspicious traffic and attack attempts. Rule-based logging is used to perform content pattern matching and to detect a variety of attacks and probes. All in all your time spent on learning and configuring this utility will be well rewarded once you see snort protecting your network.
This is a host-based intrusion detection system. Tripwire carries out a file system integrity checking programfor linux / Unix. If an intruder tries to modify the designated files, Tripwire alerts and informs the system admin of the changes.
A vulnerablity can be a software, hardware or procedural weakness that may enable an attacker to enter a computer or network and have unauthorised access to resources within the environment. The best way to mitigate the risk associated with vulnerablities is to scan your network/system with a good vulnerablity scanner.
The Nessus Security Scanner is a free, open source and easy-to-use tool on several Linux distributions. Nessus employs a client server architecture and can test an unlimited number of hosts at the same time. It offers exportable reports in various formats, multilingual support and an up-to-date security vulnerablity database.
Typically when we install a new OS on a computer, usability and performance are the prime considerations. By default, several unnecessary services are also installed. Ideally, we should harden our OS to minimise compromises by removing unnecessary services. This process is called OS Hardening.
Bastille hardening system atempts to harden the Linux Operating System. It supports RedHat, Debian, Mandrake, HP-UX, SuSE and TurboLinux.