Securing a Server

The Problem

Simply put, the Internet is a dangerous place, and you (presumably) have a web server. Web servers, by default, are relatively insecure, presenting potential attackers with a variety of attack vectors (an "attack vector" is a way to attack a server or application). Fortunately, depending on your needs, it is usually easy to properly secure your server. For this example, we will be assuming you are using a Linux based server, and you are responsible for making sure that the server is as secure as possible.

The Attack

To understand how to secure your server, you have to have some understanding of how someone would perform the attack. This description is going to be based on someone who has located and targeted your server specifically, although most of what I talk about would also apply to a script kiddie doing general searches who happens to find your server.
The first step in attacking a server is to gather intelligence. The more you know about a server, the better able you are to find it's weaknesses and exploit them. The first step is to scan the ports on the server. Every application that allows external software to connect to it is called a service, and every service listens on a port for incoming connections. In this way, the operating system on the server can route incoming connections to the correct service/application. For example, if you have an Apache web server installed and listening for connections, it typically binds to port 80. When a browser requests a page from the server, the request goes to port 80.
A port scanner requests data from every port (or from selected ports) on the server. Based on the responses, an attacker can determine exactly what ports are listening for connections, and thus what services are installed on the server.
Once the services running on the server have been identified, the attacker then exploits known weaknesses in those services. Every service should be secured on it's own, and individual services are outside the scope of this article, but some important services are Telnet, FTP and HTTP. Your server will, under most configurations, allow connections to these ports, whether such access is needed or not, trusting the application's own security.

The Defense

The first step in protecting your server is with the firewall. A properly configured firewall will not allow incoming requests on any port that is not absolutely required. For example, a server being used to serve web sites typically only needs a few ports to be available to the public (80 and 443 for the web server, and 21 for FTP). Other ports may be required for administrative access, such as port 20 or 22 for remote command line functionality, or port 10000 for access to a web based control panel. The firewall should only allow traffic to these ports from IP addresses that are known to be trustworthy.
Next, if possible, an intrusion detection system should be installed. These applications detect suspicious behavior and block it at the firewall level. Hardware IDS devices are best, as they block the traffic before it ever reaches the server, but if that is not possible, software IDS systems can be very effective as well. The reporting functions can be invaluable if the server is ever successfully attacked, and most possible attacks will be thwarted before they ever threaten the security of the server.