Iptables firewall versus nmap and hping3
Part 1 : The basics

How to harden your firewall against the powerful tools nmap and hping3, based on the Thylacine iptables based host firewall. This is part 1 : The basic settings

Article published on 20 September 2010
last modification on 30 May 2016

by Emeric Nasi

Note: In order to understand this document it is strongly recommended that you know about the TCP/IP protocol and have iptables, nmap and hping3 installed. I suggest as well the use of wireshark that is a great tool to learn about network protocols.
License : Copyright Emeric Nasi, some rights reserved
This work is licensed under a Creative Commons Attribution 4.0 International License.
Creative Commons License.

The well-known hping3 and nmap are worldwide used and are really powerful tools. You can find countless documents about "how-to monitor a firewall using nmap". In this article I will go further and explain how to monitor a firewall using nmap/hping3 AND how to tweak your iptables firewall to prevent attacks from these tools.

Note : The iptables commands are from the Thylacine host firewall that you can find in the Thylacine security hardening tool.
Note2 : Even if this article is based on a host firewall, you can easily transpose the commands to a network iptables firewall.

I divided this article into three sections:
Basics iptables features (block or accept particular IP and port)
Iptables versus port scanning attempts
Iptables versus flooding attempts

I. Basic iptables features

Iptables principal use is a port based firewall. Iptables can be way more than this (ex pattern matching, connection tracking, etc) but 90% of the people questioning about iptables ask : how to block or accept a given IP address and/or port.

1.1 Test a particular port/address using nmap and hping3

Test if a particular TCP port is open using nmap :
nmap -p <port1,portt2> <dest_host>
Example :
nmap  -p22
nmap will answer if the port is open, closed or filtered.

Note : Keep in mind that it is not because nmap tells you something that it is true.
For example, if a firewall is set to reject packets from a blacklisted IP with a TCP RST-ACK packet, an nmap scan coming from that IP will tell that the port is closed even if in fact it is filtered (other IP can access it).
For nmap, a TCP connection closed using RST-ACK flag means that the port is closed (because that is what happens normally when there is no firewall). That is why hping3 may be preferred for more advanced users because it does not interpret itself the results of the scans.

Test if a particular TCP port is open using hping3 :
hping3 -S -c1 -p<port> <dest_host>
If a response is received hping3 will display the header of the packet.
Here is a way to interpret hping3 results (and remember what is written in the previous note, interpretation is not 100% sure) :

  • If the response is a TCP SYN-ACK packet (flags=SA) you can assume the port is open.
  • If the response is a TCP RST-ACK packet (flags=RA) you can assume the port is closed.
  • If the answer is an ICMP port unreachable packet, you can assume the port is filtered and your packet was rejected.
  • If there is no answer you can assume the port is filtered and your packet was dropped.

And UDP???
Because UDP protocol is not connected (no acknowledgment packet) you never know if you get no response because the packet was dropped or because the packet was accepted. However if you receive an ICMP port unreachable packet, you can assume the port is closed.

1.2 Iptables basic settings

Here I will just give simple iptables settings examples.

Deny by default all connections :
iptables -P INPUT DROP
iptables -P OUTPUT DROP

Accept packets from a particular IPv4 address (here :
iptables -A INPUT -s -j ACCEPT

Reject packets from a particular IPv4 address :
iptables -A INPUT -s -j REJECT

Drop packets from a particular IPv4 address :
iptables -A INPUT -s -j DROP

Accept packets on port 22 TCP (SSH) :
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

Accept outgoing packets from your port 22 TCP :
iptables -A OUTPUT -p tcp --sport 22 -j ACCEPT

1.3 Fragmented packet

Tools such as nmap can use evasion techniques to avoid detection from IDS and firewalls. Fragmented packets is one of them and consist in sending several tiny packets instead of one normal size packet.
You can use fragmented packets with nmap using the "-f" option.
Example :
nmap -f -sF -p22

Nowadays most firewall and IDS detect fragmented packets, however if you are afraid about those you can brutally get rid of any incoming fragmented packets using the next iptables setting :
iptables -A INPUT -f -m comment --comment "Drop fragmented packets"  --jump DROP

1.4 IP black list

Iptables offer the possibility to create dynamic blacklist. I mean, the possibility to automatically and temporarily ban an IP address (In case of automatic ban, they should be temporary to avoid denial of service ). The "recent" module can be used to check packets and compare them to previously listed packets. You have the possibility to drop these packets according to various characteristics such as IP address, time, TTL, etc.
In Thylacine I create various dynamic blacklists I use to temporarily ban IP addresses involved in port scanning, D.O.S and other attacks.

A practical example, we will create a 3 minutes ban blacklist :
iptables -A  thyl-blacklist -m recent --name blacklist_180 --rcheck --seconds 180 -m comment --comment "Drop packet from IP inserted in blacklist last 180 sec" -j DROP
This will create a blacklist that is located in the /proc filesystem in /proc/net/xt_recent/blacklist_180 (or /proc/net/ipt_recent/blacklist_180 ). You can have a look at the blacklist by doing :
cat /proc/net/xt_recent/blacklist_180
A packet can be automatically dropped and source IP added in that blacklist using the next rule :
iptables -A INPUT <IPTABLES FILTERING OPTIONS> --name blacklist_180 --set -j DROP
You can also manually add an IP address to the blacklist, example :
echo + > /proc/net/xt_recent/blacklist_180
To manually remove an IP address :
echo - > /proc/net/xt_recent/blacklist_180

We finished with the basic part. You may now read the second part : "Port scanning"