Initial Commit

UFW-Blocklist/README.md

@@ -0,0 +1,103 @@
+# ufw-blocklist
+Add an IP blocklist to ufw, the uncomplicated Ubuntu firewall
+* integrates into ufw for pure Ubuntu
+* blocks inbound, outbound and forwarding packets
+* uses [Linux ipsets](https://ipset.netfilter.org/) for kernel-grade performance
+* the IP blocklist is refreshed daily
+* the IP blocklist is sourced from [IPsum](https://github.com/stamparm/ipsum)
+* ufw-blocklist is tested on:
+  * Armbian 22.05.3 Focal (based on Ubuntu 20.04.4 LTS (Focal Fossa))
+  * Ubuntu 22.04 LTS (Jammy Jellyfish)
+
+**This blocklist is _very_ successful at dropping a lot of uninvited traffic.** It has been intentionally designed to be very light on resource requirements and zero maintenance as the initial target platform was a single-board computer operating as a home internet gateway. After the initial installation, there are no further writes to the storage system to preserve solid state storage. I would now highly recommend it for any Ubuntu host that has a public IP address or is otherwise exposed directly to the internet, for example, by port forwarding.
+
+# Installation
+Install the ipset package
+```
+sudo apt install ipset
+```
+
+Backup the original ufw `after.init` example script
+```
+sudo cp /etc/ufw/after.init /etc/ufw/after.init.orig
+```
+
+Install the ufw-blocklist files
+```
+git clone https://github.com/poddmo/ufw-blocklist.git
+cd ufw-blocklist
+sudo cp after.init /etc/ufw/after.init
+sudo cp ufw-blocklist-ipsum /etc/cron.daily/ufw-blocklist-ipsum
+sudo chown root.root /etc/ufw/after.init /etc/cron.daily/ufw-blocklist-ipsum
+sudo chmod 750 /etc/ufw/after.init /etc/cron.daily/ufw-blocklist-ipsum
+```
+
+Download an initial IP blocklist from [IPsum](https://github.com/stamparm/ipsum)
+```
+curl -sS -f --compressed -o ipsum.4.txt 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/4.txt'
+sudo chmod 640 ipsum.4.txt
+sudo cp ipsum.4.txt /etc/ipsum.4.txt
+```
+Start ufw-blocklist
+```
+sudo /etc/ufw/after.init start
+```
+It takes time to load the blocklist entries into the ipset. Watch the progress with 
+```
+sudo ipset list ufw-blocklist-ipsum -terse | grep 'Number of entries'
+```
+
+# Usage
+The blocklist is automatically started and stopped by ufw using the enable, disable and reload options. See the [Ubuntu UFW wiki page](https://help.ubuntu.com/community/UFW) for help getting started with ufw.
+
+There are 2 additional `after.init` commands available: status and flush-all
+- The **status** option shows the count of entries in the blocklist, the hit count of packets that have been blocked and the last 10 log entries. The status option is further explained in the [Status](#status) section below.
+- The **flush-all** option deletes all entries in the blocklist and zeros the iptables hit counters:
+```
+sudo /etc/ufw/after.init flush-all
+```
+From this state you can manually add IP addresses to the list like this:
+```
+sudo ipset add ufw-blocklist-ipsum a.b.c.d
+```
+This is useful for testing. Use `/etc/cron.daily/ufw-blocklist-ipsum` to download the latest list and fully restore the blocklist.
+
+# Status
+Calling `after.init` with the status option displays the current count of the entries in the blocklist, the hit counts on the firewall rules (column 1 is hits, column 2 is bytes) and the last 10 log messages. Here is a sample output:
+```
+user@ubunturouter:~# sudo /etc/ufw/after.init status
+Name: ufw-blocklist-ipsum
+Type: hash:net
+Revision: 6
+Header: family inet hashsize 4096 maxelem 65536
+Size in memory: 357312
+References: 3
+Number of entries: 12789
+   76998  4403836 ufw-blocklist-input  all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set ufw-blocklist-ipsum src
+       4      160 ufw-blocklist-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set ufw-blocklist-ipsum dst
+      11      868 ufw-blocklist-output  all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set ufw-blocklist-ipsum dst
+Sep 24 06:25:01 ubunturouter ufw-blocklist-ipsum[535172]: starting update of ufw-blocklist-ipsum with 12654 entries from https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt
+Sep 24 06:26:02 ubunturouter ufw-blocklist-ipsum[547387]: finished updating ufw-blocklist-ipsum. Old entry count: 12654 New count: 12181 of 12181
+Sep 24 22:23:21 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=194.165.16.37 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65058 WINDOW=0 RES=0x00 ACK RST URGP=0
+Sep 25 06:25:02 ubunturouter ufw-blocklist-ipsum[598717]: starting update of ufw-blocklist-ipsum with 12181 entries from https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt
+Sep 25 06:26:07 ubunturouter ufw-blocklist-ipsum[611761]: finished updating ufw-blocklist-ipsum. Old entry count: 12181 New count: 13008 of 13008
+Sep 25 21:19:42 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=45.227.254.8 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65469 WINDOW=0 RES=0x00 ACK RST URGP=0
+Sep 25 21:19:45 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=45.227.254.8 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65469 WINDOW=0 RES=0x00 ACK RST URGP=0
+Sep 25 21:19:51 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=45.227.254.8 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65469 WINDOW=0 RES=0x00 ACK RST URGP=0
+Sep 26 06:25:02 ubunturouter ufw-blocklist-ipsum[661335]: starting update of ufw-blocklist-ipsum with 13008 entries from https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt
+Sep 26 06:26:06 ubunturouter ufw-blocklist-ipsum[674158]: finished updating ufw-blocklist-ipsum. Old entry count: 13008 New count: 12789 of 12789
+```
+- Hits on the OUTPUT or FORWARD drop rules may indicate an issue with an internal host and are logged. In the example status shown above, the hits on the FORWARD rule are related to an internal torrent client.
+- INPUT hits are not logged. The status output above shows **76998 dropped INPUT packets** after the system has been up 9 days, 22:45 hours.
+
+# Todo
+These scripts have run flawlessly for 2 years. The next steps will take advantage of this extended ufw-framework and generalise the blocklist case to arbitrary ipsets, for example, to block bogans or by geoblock
+- test and document use of after.init_run-parts
+- test and document geo-block example for blocking geographic subnets. Geo-based blocks are useful for blocking botnets or "citizen activists." Geo-based subnets can be found at:
+  - https://www.ip2location.com/free/visitor-blocker
+  - https://www.ipdeny.com/ipblocks/
+- test and document blocking bogan IP addresses. Bogon lists can be found at:
+  - FireHOL includes fullbogons: https://iplists.firehol.org/
+  - so does team Cymru. See fullbogons at: https://www.team-cymru.com/bogon-reference-http
+- develop a whitelist an ip/cidr address
+- develop test of entries as valid ip/cidr addresses

UFW-Blocklist/after.init

@@ -0,0 +1,176 @@
+#!/bin/bash
+#
+# after.init: if executable, called by ufw-init. See 'man ufw-framework' for
+#             details. Note that output from these scripts is not seen via the
+#             the ufw command, but instead via ufw-init.
+#
+# Copyright 2013 Canonical Ltd.
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3,
+#    as published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# ufw-blocklist edition: IP blocklist extension for Ubuntu ufw
+#  https://github.com/poddmo/ufw-blocklist
+#
+set -e
+
+export ipsetname=ufw-blocklist-ipsum
+
+# seed file containing the list of IP addresses to be blocked, one per line
+# curl -sS -f --compressed 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/4.txt' > /etc/ipsum.4.txt
+# ipset is updated daily by /etc/cron.daily/ufw-blocklist-ipsum
+export seedlist=/etc/ipsum.4.txt
+
+export IPSET_EXE="/sbin/ipset"
+# check ipset exists and is executable
+[ -x "$IPSET_EXE" ] || {
+    echo "$IPSET_EXE is not executable"
+    exit 1
+    }
+
+
+# Function to check if a chain exists (chain_exists ufw_blocklist_input && action if true || action if false)
+chain_exists() {
+    {   
+        [ $# -lt 1 ] || [ $# -gt 2 ] && { 
+            echo "Usage: chain_exists <chain_name> [table]" >&2
+            return 1
+        }
+        
+        local chain_name="$1" ; shift
+        [ $# -eq 1 ] && local table="--table $1"
+        iptables "$table" -n --list "$chain_name" >/dev/null 2>&1
+    }
+}
+
+# Function to check if an set exists (set_exists setname && action if true || action if false)
+set_exists() {
+    {   
+        [ $# -ne 1 ] && { 
+            echo "Usage: set_exists <set_name>" >&2
+            return 1
+        }
+        
+        local set_name="$1"
+        ipset list "$set_name" -name >/dev/null 2>&1
+    }
+}
+
+case "$1" in
+start)
+    # check that blocklist seed file exists
+    if [ ! -f "$seedlist" ]; then
+        echo "ufw after.init: $seedlist does not exist."
+        exit 1
+    fi
+
+    # create an empty ipset
+    $IPSET_EXE create  $ipsetname hash:net -exist
+    $IPSET_EXE flush   $ipsetname
+    
+    ## Insert firewall rules to take precedence, removing them and adding them back if they already existed
+    # Block inbound to localhost from blocklist
+    if chain_exists ufw-blocklist-input; then
+        iptables -D INPUT -m set --match-set $ipsetname src -j ufw-blocklist-input || true
+        iptables -F ufw-blocklist-input
+        iptables -X ufw-blocklist-input
+    fi
+
+    iptables -N ufw-blocklist-input
+    iptables -A ufw-blocklist-input -j DROP -m comment --comment "ufw-blocklist-input"
+    iptables -I INPUT -m set --match-set $ipsetname src -j ufw-blocklist-input
+
+    # Log and drop outbound to blocklist. Hits here may indicate compromised localhost
+    if chain_exists ufw-blocklist-output; then
+        iptables -D OUTPUT -m set --match-set $ipsetname dst -j ufw-blocklist-output || true
+        iptables -F ufw-blocklist-output
+        iptables -X ufw-blocklist-output
+    fi
+
+    iptables -N ufw-blocklist-output
+    iptables -A ufw-blocklist-output -j LOG --log-level 3 --log-prefix "[UFW BLOCKLIST OUTPUT] " -m limit --limit 3/minute --limit-burst 10
+    iptables -A ufw-blocklist-output -j DROP -m comment --comment "ufw-blocklist-output"
+    iptables -I OUTPUT -m set --match-set $ipsetname dst -j ufw-blocklist-output
+
+    # Log and drop forwarding to blocklist. Hits here may indicate compromised internal hosts
+    if chain_exists ufw-blocklist-forward; then
+        iptables -D FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward || true
+        iptables -F ufw-blocklist-forward
+        iptables -X ufw-blocklist-forward
+    fi
+
+    iptables -N ufw-blocklist-forward
+    iptables -A ufw-blocklist-forward -j LOG --log-level 3 --log-prefix "[UFW BLOCKLIST FORWARD] " -m limit --limit 3/minute --limit-burst 10
+    iptables -A ufw-blocklist-forward -j DROP -m comment --comment "ufw-blocklist-forward"
+    iptables -I FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward
+
+    # add members to the ipset
+    # start this in a subshell and then disown the job so we return quickly.
+    (
+    while read -r ip < "$seedlist"
+        do
+            $IPSET_EXE add "$ipsetname" "$ip"
+        done
+    ) < /dev/null &> /dev/null & disown -h
+    ;;
+stop)
+
+    # delete resources created above
+    if chain_exists ufw-blocklist-input; then
+        iptables -D INPUT -m set --match-set $ipsetname src -j ufw-blocklist-input || true
+        iptables -F ufw-blocklist-input
+        iptables -X ufw-blocklist-input
+    fi
+    if chain_exists ufw-blocklist-output; then
+        iptables -D OUTPUT -m set --match-set $ipsetname dst -j ufw-blocklist-output || true
+        iptables -F ufw-blocklist-output
+        iptables -X ufw-blocklist-output
+    fi
+    if chain_exists ufw-blocklist-forward; then
+        iptables -D FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward || true
+        iptables -F ufw-blocklist-forward
+        iptables -X ufw-blocklist-forward
+    fi
+    if set_exists $ipsetname; then
+	$IPSET_EXE flush   $ipsetname
+	$IPSET_EXE destroy $ipsetname
+    fi
+    ;;
+status)
+    # display details of the ipset
+    $IPSET_EXE list "$ipsetname" -t
+    # show iptables hit/byte counts
+    iptables -L -nvx | grep "$ipsetname" | grep 'match-set'
+    # show the last 10 lines from the logs
+    journalctl | grep -i blocklist | tail
+    ;;
+flush-all)
+    # flush sets created above. Use /etc/cron.daily/ufw-blocklist-ipsum to repopulate
+    $IPSET_EXE flush  $ipsetname
+    # reset iptables accounting
+    ipz=$( iptables -L INPUT -nvx --line-numbers | grep ufw-blocklist-input | awk '{print $1}')
+    iptables -Z INPUT "$ipz"
+    iptables -Z ufw-blocklist-input
+
+    ipz=$( iptables -L OUTPUT -nvx --line-numbers | grep ufw-blocklist-output | awk '{print $1}')
+    iptables -Z OUTPUT "$ipz"
+    iptables -Z ufw-blocklist-output
+
+    ipz=$( iptables -L FORWARD -nvx --line-numbers | grep ufw-blocklist-forward | awk '{print $1}')
+    iptables -Z FORWARD "$ipz"
+    iptables -Z ufw-blocklist-forward
+    ;;
+*)
+    echo "'$1' not supported"
+    echo "Usage: /etc/ufw/after.init {start|stop|flush-all|status}"
+    ;;
+esac

UFW-Blocklist/after.init_run-parts

@@ -0,0 +1,64 @@
+#!/bin/sh
+#
+# after.init: if executable, called by ufw-init. See 'man ufw-framework' for
+#             details. Note that output from these scripts is not seen via the
+#             the ufw command, but instead via ufw-init.
+#
+# Copyright 2013 Canonical Ltd.
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3,
+#    as published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# ufw-blocklist dynamic edition: IP blocklist extension for Ubuntu ufw
+#  https://github.com/poddmo/ufw-blocklist
+#
+# Install this script executable as /etc/ufw/after.init
+# Install user scripts to be run after ufw has initialised into /etc/ufw/after.init.d/
+# ** Scripts will run with ufw privilege, ie root **
+# Script names must match the filename format: numbernumber-*.ufw
+# Order of execution is critical. Lower (00) numbered scripts will execute first
+# Higher numbered scripts have higher precedence
+# Example filenames: 10-ipblocklist-ipsum.ufw, 20-subnetblocklist.ufw
+#
+set -e
+
+afterinitdir='/etc/ufw/after.init.d'
+if [ ! -d "$afterinitdir" ];
+then 
+	echo "$afterinitdir does not exist. nothing to do"
+	exit 0
+fi
+
+runpartsfunc ()
+{
+	run-parts --report --regex='^[0-9]{2}-.*.ufw$' --arg="$1" "${afterinitdir}"
+	exit $?
+}
+
+case "$1" in
+start)
+    runpartsfunc start
+    ;;
+stop)
+    runpartsfunc stop
+    ;;
+status)
+    runpartsfunc status
+    ;;
+flush-all)
+    runpartsfunc flush-all
+    ;;
+*)
+    echo "'$1' not supported"
+    echo "Usage: after.init {start|stop|flush-all|status}"
+    ;;
+esac

UFW-Blocklist/ufw-blocklist-ipsum

@@ -0,0 +1,111 @@
+#!/bin/bash
+
+#
+# This script will download the latest ipsum IP address list and add the
+# addresses to the ufw-blocklist ipset
+# Use memory instead of file system to reduce writes to sd card
+
+# todo: better validation of new list, eg is valid IP address
+#       whitelist to exclude from adding to the set
+
+# install this file into /etc/cron.daily/ufw-blocklist-ipsum
+
+## URL must return a text file with one IP address per line
+ipsumurl='https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt'
+# reject the new list if there are fewer than minlen number of ip addresses
+minlen=1000
+
+ipsetname=ufw-blocklist-ipsum
+ipset_exe=/usr/sbin/ipset
+logger="/usr/bin/logger -t ${ipsetname}"
+
+## Check if ipsetname exists. exit if not - ie no set to update
+ipsetstatus=$("${ipset_exe}" -t list "${ipsetname}" 2>/dev/null )
+RET=$?
+if [ $RET -ne 0 ]; then
+	$logger -s "ipset named $ipsetname does not exist. is UFW started? exiting"
+	exit 1
+fi
+
+ipsetcount=$(echo "$ipsetstatus" | grep '^Number of entries:' | cut -d' ' -f4)
+
+$logger "starting update of ${ipsetname} with ${ipsetcount} entries from ${ipsumurl}"
+
+## Download the latest list
+rawlist=$(curl -sS -f --compressed "$ipsumurl" 2>/dev/null)
+RET=$?
+if [ $RET -ne 0 ]; then
+	$logger -s "curl error code $RET for $ipsumurl"
+	exit 1
+fi
+
+## Read the list into an array
+declare -a scrublist
+readarray -t scrublist < <(echo "$rawlist")
+
+
+## Validate the list length
+scrublistlen="${#scrublist[@]}"
+#echo "length of scrublist array: $scrublistlen"
+if [ $scrublistlen -lt $minlen ]; then
+	$logger -s "$scrublistlen less than $minlen IPs. something must be wrong with $ipsumurl"
+	exit 1
+fi
+
+## define a function to validate ip addresses
+isvalidip () {
+	# True=valid IP, false=not valid IP
+	# use BASH regex matching to reduce forking IO
+	#echo -n "$i "
+	# if [[ ! "$i" =~ ^((25[0-5]|(2[0-4]|1\d|[1-9]|)\d)\.?\b){4}$ ]]; then
+	#if [[ "$i" =~ ^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$ ]]; then
+	#	echo 'invalid ip address'
+	return
+}
+
+## create a temporary ipset
+tmpsetname="$(mktemp -u | cut -f2 -d'.')-tmp"
+$ipset_exe -q create "$tmpsetname" hash:net
+RET=$?
+if [ $RET -ne 0 ]; then
+        $logger -s "error code $RET creating temporary ipset $tmpsetname"
+	$ipset_exe -q destroy "$tmpsetname"
+        exit 1
+fi
+
+
+
+## loop through each IP address in the scrublist array and add it to the temporary ipset
+cnt=0
+for i in "${scrublist[@]}"
+do
+	## Validate IP address is correct format
+	# if not valid ip
+	#	log, cleanup and exit
+	# fi
+
+        # Add that IP to the ipset blocklist
+        #echo -e "Adding $i to ipset blocklist...\n"
+        $ipset_exe add "$tmpsetname" $i
+	cnt=$((cnt+1))
+done
+
+
+## ipset swap FROM-SETNAME TO-SETNAME
+##        Swap the content of two existing sets
+$ipset_exe swap "$tmpsetname" "$ipsetname"
+RET=$?
+if [ $RET -ne 0 ]; then
+        $logger -s "error code $RET ipset swapping $tmpsetname to $ipsetname"
+	$ipset_exe -q destroy "$tmpsetname"
+        exit 1
+fi
+
+$ipset_exe -q destroy "$tmpsetname"
+RET=$?
+if [ $RET -ne 0 ]; then
+        $logger -s "error code $RET destroying ipset $tmpsetname"
+        exit 1
+fi
+
+$logger "finished updating $ipsetname. Old entry count: $ipsetcount New count: $cnt of $scrublistlen"