Initial Commit

This commit is contained in:
Phil Connor 2024-06-13 11:27:27 -05:00
parent 34a19f6c1b
commit c052ec3a70
4 changed files with 454 additions and 0 deletions

103
UFW-Blocklist/README.md Normal file
View File

@ -0,0 +1,103 @@
# ufw-blocklist
Add an IP blocklist to ufw, the uncomplicated Ubuntu firewall
* integrates into ufw for pure Ubuntu
* blocks inbound, outbound and forwarding packets
* uses [Linux ipsets](https://ipset.netfilter.org/) for kernel-grade performance
* the IP blocklist is refreshed daily
* the IP blocklist is sourced from [IPsum](https://github.com/stamparm/ipsum)
* ufw-blocklist is tested on:
* Armbian 22.05.3 Focal (based on Ubuntu 20.04.4 LTS (Focal Fossa))
* Ubuntu 22.04 LTS (Jammy Jellyfish)
**This blocklist is _very_ successful at dropping a lot of uninvited traffic.** It has been intentionally designed to be very light on resource requirements and zero maintenance as the initial target platform was a single-board computer operating as a home internet gateway. After the initial installation, there are no further writes to the storage system to preserve solid state storage. I would now highly recommend it for any Ubuntu host that has a public IP address or is otherwise exposed directly to the internet, for example, by port forwarding.
# Installation
Install the ipset package
```
sudo apt install ipset
```
Backup the original ufw `after.init` example script
```
sudo cp /etc/ufw/after.init /etc/ufw/after.init.orig
```
Install the ufw-blocklist files
```
git clone https://github.com/poddmo/ufw-blocklist.git
cd ufw-blocklist
sudo cp after.init /etc/ufw/after.init
sudo cp ufw-blocklist-ipsum /etc/cron.daily/ufw-blocklist-ipsum
sudo chown root.root /etc/ufw/after.init /etc/cron.daily/ufw-blocklist-ipsum
sudo chmod 750 /etc/ufw/after.init /etc/cron.daily/ufw-blocklist-ipsum
```
Download an initial IP blocklist from [IPsum](https://github.com/stamparm/ipsum)
```
curl -sS -f --compressed -o ipsum.4.txt 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/4.txt'
sudo chmod 640 ipsum.4.txt
sudo cp ipsum.4.txt /etc/ipsum.4.txt
```
Start ufw-blocklist
```
sudo /etc/ufw/after.init start
```
It takes time to load the blocklist entries into the ipset. Watch the progress with
```
sudo ipset list ufw-blocklist-ipsum -terse | grep 'Number of entries'
```
# Usage
The blocklist is automatically started and stopped by ufw using the enable, disable and reload options. See the [Ubuntu UFW wiki page](https://help.ubuntu.com/community/UFW) for help getting started with ufw.
There are 2 additional `after.init` commands available: status and flush-all
- The **status** option shows the count of entries in the blocklist, the hit count of packets that have been blocked and the last 10 log entries. The status option is further explained in the [Status](#status) section below.
- The **flush-all** option deletes all entries in the blocklist and zeros the iptables hit counters:
```
sudo /etc/ufw/after.init flush-all
```
From this state you can manually add IP addresses to the list like this:
```
sudo ipset add ufw-blocklist-ipsum a.b.c.d
```
This is useful for testing. Use `/etc/cron.daily/ufw-blocklist-ipsum` to download the latest list and fully restore the blocklist.
# Status
Calling `after.init` with the status option displays the current count of the entries in the blocklist, the hit counts on the firewall rules (column 1 is hits, column 2 is bytes) and the last 10 log messages. Here is a sample output:
```
user@ubunturouter:~# sudo /etc/ufw/after.init status
Name: ufw-blocklist-ipsum
Type: hash:net
Revision: 6
Header: family inet hashsize 4096 maxelem 65536
Size in memory: 357312
References: 3
Number of entries: 12789
76998 4403836 ufw-blocklist-input all -- * * 0.0.0.0/0 0.0.0.0/0 match-set ufw-blocklist-ipsum src
4 160 ufw-blocklist-forward all -- * * 0.0.0.0/0 0.0.0.0/0 match-set ufw-blocklist-ipsum dst
11 868 ufw-blocklist-output all -- * * 0.0.0.0/0 0.0.0.0/0 match-set ufw-blocklist-ipsum dst
Sep 24 06:25:01 ubunturouter ufw-blocklist-ipsum[535172]: starting update of ufw-blocklist-ipsum with 12654 entries from https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt
Sep 24 06:26:02 ubunturouter ufw-blocklist-ipsum[547387]: finished updating ufw-blocklist-ipsum. Old entry count: 12654 New count: 12181 of 12181
Sep 24 22:23:21 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=194.165.16.37 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65058 WINDOW=0 RES=0x00 ACK RST URGP=0
Sep 25 06:25:02 ubunturouter ufw-blocklist-ipsum[598717]: starting update of ufw-blocklist-ipsum with 12181 entries from https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt
Sep 25 06:26:07 ubunturouter ufw-blocklist-ipsum[611761]: finished updating ufw-blocklist-ipsum. Old entry count: 12181 New count: 13008 of 13008
Sep 25 21:19:42 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=45.227.254.8 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65469 WINDOW=0 RES=0x00 ACK RST URGP=0
Sep 25 21:19:45 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=45.227.254.8 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65469 WINDOW=0 RES=0x00 ACK RST URGP=0
Sep 25 21:19:51 ubunturouter kernel: [UFW BLOCKLIST FORWARD] IN=eth1 OUT=ppp0 MAC=11:22:33:44:55:66:77:88:99:00:aa:bb:cc:dd SRC=192.168.1.11 DST=45.227.254.8 LEN=40 TOS=0x00 PREC=0x00 TTL=62 ID=0 DF PROTO=TCP SPT=51413 DPT=65469 WINDOW=0 RES=0x00 ACK RST URGP=0
Sep 26 06:25:02 ubunturouter ufw-blocklist-ipsum[661335]: starting update of ufw-blocklist-ipsum with 13008 entries from https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt
Sep 26 06:26:06 ubunturouter ufw-blocklist-ipsum[674158]: finished updating ufw-blocklist-ipsum. Old entry count: 13008 New count: 12789 of 12789
```
- Hits on the OUTPUT or FORWARD drop rules may indicate an issue with an internal host and are logged. In the example status shown above, the hits on the FORWARD rule are related to an internal torrent client.
- INPUT hits are not logged. The status output above shows **76998 dropped INPUT packets** after the system has been up 9 days, 22:45 hours.
# Todo
These scripts have run flawlessly for 2 years. The next steps will take advantage of this extended ufw-framework and generalise the blocklist case to arbitrary ipsets, for example, to block bogans or by geoblock
- test and document use of after.init_run-parts
- test and document geo-block example for blocking geographic subnets. Geo-based blocks are useful for blocking botnets or "citizen activists." Geo-based subnets can be found at:
- https://www.ip2location.com/free/visitor-blocker
- https://www.ipdeny.com/ipblocks/
- test and document blocking bogan IP addresses. Bogon lists can be found at:
- FireHOL includes fullbogons: https://iplists.firehol.org/
- so does team Cymru. See fullbogons at: https://www.team-cymru.com/bogon-reference-http
- develop a whitelist an ip/cidr address
- develop test of entries as valid ip/cidr addresses

176
UFW-Blocklist/after.init Normal file
View File

@ -0,0 +1,176 @@
#!/bin/bash
#
# after.init: if executable, called by ufw-init. See 'man ufw-framework' for
# details. Note that output from these scripts is not seen via the
# the ufw command, but instead via ufw-init.
#
# Copyright 2013 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3,
# as published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# ufw-blocklist edition: IP blocklist extension for Ubuntu ufw
# https://github.com/poddmo/ufw-blocklist
#
set -e
export ipsetname=ufw-blocklist-ipsum
# seed file containing the list of IP addresses to be blocked, one per line
# curl -sS -f --compressed 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/4.txt' > /etc/ipsum.4.txt
# ipset is updated daily by /etc/cron.daily/ufw-blocklist-ipsum
export seedlist=/etc/ipsum.4.txt
export IPSET_EXE="/sbin/ipset"
# check ipset exists and is executable
[ -x "$IPSET_EXE" ] || {
echo "$IPSET_EXE is not executable"
exit 1
}
# Function to check if a chain exists (chain_exists ufw_blocklist_input && action if true || action if false)
chain_exists() {
{
[ $# -lt 1 ] || [ $# -gt 2 ] && {
echo "Usage: chain_exists <chain_name> [table]" >&2
return 1
}
local chain_name="$1" ; shift
[ $# -eq 1 ] && local table="--table $1"
iptables "$table" -n --list "$chain_name" >/dev/null 2>&1
}
}
# Function to check if an set exists (set_exists setname && action if true || action if false)
set_exists() {
{
[ $# -ne 1 ] && {
echo "Usage: set_exists <set_name>" >&2
return 1
}
local set_name="$1"
ipset list "$set_name" -name >/dev/null 2>&1
}
}
case "$1" in
start)
# check that blocklist seed file exists
if [ ! -f "$seedlist" ]; then
echo "ufw after.init: $seedlist does not exist."
exit 1
fi
# create an empty ipset
$IPSET_EXE create $ipsetname hash:net -exist
$IPSET_EXE flush $ipsetname
## Insert firewall rules to take precedence, removing them and adding them back if they already existed
# Block inbound to localhost from blocklist
if chain_exists ufw-blocklist-input; then
iptables -D INPUT -m set --match-set $ipsetname src -j ufw-blocklist-input || true
iptables -F ufw-blocklist-input
iptables -X ufw-blocklist-input
fi
iptables -N ufw-blocklist-input
iptables -A ufw-blocklist-input -j DROP -m comment --comment "ufw-blocklist-input"
iptables -I INPUT -m set --match-set $ipsetname src -j ufw-blocklist-input
# Log and drop outbound to blocklist. Hits here may indicate compromised localhost
if chain_exists ufw-blocklist-output; then
iptables -D OUTPUT -m set --match-set $ipsetname dst -j ufw-blocklist-output || true
iptables -F ufw-blocklist-output
iptables -X ufw-blocklist-output
fi
iptables -N ufw-blocklist-output
iptables -A ufw-blocklist-output -j LOG --log-level 3 --log-prefix "[UFW BLOCKLIST OUTPUT] " -m limit --limit 3/minute --limit-burst 10
iptables -A ufw-blocklist-output -j DROP -m comment --comment "ufw-blocklist-output"
iptables -I OUTPUT -m set --match-set $ipsetname dst -j ufw-blocklist-output
# Log and drop forwarding to blocklist. Hits here may indicate compromised internal hosts
if chain_exists ufw-blocklist-forward; then
iptables -D FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward || true
iptables -F ufw-blocklist-forward
iptables -X ufw-blocklist-forward
fi
iptables -N ufw-blocklist-forward
iptables -A ufw-blocklist-forward -j LOG --log-level 3 --log-prefix "[UFW BLOCKLIST FORWARD] " -m limit --limit 3/minute --limit-burst 10
iptables -A ufw-blocklist-forward -j DROP -m comment --comment "ufw-blocklist-forward"
iptables -I FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward
# add members to the ipset
# start this in a subshell and then disown the job so we return quickly.
(
while read -r ip < "$seedlist"
do
$IPSET_EXE add "$ipsetname" "$ip"
done
) < /dev/null &> /dev/null & disown -h
;;
stop)
# delete resources created above
if chain_exists ufw-blocklist-input; then
iptables -D INPUT -m set --match-set $ipsetname src -j ufw-blocklist-input || true
iptables -F ufw-blocklist-input
iptables -X ufw-blocklist-input
fi
if chain_exists ufw-blocklist-output; then
iptables -D OUTPUT -m set --match-set $ipsetname dst -j ufw-blocklist-output || true
iptables -F ufw-blocklist-output
iptables -X ufw-blocklist-output
fi
if chain_exists ufw-blocklist-forward; then
iptables -D FORWARD -m set --match-set $ipsetname dst -j ufw-blocklist-forward || true
iptables -F ufw-blocklist-forward
iptables -X ufw-blocklist-forward
fi
if set_exists $ipsetname; then
$IPSET_EXE flush $ipsetname
$IPSET_EXE destroy $ipsetname
fi
;;
status)
# display details of the ipset
$IPSET_EXE list "$ipsetname" -t
# show iptables hit/byte counts
iptables -L -nvx | grep "$ipsetname" | grep 'match-set'
# show the last 10 lines from the logs
journalctl | grep -i blocklist | tail
;;
flush-all)
# flush sets created above. Use /etc/cron.daily/ufw-blocklist-ipsum to repopulate
$IPSET_EXE flush $ipsetname
# reset iptables accounting
ipz=$( iptables -L INPUT -nvx --line-numbers | grep ufw-blocklist-input | awk '{print $1}')
iptables -Z INPUT "$ipz"
iptables -Z ufw-blocklist-input
ipz=$( iptables -L OUTPUT -nvx --line-numbers | grep ufw-blocklist-output | awk '{print $1}')
iptables -Z OUTPUT "$ipz"
iptables -Z ufw-blocklist-output
ipz=$( iptables -L FORWARD -nvx --line-numbers | grep ufw-blocklist-forward | awk '{print $1}')
iptables -Z FORWARD "$ipz"
iptables -Z ufw-blocklist-forward
;;
*)
echo "'$1' not supported"
echo "Usage: /etc/ufw/after.init {start|stop|flush-all|status}"
;;
esac

View File

@ -0,0 +1,64 @@
#!/bin/sh
#
# after.init: if executable, called by ufw-init. See 'man ufw-framework' for
# details. Note that output from these scripts is not seen via the
# the ufw command, but instead via ufw-init.
#
# Copyright 2013 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3,
# as published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# ufw-blocklist dynamic edition: IP blocklist extension for Ubuntu ufw
# https://github.com/poddmo/ufw-blocklist
#
# Install this script executable as /etc/ufw/after.init
# Install user scripts to be run after ufw has initialised into /etc/ufw/after.init.d/
# ** Scripts will run with ufw privilege, ie root **
# Script names must match the filename format: numbernumber-*.ufw
# Order of execution is critical. Lower (00) numbered scripts will execute first
# Higher numbered scripts have higher precedence
# Example filenames: 10-ipblocklist-ipsum.ufw, 20-subnetblocklist.ufw
#
set -e
afterinitdir='/etc/ufw/after.init.d'
if [ ! -d "$afterinitdir" ];
then
echo "$afterinitdir does not exist. nothing to do"
exit 0
fi
runpartsfunc ()
{
run-parts --report --regex='^[0-9]{2}-.*.ufw$' --arg="$1" "${afterinitdir}"
exit $?
}
case "$1" in
start)
runpartsfunc start
;;
stop)
runpartsfunc stop
;;
status)
runpartsfunc status
;;
flush-all)
runpartsfunc flush-all
;;
*)
echo "'$1' not supported"
echo "Usage: after.init {start|stop|flush-all|status}"
;;
esac

View File

@ -0,0 +1,111 @@
#!/bin/bash
#
# This script will download the latest ipsum IP address list and add the
# addresses to the ufw-blocklist ipset
# Use memory instead of file system to reduce writes to sd card
# todo: better validation of new list, eg is valid IP address
# whitelist to exclude from adding to the set
# install this file into /etc/cron.daily/ufw-blocklist-ipsum
## URL must return a text file with one IP address per line
ipsumurl='https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt'
# reject the new list if there are fewer than minlen number of ip addresses
minlen=1000
ipsetname=ufw-blocklist-ipsum
ipset_exe=/usr/sbin/ipset
logger="/usr/bin/logger -t ${ipsetname}"
## Check if ipsetname exists. exit if not - ie no set to update
ipsetstatus=$("${ipset_exe}" -t list "${ipsetname}" 2>/dev/null )
RET=$?
if [ $RET -ne 0 ]; then
$logger -s "ipset named $ipsetname does not exist. is UFW started? exiting"
exit 1
fi
ipsetcount=$(echo "$ipsetstatus" | grep '^Number of entries:' | cut -d' ' -f4)
$logger "starting update of ${ipsetname} with ${ipsetcount} entries from ${ipsumurl}"
## Download the latest list
rawlist=$(curl -sS -f --compressed "$ipsumurl" 2>/dev/null)
RET=$?
if [ $RET -ne 0 ]; then
$logger -s "curl error code $RET for $ipsumurl"
exit 1
fi
## Read the list into an array
declare -a scrublist
readarray -t scrublist < <(echo "$rawlist")
## Validate the list length
scrublistlen="${#scrublist[@]}"
#echo "length of scrublist array: $scrublistlen"
if [ $scrublistlen -lt $minlen ]; then
$logger -s "$scrublistlen less than $minlen IPs. something must be wrong with $ipsumurl"
exit 1
fi
## define a function to validate ip addresses
isvalidip () {
# True=valid IP, false=not valid IP
# use BASH regex matching to reduce forking IO
#echo -n "$i "
# if [[ ! "$i" =~ ^((25[0-5]|(2[0-4]|1\d|[1-9]|)\d)\.?\b){4}$ ]]; then
#if [[ "$i" =~ ^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$ ]]; then
# echo 'invalid ip address'
return
}
## create a temporary ipset
tmpsetname="$(mktemp -u | cut -f2 -d'.')-tmp"
$ipset_exe -q create "$tmpsetname" hash:net
RET=$?
if [ $RET -ne 0 ]; then
$logger -s "error code $RET creating temporary ipset $tmpsetname"
$ipset_exe -q destroy "$tmpsetname"
exit 1
fi
## loop through each IP address in the scrublist array and add it to the temporary ipset
cnt=0
for i in "${scrublist[@]}"
do
## Validate IP address is correct format
# if not valid ip
# log, cleanup and exit
# fi
# Add that IP to the ipset blocklist
#echo -e "Adding $i to ipset blocklist...\n"
$ipset_exe add "$tmpsetname" $i
cnt=$((cnt+1))
done
## ipset swap FROM-SETNAME TO-SETNAME
## Swap the content of two existing sets
$ipset_exe swap "$tmpsetname" "$ipsetname"
RET=$?
if [ $RET -ne 0 ]; then
$logger -s "error code $RET ipset swapping $tmpsetname to $ipsetname"
$ipset_exe -q destroy "$tmpsetname"
exit 1
fi
$ipset_exe -q destroy "$tmpsetname"
RET=$?
if [ $RET -ne 0 ]; then
$logger -s "error code $RET destroying ipset $tmpsetname"
exit 1
fi
$logger "finished updating $ipsetname. Old entry count: $ipsetcount New count: $cnt of $scrublistlen"