Old:UpHosts
Vai alla navigazione
Vai alla ricerca
Attenzione. Questa guida è obsoleta. Viene mantenuta sul Wiki solo per motivi di natura storica e didattica. |
Introduzione
Premessa: esistono svariati sistemi di content filtering decisamente più raffinati e personalizzabili di questo.
Scopo di questo script è semplicemente bloccare l'accesso a hosts a vario titolo "malevoli" basandosi su liste disponibili online.
Tali host vengono inseriti in /etc/hosts come corrispondenti a 0.0.0.0 , bloccandone di fatto l'accesso.
Paolo321 , 31 dicembre 2010
Funzionamento
Spero che lo script sia sufficientemente chiaro e si "spieghi da solo", comunque lo script esegue:
- Controllo privilegi di root e presenza software richiesti
- Controllo di quanto tempo è passato dall'ultima esecuzione (per evitare download troppo frequenti, vedi esempio sotto)
- Backup dell'hosts file presistente (se non già effettuato)
- Download delle liste
- Adattamento del formato delle liste
- Ogni lista richiede differenti azioni, questo purtroppo implica che le liste sono hard-coded e per aggiungerne o toglierne una è necessario modificare buona parte dello script
- Lettura blacklist locale
- Merge di tutte le liste, con ordinamento e eliminazione duplicati
- Lettura whitelist locale e rimozione dalla blacklist degli host presenti
- Creazione del nuovo file hosts
- Importazione del file hosts di backup
- Questo è necessario per garantire che host precedentemente presenti in /etc/hosts non vengano rimossi
- A ogni esecuzione dello script il precedente file hosts è sovrascritto, e il file hosts di backup è reimportato
- Eventuali modifiche permanenti al file hosts andranno quindi effettuate non in /etc/hosts ma nel file hosts di backup
- Report numero hosts bloccati e uscita dello script
Note
- Come default lo script cerca blacklist e whitelist locali nella sua directory, quindi si può ad esempio collocare il tutto in /opt/uphosts. Ovviamente si può cambiare i path nello script e usarne altri.
- Uno script di questo tipo è comodo sia eseguito "unattended"; iIl wrapper zzz-uphosts-run esegue uphosts e reindirizza l'output nei log di sistema. Può ad esempio essere eseguito da /etc/network/if-up.d, oppure tramite cron (ma in questo secondo caso c'è da aggiungere un controllo sulla presenza di connessione internet). Lo script all'avvio controlla comunque quanto tempo è passato dall'ultima esecuzione (per evitare che ad esempio dieci connessioni-disconnessioni al giorno portino a dieci download delle liste)
Script
uphosts.sh
#!/bin/bash # uphosts - Hosts File Updater # README: # Bad hosts are blocked putting them in the hosts file as 0.0.0.0 # To add other sources script must be manually modified # Permanent entries must be added to the original file # THIS SCRIPT HAS NO WARRANTY ! # Thanks to: # http://ubuntuedintorni.wordpress.com/2009/06/29/di-script-dns-e-file-host/ # http://hostsfile.mine.nu/downloads/updatehosts.sh.txt # 20101216 Paolo #----------------------------------------------------------------------- HOSTSPATH="/tmp/hosts-`date +%s`" # Temp directory HOSTSFILE="/etc/hosts" # Hosts file ORIGFILE="$HOSTSFILE.original" # Backup file CONFDIR="$(dirname $(readlink -f $0))" # Parent directory of the script BLACKLIST="$CONFDIR/uphosts-blacklist" # Local Blacklist WHITELIST="$CONFDIR/uphosts-whitelist" # Whitelist PROXYUSER="" #PROXYUSER="--proxy-user=user.name" PROXYPASS="" #PROXYPASS="--proxy-password='password" DAYS="2" # Update frequency #----------------------------------------------------------------------- # Checks for root privileges if [ "$(whoami)" != 'root' ] ; then echo "You need to be root to execute uphosts. Exiting!" exit 1 fi # Checks required packages ABORT=0 builtin type -P wget &>/dev/null || { echo -n "wget is missing."; ABORT=1; } builtin type -P unzip &>/dev/null || { echo -n "unzip is missing."; ABORT=1; } builtin type -P fromdos &>/dev/null || { echo -n "fromdos(tofrodos) is missing."; ABORT=1; } builtin type -P grep &>/dev/null || { echo -n "grep is missing."; ABORT=1; } if [ $ABORT != 0 ] ; then echo " Exiting!" exit 2 fi # Limits updates if uphosts is run often (i.e. at every if-up) # If there is no original hosts file this is the first run on a fresh system, and update runs anyway if [ -f "$ORIGFILE" ] && [ `find $HOSTSFILE -mtime -$DAYS` ] ; then echo "$HOSTSFILE is less than $DAYS days old. Exiting!" exit 3 fi # If there is no original hosts file this is the first run on a fresh system # (as above, but now original hosts file is saved) if [ ! -f "$ORIGFILE" ] ; then echo "Backing up your previous hosts file ..." cp $HOSTSFILE $ORIGFILE #echo "OK" fi #----------------------------------------------------------------------- mkdir $HOSTSPATH # Every list has some specific tweaks :-) # If there are any errors, the script exits. Merging lists partially could be unsafe #### hphosts list HFNAME1="hphosts" HFSERVER1="http://support.it-mate.co.uk/downloads" HFILE1="hphosts.zip" HFILE1INT="HOSTS.txt" # Needed to specify which file from zip # Downloading echo -n "Retrieving $HFNAME1 from $HFSERVER1 ..." wget -q -O $HOSTSPATH/$HFNAME1 $HFSERVER1/$HFILE1 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 11; } unzip -p $HOSTSPATH/$HFNAME1 $HFILE1INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME1 echo " OK" #### hphosts-partial list HFNAME2="hphosts-partial" HFSERVER2="http://www.hosts-file.net" HFILE2="hphosts-partial.asp" # Downloading echo -n "Retrieving $HFNAME2 from $HFSERVER2 ..." wget -q -O $HOSTSPATH/$HFNAME2 $HFSERVER2/$HFILE2 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 12; } cat $HOSTSPATH/$HFNAME2 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME2 echo " OK" #### MVPs list HFNAME3="mvps" HFSERVER3="http://www.mvps.org/winhelp2002" HFILE3="hosts.zip" HFILE3INT="HOSTS" # Needed to specify which file from zip # Downloading echo -n "Retrieving $HFNAME3 from $HFSERVER3 ..." wget -q -O $HOSTSPATH/$HFNAME3 $HFSERVER3/$HFILE3 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 13; } unzip -p $HOSTSPATH/$HFNAME3 $HFILE3INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME3 echo " OK" #### hostsfile.mine.nu, 0.0.0.0 format #HFNAME4="mine-nu-0" #HFSERVER4="http://hostsfile.mine.nu.nyud.net" #HFILE4="hosts0.zip" ## Downloading #echo -n "Retrieving $HFNAME4 from $HFSERVER4 ..." #wget -q -O $HOSTSPATH/$HFNAME4 $HFSERVER4/$HFILE4 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 14; } #unzip -p $HOSTSPATH/$HFNAME4 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/hosts-$HFNAME4 #echo " OK" #----------------------------------------------------------------------- echo -n "Processing local blacklist $BLACKLIST ..." if [ -f "$BLACKLIST" ] ; then cat $BLACKLIST | sed -e '/#.*/ d' -e '/^$/ d' -e 's/^/0.0.0.0 /g' > $HOSTSPATH/blacklist.ready echo " OK" echo -n "Merging lists ..." #cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all echo " OK" else echo " NOT FOUND" echo -n "Merging lists ..." #cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 | sort | uniq > $HOSTSPATH/hosts.all cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 | sort | uniq > $HOSTSPATH/hosts.all echo " OK" fi echo -n "Processing whitelist $WHITELIST ..." if [ -f "$WHITELIST" ] ; then cat $WHITELIST | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/whitelist.ready grep -Fvf $HOSTSPATH/whitelist.ready $HOSTSPATH/hosts.all > $HOSTSPATH/hosts.all.2 mv $HOSTSPATH/hosts.all.2 $HOSTSPATH/hosts.all echo " OK" else echo " NOT FOUND" fi #----------------------------------------------------------------------- echo -n "Writing hosts file $HOSTSFILE ..." cat $ORIGFILE > $HOSTSFILE echo " OK" HOSTCOUNT=`cat $HOSTSPATH/hosts.all | wc -l` echo "" >> $HOSTSFILE # to be sure the original file ends in a new-line echo "" >> $HOSTSFILE cat >> $HOSTSFILE << EOF #================================================================== # # `date` # $HOSTCOUNT hosts blocked by uphosts # # Original file: $ORIGFILE # Permanent changes can be done there, it is imported at every run # #================================================================== EOF echo "" >> $HOSTSFILE cat $HOSTSPATH/hosts.all >> $HOSTSFILE #rm -fv $HOSTSPATH/hosts* echo "Update process complete - $HOSTCOUNT hosts blocked!"
zzz-uphosts-run
#!/bin/sh # uphosts - Hosts File Updater # zzz-uphosts-run # wrapper for running uphosts logging output # for example, from /etc/network/if-up.d # In that case, this script is run by run-parts # Check run-parts naming conventions: "must consist entirely of upper and lower case letters, digits, underscores, and hyphens" # ie do not name this script foo.sh !!! # 20101230 Paolo UPHOSTSFILE="/opt/uphosts/uphosts.sh" LOGGERPARAMS="-t uphosts" $UPHOSTSFILE | logger $LOGGERPARAMS &
uphosts-blacklist
# uphosts - Hosts file updater # Local Blacklist file # Lines starting with hash are ignored # Add hostnames below, one per line # These entries are just merged adding 0.0.0.0 # Unlike whitelist, here foo.com means ONLY foo.com ! # eg. # test12345.com # test67890.com
uphosts-whitelist
# uphosts - Hosts file updater # Whitelist file # Lines starting with hash are ignored # Add hostnames below, one per line # These are grep patterns, so foo.com means EVERY matching line ! # eg. # test12345.com # test67890.com