Old:UpHosts: differenze tra le versioni
Vai alla navigazione
Vai alla ricerca
(update script) |
m (ha spostato UpHosts a Old:UpHosts) |
||
| (8 versioni intermedie di 4 utenti non mostrate) | |||
| Riga 1: | Riga 1: | ||
{{Old}} | |||
==Introduzione== | ==Introduzione== | ||
Premessa: esistono svariati sistemi di content filtering decisamente più raffinati e personalizzabili di questo. | Premessa: esistono svariati sistemi di content filtering decisamente più raffinati e personalizzabili di questo. | ||
| Riga 9: | Riga 10: | ||
(ringrazio tutti i rispettivi autori) e nonostante abbia cercato di testarlo al meglio, non posso offrire alcuna garanzia :-) }} | (ringrazio tutti i rispettivi autori) e nonostante abbia cercato di testarlo al meglio, non posso offrire alcuna garanzia :-) }} | ||
[[Utente:Paolo321|Paolo321]] | [[Utente:Paolo321|Paolo321]] , 31 dicembre 2010 | ||
==Funzionamento== | ==Funzionamento== | ||
Spero che lo script sia sufficientemente chiaro e si "spieghi da solo", comunque lo script esegue: | Spero che lo script sia sufficientemente chiaro e si "spieghi da solo", comunque lo script esegue: | ||
*Controllo presenza software richiesti | *Controllo privilegi di root e presenza software richiesti | ||
*Controllo di quanto tempo è passato dall'ultima esecuzione (per evitare download troppo frequenti, vedi esempio sotto) | |||
*Backup dell'hosts file presistente (se non già effettuato) | *Backup dell'hosts file presistente (se non già effettuato) | ||
*Download delle liste | *Download delle liste | ||
| Riga 28: | Riga 30: | ||
**Eventuali modifiche permanenti al file hosts andranno quindi effettuate non in /etc/hosts ma nel file hosts di backup | **Eventuali modifiche permanenti al file hosts andranno quindi effettuate non in /etc/hosts ma nel file hosts di backup | ||
*Report numero hosts bloccati e uscita dello script | *Report numero hosts bloccati e uscita dello script | ||
==Note== | |||
*Come default lo script cerca blacklist e whitelist locali nella sua directory, quindi si può ad esempio collocare il tutto in /opt/uphosts. Ovviamente si può cambiare i path nello script e usarne altri. | |||
*Uno script di questo tipo è comodo sia eseguito "unattended"; iIl wrapper zzz-uphosts-run esegue uphosts e reindirizza l'output nei log di sistema. Può ad esempio essere eseguito da /etc/network/if-up.d, oppure tramite cron (ma in questo secondo caso c'è da aggiungere un controllo sulla presenza di connessione internet). Lo script all'avvio controlla comunque quanto tempo è passato dall'ultima esecuzione (per evitare che ad esempio dieci connessioni-disconnessioni al giorno portino a dieci download delle liste) | |||
==Script== | ==Script== | ||
===uphosts.sh=== | ===uphosts.sh=== | ||
<pre> | <pre> | ||
#!/bin/bash | #!/bin/bash | ||
# uphosts - Hosts | # uphosts - Hosts File Updater | ||
# README: | # README: | ||
| Riga 51: | Riga 56: | ||
# http://hostsfile.mine.nu/downloads/updatehosts.sh.txt | # http://hostsfile.mine.nu/downloads/updatehosts.sh.txt | ||
# | # 20101216 Paolo | ||
#----------------------------------------------------------------------- | #----------------------------------------------------------------------- | ||
HOSTSPATH="/tmp/hosts-`date +%s`" # Temp directory | HOSTSPATH="/tmp/hosts-`date +%s`" # Temp directory | ||
| Riga 61: | Riga 64: | ||
ORIGFILE="$HOSTSFILE.original" # Backup file | ORIGFILE="$HOSTSFILE.original" # Backup file | ||
CONFDIR=" | CONFDIR="$(dirname $(readlink -f $0))" # Parent directory of the script | ||
BLACKLIST="$CONFDIR/uphosts-blacklist" # Local Blacklist | BLACKLIST="$CONFDIR/uphosts-blacklist" # Local Blacklist | ||
WHITELIST="$CONFDIR/uphosts-whitelist" # Whitelist | WHITELIST="$CONFDIR/uphosts-whitelist" # Whitelist | ||
| Riga 68: | Riga 71: | ||
PROXYPASS="" #PROXYPASS="--proxy-password='password" | PROXYPASS="" #PROXYPASS="--proxy-password='password" | ||
# | DAYS="2" # Update frequency | ||
#----------------------------------------------------------------------- | |||
echo " | # Checks for root privileges | ||
builtin type -P wget &>/dev/null || { echo "wget is missing."; ABORT=1; } | if [ "$(whoami)" != 'root' ] ; then | ||
builtin type -P unzip &>/dev/null || { echo "unzip is missing."; ABORT=1; } | echo "You need to be root to execute uphosts. Exiting!" | ||
builtin type -P fromdos &>/dev/null || { echo "fromdos(tofrodos) is missing."; ABORT=1; } | exit 1 | ||
builtin type -P grep &>/dev/null || { echo "grep is missing."; ABORT=1; } | fi | ||
# Checks required packages | |||
ABORT=0 | |||
builtin type -P wget &>/dev/null || { echo -n "wget is missing."; ABORT=1; } | |||
builtin type -P unzip &>/dev/null || { echo -n "unzip is missing."; ABORT=1; } | |||
builtin type -P fromdos &>/dev/null || { echo -n "fromdos(tofrodos) is missing."; ABORT=1; } | |||
builtin type -P grep &>/dev/null || { echo -n "grep is missing."; ABORT=1; } | |||
if [ $ABORT != 0 ] ; then | if [ $ABORT != 0 ] ; then | ||
echo "Exiting!" | echo " Exiting!" | ||
exit | exit 2 | ||
fi | fi | ||
echo " | # Limits updates if uphosts is run often (i.e. at every if-up) | ||
# If there is no original hosts file this is the first run on a fresh system, and update runs anyway | |||
if [ -f "$ORIGFILE" ] && [ `find $HOSTSFILE -mtime -$DAYS` ] ; then | |||
echo "$HOSTSFILE is less than $DAYS days old. Exiting!" | |||
exit 3 | |||
fi | |||
# If there is no original hosts file this is the first run on a fresh system | |||
# (as above, but now original hosts file is saved) | |||
if [ ! -f "$ORIGFILE" ] ; then | if [ ! -f "$ORIGFILE" ] ; then | ||
echo "Backing up your previous hosts file ..." | echo "Backing up your previous hosts file ..." | ||
cp $HOSTSFILE $ORIGFILE | cp $HOSTSFILE $ORIGFILE | ||
echo "OK" | #echo "OK" | ||
fi | fi | ||
#----------------------------------------------------------------------- | |||
mkdir $HOSTSPATH | mkdir $HOSTSPATH | ||
# | # Every list has some specific tweaks :-) | ||
# If there are any errors, the script exits. Merging lists partially could be unsafe | |||
#### hphosts list | #### hphosts list | ||
| Riga 106: | Riga 121: | ||
HFILE1INT="HOSTS.txt" # Needed to specify which file from zip | HFILE1INT="HOSTS.txt" # Needed to specify which file from zip | ||
# Downloading | # Downloading | ||
echo "Retrieving $HFNAME1 from $HFSERVER1 ..." | echo -n "Retrieving $HFNAME1 from $HFSERVER1 ..." | ||
wget -q -O $HOSTSPATH/$HFNAME1 $HFSERVER1/$HFILE1 $PROXYUSER $PROXYPASS | wget -q -O $HOSTSPATH/$HFNAME1 $HFSERVER1/$HFILE1 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 11; } | ||
unzip -p $HOSTSPATH/$HFNAME1 $HFILE1INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME1 | unzip -p $HOSTSPATH/$HFNAME1 $HFILE1INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME1 | ||
echo "OK" | echo " OK" | ||
#### hphosts-partial list | #### hphosts-partial list | ||
| Riga 116: | Riga 131: | ||
HFILE2="hphosts-partial.asp" | HFILE2="hphosts-partial.asp" | ||
# Downloading | # Downloading | ||
echo "Retrieving $HFNAME2 from $HFSERVER2 ..." | echo -n "Retrieving $HFNAME2 from $HFSERVER2 ..." | ||
wget -q -O $HOSTSPATH/$HFNAME2 $HFSERVER2/$HFILE2 $PROXYUSER $PROXYPASS | wget -q -O $HOSTSPATH/$HFNAME2 $HFSERVER2/$HFILE2 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 12; } | ||
cat $HOSTSPATH/$HFNAME2 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME2 | cat $HOSTSPATH/$HFNAME2 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME2 | ||
echo "OK" | echo " OK" | ||
#### MVPs list | #### MVPs list | ||
| Riga 127: | Riga 142: | ||
HFILE3INT="HOSTS" # Needed to specify which file from zip | HFILE3INT="HOSTS" # Needed to specify which file from zip | ||
# Downloading | # Downloading | ||
echo "Retrieving $HFNAME3 from $HFSERVER3 ..." | echo -n "Retrieving $HFNAME3 from $HFSERVER3 ..." | ||
wget -q -O $HOSTSPATH/$HFNAME3 $HFSERVER3/$HFILE3 $PROXYUSER $PROXYPASS | wget -q -O $HOSTSPATH/$HFNAME3 $HFSERVER3/$HFILE3 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 13; } | ||
unzip -p $HOSTSPATH/$HFNAME3 $HFILE3INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME3 | unzip -p $HOSTSPATH/$HFNAME3 $HFILE3INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME3 | ||
echo "OK" | echo " OK" | ||
#### hostsfile.mine.nu, 0.0.0.0 format | #### hostsfile.mine.nu, 0.0.0.0 format | ||
| Riga 137: | Riga 152: | ||
#HFILE4="hosts0.zip" | #HFILE4="hosts0.zip" | ||
## Downloading | ## Downloading | ||
#echo "Retrieving $HFNAME4 from $HFSERVER4 ..." | #echo -n "Retrieving $HFNAME4 from $HFSERVER4 ..." | ||
#wget -q -O $HOSTSPATH/$HFNAME4 $HFSERVER4/$HFILE4 $PROXYUSER $PROXYPASS | #wget -q -O $HOSTSPATH/$HFNAME4 $HFSERVER4/$HFILE4 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 14; } | ||
#unzip -p $HOSTSPATH/$HFNAME4 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/hosts-$HFNAME4 | #unzip -p $HOSTSPATH/$HFNAME4 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/hosts-$HFNAME4 | ||
#echo "OK" | #echo " OK" | ||
# | #----------------------------------------------------------------------- | ||
echo "Processing local blacklist $BLACKLIST ..." | echo -n "Processing local blacklist $BLACKLIST ..." | ||
if [ -f "$BLACKLIST" ] ; then | if [ -f "$BLACKLIST" ] ; then | ||
cat $BLACKLIST | sed -e '/#.*/ d' -e '/^$/ d' -e 's/^/0.0.0.0 /g' > $HOSTSPATH/blacklist.ready | cat $BLACKLIST | sed -e '/#.*/ d' -e '/^$/ d' -e 's/^/0.0.0.0 /g' > $HOSTSPATH/blacklist.ready | ||
echo "OK" | echo " OK" | ||
echo "Merging lists ..." | echo -n "Merging lists ..." | ||
#cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all | #cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all | ||
cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all | cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all | ||
echo "OK" | echo " OK" | ||
else | else | ||
echo "NOT FOUND" | echo " NOT FOUND" | ||
echo "Merging lists ..." | echo -n "Merging lists ..." | ||
#cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 | sort | uniq > $HOSTSPATH/hosts.all | #cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 | sort | uniq > $HOSTSPATH/hosts.all | ||
cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 | sort | uniq > $HOSTSPATH/hosts.all | cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 | sort | uniq > $HOSTSPATH/hosts.all | ||
echo "OK" | echo " OK" | ||
fi | fi | ||
echo "Processing whitelist $WHITELIST ..." | echo -n "Processing whitelist $WHITELIST ..." | ||
if [ -f "$WHITELIST" ] ; then | if [ -f "$WHITELIST" ] ; then | ||
cat $WHITELIST | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/whitelist.ready | cat $WHITELIST | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/whitelist.ready | ||
grep -Fvf $HOSTSPATH/whitelist.ready $HOSTSPATH/hosts.all > $HOSTSPATH/hosts.all.2 | grep -Fvf $HOSTSPATH/whitelist.ready $HOSTSPATH/hosts.all > $HOSTSPATH/hosts.all.2 | ||
mv $HOSTSPATH/hosts.all.2 $HOSTSPATH/hosts.all | mv $HOSTSPATH/hosts.all.2 $HOSTSPATH/hosts.all | ||
echo "OK" | echo " OK" | ||
else | else | ||
echo "NOT FOUND" | echo " NOT FOUND" | ||
fi | fi | ||
#----------------------------------------------------------------------- | |||
echo -n "Writing hosts file $HOSTSFILE ..." | |||
cat $ORIGFILE > $HOSTSFILE | cat $ORIGFILE > $HOSTSFILE | ||
echo "OK" | echo " OK" | ||
HOSTCOUNT=`cat $HOSTSPATH/hosts.all | wc -l` | HOSTCOUNT=`cat $HOSTSPATH/hosts.all | wc -l` | ||
| Riga 181: | Riga 197: | ||
cat >> $HOSTSFILE << EOF | cat >> $HOSTSFILE << EOF | ||
#============================================================= | #================================================================== | ||
# | # | ||
# `date` | # `date` | ||
# $HOSTCOUNT hosts blocked by uphosts | # $HOSTCOUNT hosts blocked by uphosts | ||
# | # | ||
# Original file: $ORIGFILE | # Original file: $ORIGFILE | ||
# Permanent changes can be done there | # Permanent changes can be done there, it is imported at every run | ||
# | # | ||
#============================================================= | #================================================================== | ||
EOF | EOF | ||
| Riga 198: | Riga 213: | ||
#rm -fv $HOSTSPATH/hosts* | #rm -fv $HOSTSPATH/hosts* | ||
echo "Update process complete - $HOSTCOUNT hosts blocked!" | echo "Update process complete - $HOSTCOUNT hosts blocked!" | ||
</pre> | |||
===zzz-uphosts-run=== | |||
<pre> | |||
#!/bin/sh | |||
# uphosts - Hosts File Updater | |||
# zzz-uphosts-run | |||
# wrapper for running uphosts logging output | |||
# for example, from /etc/network/if-up.d | |||
# In that case, this script is run by run-parts | |||
# Check run-parts naming conventions: "must consist entirely of upper and lower case letters, digits, underscores, and hyphens" | |||
# ie do not name this script foo.sh !!! | |||
# 20101230 Paolo | |||
UPHOSTSFILE="/opt/uphosts/uphosts.sh" | |||
LOGGERPARAMS="-t uphosts" | |||
$UPHOSTSFILE | logger $LOGGERPARAMS & | |||
</pre> | </pre> | ||
| Riga 227: | Riga 264: | ||
# test12345.com | # test12345.com | ||
# test67890.com | # test67890.com | ||
</pre> | </pre> | ||
Versione attuale delle 18:29, 3 nov 2019
|
Attenzione. Questa guida è obsoleta. Viene mantenuta sul Wiki solo per motivi di natura storica e didattica. |
Introduzione
Premessa: esistono svariati sistemi di content filtering decisamente più raffinati e personalizzabili di questo.
Scopo di questo script è semplicemente bloccare l'accesso a hosts a vario titolo "malevoli" basandosi su liste disponibili online.
Tali host vengono inseriti in /etc/hosts come corrispondenti a 0.0.0.0 , bloccandone di fatto l'accesso.
|
ATTENZIONE questo script è stato creato per uso personale, in parte riadattando e modificando script altrui (ringrazio tutti i rispettivi autori) e nonostante abbia cercato di testarlo al meglio, non posso offrire alcuna garanzia :-) |
Paolo321 , 31 dicembre 2010
Funzionamento
Spero che lo script sia sufficientemente chiaro e si "spieghi da solo", comunque lo script esegue:
- Controllo privilegi di root e presenza software richiesti
- Controllo di quanto tempo è passato dall'ultima esecuzione (per evitare download troppo frequenti, vedi esempio sotto)
- Backup dell'hosts file presistente (se non già effettuato)
- Download delle liste
- Adattamento del formato delle liste
- Ogni lista richiede differenti azioni, questo purtroppo implica che le liste sono hard-coded e per aggiungerne o toglierne una è necessario modificare buona parte dello script
- Lettura blacklist locale
- Merge di tutte le liste, con ordinamento e eliminazione duplicati
- Lettura whitelist locale e rimozione dalla blacklist degli host presenti
- Creazione del nuovo file hosts
- Importazione del file hosts di backup
- Questo è necessario per garantire che host precedentemente presenti in /etc/hosts non vengano rimossi
- A ogni esecuzione dello script il precedente file hosts è sovrascritto, e il file hosts di backup è reimportato
- Eventuali modifiche permanenti al file hosts andranno quindi effettuate non in /etc/hosts ma nel file hosts di backup
- Report numero hosts bloccati e uscita dello script
Note
- Come default lo script cerca blacklist e whitelist locali nella sua directory, quindi si può ad esempio collocare il tutto in /opt/uphosts. Ovviamente si può cambiare i path nello script e usarne altri.
- Uno script di questo tipo è comodo sia eseguito "unattended"; iIl wrapper zzz-uphosts-run esegue uphosts e reindirizza l'output nei log di sistema. Può ad esempio essere eseguito da /etc/network/if-up.d, oppure tramite cron (ma in questo secondo caso c'è da aggiungere un controllo sulla presenza di connessione internet). Lo script all'avvio controlla comunque quanto tempo è passato dall'ultima esecuzione (per evitare che ad esempio dieci connessioni-disconnessioni al giorno portino a dieci download delle liste)
Script
uphosts.sh
#!/bin/bash
# uphosts - Hosts File Updater
# README:
# Bad hosts are blocked putting them in the hosts file as 0.0.0.0
# To add other sources script must be manually modified
# Permanent entries must be added to the original file
# THIS SCRIPT HAS NO WARRANTY !
# Thanks to:
# http://ubuntuedintorni.wordpress.com/2009/06/29/di-script-dns-e-file-host/
# http://hostsfile.mine.nu/downloads/updatehosts.sh.txt
# 20101216 Paolo
#-----------------------------------------------------------------------
HOSTSPATH="/tmp/hosts-`date +%s`" # Temp directory
HOSTSFILE="/etc/hosts" # Hosts file
ORIGFILE="$HOSTSFILE.original" # Backup file
CONFDIR="$(dirname $(readlink -f $0))" # Parent directory of the script
BLACKLIST="$CONFDIR/uphosts-blacklist" # Local Blacklist
WHITELIST="$CONFDIR/uphosts-whitelist" # Whitelist
PROXYUSER="" #PROXYUSER="--proxy-user=user.name"
PROXYPASS="" #PROXYPASS="--proxy-password='password"
DAYS="2" # Update frequency
#-----------------------------------------------------------------------
# Checks for root privileges
if [ "$(whoami)" != 'root' ] ; then
echo "You need to be root to execute uphosts. Exiting!"
exit 1
fi
# Checks required packages
ABORT=0
builtin type -P wget &>/dev/null || { echo -n "wget is missing."; ABORT=1; }
builtin type -P unzip &>/dev/null || { echo -n "unzip is missing."; ABORT=1; }
builtin type -P fromdos &>/dev/null || { echo -n "fromdos(tofrodos) is missing."; ABORT=1; }
builtin type -P grep &>/dev/null || { echo -n "grep is missing."; ABORT=1; }
if [ $ABORT != 0 ] ; then
echo " Exiting!"
exit 2
fi
# Limits updates if uphosts is run often (i.e. at every if-up)
# If there is no original hosts file this is the first run on a fresh system, and update runs anyway
if [ -f "$ORIGFILE" ] && [ `find $HOSTSFILE -mtime -$DAYS` ] ; then
echo "$HOSTSFILE is less than $DAYS days old. Exiting!"
exit 3
fi
# If there is no original hosts file this is the first run on a fresh system
# (as above, but now original hosts file is saved)
if [ ! -f "$ORIGFILE" ] ; then
echo "Backing up your previous hosts file ..."
cp $HOSTSFILE $ORIGFILE
#echo "OK"
fi
#-----------------------------------------------------------------------
mkdir $HOSTSPATH
# Every list has some specific tweaks :-)
# If there are any errors, the script exits. Merging lists partially could be unsafe
#### hphosts list
HFNAME1="hphosts"
HFSERVER1="http://support.it-mate.co.uk/downloads"
HFILE1="hphosts.zip"
HFILE1INT="HOSTS.txt" # Needed to specify which file from zip
# Downloading
echo -n "Retrieving $HFNAME1 from $HFSERVER1 ..."
wget -q -O $HOSTSPATH/$HFNAME1 $HFSERVER1/$HFILE1 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 11; }
unzip -p $HOSTSPATH/$HFNAME1 $HFILE1INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME1
echo " OK"
#### hphosts-partial list
HFNAME2="hphosts-partial"
HFSERVER2="http://www.hosts-file.net"
HFILE2="hphosts-partial.asp"
# Downloading
echo -n "Retrieving $HFNAME2 from $HFSERVER2 ..."
wget -q -O $HOSTSPATH/$HFNAME2 $HFSERVER2/$HFILE2 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 12; }
cat $HOSTSPATH/$HFNAME2 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME2
echo " OK"
#### MVPs list
HFNAME3="mvps"
HFSERVER3="http://www.mvps.org/winhelp2002"
HFILE3="hosts.zip"
HFILE3INT="HOSTS" # Needed to specify which file from zip
# Downloading
echo -n "Retrieving $HFNAME3 from $HFSERVER3 ..."
wget -q -O $HOSTSPATH/$HFNAME3 $HFSERVER3/$HFILE3 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 13; }
unzip -p $HOSTSPATH/$HFNAME3 $HFILE3INT | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' -e 's/127.0.0.1/0.0.0.0/g' > $HOSTSPATH/hosts-$HFNAME3
echo " OK"
#### hostsfile.mine.nu, 0.0.0.0 format
#HFNAME4="mine-nu-0"
#HFSERVER4="http://hostsfile.mine.nu.nyud.net"
#HFILE4="hosts0.zip"
## Downloading
#echo -n "Retrieving $HFNAME4 from $HFSERVER4 ..."
#wget -q -O $HOSTSPATH/$HFNAME4 $HFSERVER4/$HFILE4 $PROXYUSER $PROXYPASS || { echo " ERROR! Exiting!"; exit 14; }
#unzip -p $HOSTSPATH/$HFNAME4 | fromdos | grep -v localhost | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/hosts-$HFNAME4
#echo " OK"
#-----------------------------------------------------------------------
echo -n "Processing local blacklist $BLACKLIST ..."
if [ -f "$BLACKLIST" ] ; then
cat $BLACKLIST | sed -e '/#.*/ d' -e '/^$/ d' -e 's/^/0.0.0.0 /g' > $HOSTSPATH/blacklist.ready
echo " OK"
echo -n "Merging lists ..."
#cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all
cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/blacklist.ready | sort | uniq > $HOSTSPATH/hosts.all
echo " OK"
else
echo " NOT FOUND"
echo -n "Merging lists ..."
#cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 $HOSTSPATH/hosts-$HFNAME4 | sort | uniq > $HOSTSPATH/hosts.all
cat $HOSTSPATH/hosts-$HFNAME1 $HOSTSPATH/hosts-$HFNAME2 $HOSTSPATH/hosts-$HFNAME3 | sort | uniq > $HOSTSPATH/hosts.all
echo " OK"
fi
echo -n "Processing whitelist $WHITELIST ..."
if [ -f "$WHITELIST" ] ; then
cat $WHITELIST | sed -e '/#.*/ d' -e '/^$/ d' > $HOSTSPATH/whitelist.ready
grep -Fvf $HOSTSPATH/whitelist.ready $HOSTSPATH/hosts.all > $HOSTSPATH/hosts.all.2
mv $HOSTSPATH/hosts.all.2 $HOSTSPATH/hosts.all
echo " OK"
else
echo " NOT FOUND"
fi
#-----------------------------------------------------------------------
echo -n "Writing hosts file $HOSTSFILE ..."
cat $ORIGFILE > $HOSTSFILE
echo " OK"
HOSTCOUNT=`cat $HOSTSPATH/hosts.all | wc -l`
echo "" >> $HOSTSFILE # to be sure the original file ends in a new-line
echo "" >> $HOSTSFILE
cat >> $HOSTSFILE << EOF
#==================================================================
#
# `date`
# $HOSTCOUNT hosts blocked by uphosts
#
# Original file: $ORIGFILE
# Permanent changes can be done there, it is imported at every run
#
#==================================================================
EOF
echo "" >> $HOSTSFILE
cat $HOSTSPATH/hosts.all >> $HOSTSFILE
#rm -fv $HOSTSPATH/hosts*
echo "Update process complete - $HOSTCOUNT hosts blocked!"
zzz-uphosts-run
#!/bin/sh # uphosts - Hosts File Updater # zzz-uphosts-run # wrapper for running uphosts logging output # for example, from /etc/network/if-up.d # In that case, this script is run by run-parts # Check run-parts naming conventions: "must consist entirely of upper and lower case letters, digits, underscores, and hyphens" # ie do not name this script foo.sh !!! # 20101230 Paolo UPHOSTSFILE="/opt/uphosts/uphosts.sh" LOGGERPARAMS="-t uphosts" $UPHOSTSFILE | logger $LOGGERPARAMS &
uphosts-blacklist
# uphosts - Hosts file updater # Local Blacklist file # Lines starting with hash are ignored # Add hostnames below, one per line # These entries are just merged adding 0.0.0.0 # Unlike whitelist, here foo.com means ONLY foo.com ! # eg. # test12345.com # test67890.com
uphosts-whitelist
# uphosts - Hosts file updater # Whitelist file # Lines starting with hash are ignored # Add hostnames below, one per line # These are grep patterns, so foo.com means EVERY matching line ! # eg. # test12345.com # test67890.com