Transparent HTTPS Filtering Proxy On CentOS

Home About Linux Q & A List Write for Us Advertise Contact
Xmodulo
Linux FAQs, tips and tutorials
Boost BGP Network Performance by 42% - [Free

Demo]
How to set up a transparent HTTPS filtering proxy

on CentOS
Last updated on April 8, 2014 Authored by Rafael Akchurin 12 Comments
HTTPS protocol is used more and more in today’s web. While this may be good for privacy, it
leaves modern network administrator without any means to prevent questionable or adult
contents from entering his/her network. Previously it was assumed that this problem does not
have a decent solution. Our how-to guide will try to prove otherwise.
This guide will tell you how to set up Squid on CentOS / RedHat Linux for transparent filtering
of HTTP and HTTPS traffic with help of Diladele Web Safety ICAP server, which is a
commercial solution for Linux, BSD and MacOS. The Linux installer of Diladele Web Safety
used in this tutorial contains fully featured keys which remain valid for 3 month period, so you
can test its full features during this trial period.
open in browser PRO version Are you a developer? Try out the HTML to PDF API pdfcrowd.com
Assumptions and Requirements
In this tutorial, I will assume the following. You have a network with IP addresses from
192.168.1.0 subnet, network mask is 255.255.255.0, and all workstations are set to use
192.168.1.1 as default gateway. On this default gateway, you have two NICs - one facing LAN
with IP address 192.168.1.1, the other is plugged in into ISP network and gets its public
Internet address through DHCP. It is also assumed your gateway has CentOS or RedHat
Linux up and running.
Linux FAQ categories

Step 1. Update and Upgrade
Before going further, run the following script to upgrade your system to the most recent state.
Amazon AWS
Android 1 #!/bin/bash
Arch Linux 2 set -e
3
Audio 4 # update should be done as root
Business 5 if [[ $EUID -ne 0 ]]; then
CentOS 6 echo "This script must be run as root" 1>&2
7 exit 1
Chromium OS 8 fi
Cisco 9
10 # update and upgrade
Cloud Services
11 yum update && yum upgrade
CloudStack 12
Databases 13 # disable selinux
14 sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/confi
Debian 15
Desktop 16 # and reboot
Development 17 reboot
Editors
Editors
Fedora
Step 2. Install Apache Web Server
Filesystem Diladele Web Safety has sophisticated a web administrator console to easily manage filtering
Games settings and policies. This Web UI is built using Python Django web framework, and requires
Google Apache web server to function correctly. Run the following script to install them.
Graphics
1 #!/bin/bash
Hadoop 2 set -e
Hardware 3
Java 4 # all web packages are installed as root
5 if [[ $EUID -ne 0 ]]; then
Kernel 6 echo "This script must be run as root" 1>&2
KVM 7 exit 1
8 fi
Lightweight Linux 9
Linux Mint 10 # install python libs
Math 11 yum install python-setuptools python-ldap
12
Networking 13 # install python django for web ui
Open vSwitch 14 easy_install django==1.5
OpenStack 15
16 # install apache web server to run web ui
OpenWRT 17 yum install httpd php mod_wsgi
Perl 18
19 # make apache autostart on reboot
PHP 20 chkconfig httpd on
Productivity 21
Publishing 22 # this fixes some apache errors when working with python-django
23 echo "WSGISocketPrefix /var/run/wsgi" >> /etc/httpd/conf.d/wsgi
Python 24
Raspberry Pi 25 # and restart apache
Security 26 service httpd restart
Shells
Shells
System
Step 3. Install Diladele Web Safety
System Download and install the latest version of Diladele Web Safety using the following script.
Administration
1 #!/bin/bash
Ubuntu 2
Utilities 3 # all packages are installed as root
Video
5 echo "This script must be run as root" 1>&2
VirtualBox 6 exit 1
Virtualization 7 fi
8
VMware 9 # detect current architecture (default assumes x86_64)
Vyatta 10 ARCH_1=`uname -m`
Web 11 ARCH_2="amd64"
12 if [[ $ARCH_1 == 'i686' ]]; then
Windows Azure 13 ARCH_1="i386"
Xen 14 ARCH_2="i386"
15 fi
XenServer
16
17 # bail out on any error
18 set -e
19
20 # get latest qlproxy
21 curl http://updates.diladele.com/qlproxy/binaries/3.2.0.4CAF/$AR
22
23 # install it
24 yum -y --nogpgcheck localinstall qlproxy-3.2.0-4CAF.$ARCH_1.rpm
25
26 # qlproxy installed everything needed for apache, so just restar
27 service httpd restart
Step 4. Install Required Build Tools

To be able to perform HTTP/HTTPS transparent filtering, we need to get the latest version of
Squid (the one that comes with CentOS / RedHat by default is too outdated), and rebuild it
from source. The following script installs all build tools required.
1 #!/bin/bash
2
3 # install all build tools
6 exit 1
7 fi
8
9 # install development packages required
10 yum install -y gcc-c++ pam-devel db4-devel expat-devel libxml2-d
11
12 # squid needs perl and needs additional perl modules not present
13 curl http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-
14 rpm -Uvh epel-release-6*.rpm
15 yum install -y perl-Crypt-OpenSSL-X509
Step 5. Build Squid from Source

Rebuild the Squid RPM by running the following script.
1 #!/bin/bash
2
3 # stop on any error
4 set -e
5
6 # rpm build MUST be run as normal user
7 if [[ $EUID -eq 0 ]]; then
8 echo "This script must NOT be run as root" 1>&2
9 exit 1
10 fi
11
12 # get squid sources
13 pushd rpmbuild/SOURCES
14 curl http://www.squid-cache.org/Versions/v3/3.4/squid-3.4.4.tar
15 curl http://www.squid-cache.org/Versions/v3/3.4/squid-3.4.4.tar
16 popd
16 popd
17
18 # build the binaries RPMs out of sources
19 pushd rpmbuild/SPECS
20 rpmbuild -v -bb squid.spec
21 popd
Step 6. Install Squid

After build finishes, install Squid. It is advisable to uncomment the lines which generate your
own root certification authority. Default installation of Diladele Web Safety does have its own
ca, but trusting it may pose serious security risk if your devices are used by users outside of
your network.
1 #!/bin/bash
2
3 # stop on every error
4 set -e
5
6 # install RPMs as root
9 exit 1
10 fi
11
12 # detect current architecture (default assumes x86_64)
13 ARCH_1=`uname -m`
14 ARCH_2="amd64"
15 ARCH_3="lib64"
16
17 if [[ $ARCH_1 == 'i686' ]]; then
18 ARCH_2="i386"
19 ARCH_3="lib"
20 fi
20 fi
21
22 pushd rpmbuild/RPMS/$ARCH_1
23 yum localinstall -y squid-3.4.4-0.el6.$ARCH_1.rpm
24 popd
25
26 # set up the ssl_crtd daemon
27 if [ -f /bin/ssl_crtd ]; then
28 rm -f /bin/ssl_crtd
29 fi
30
31 ln -s /usr/$ARCH_3/squid/ssl_crtd /bin/ssl_crtd
32 /bin/ssl_crtd -c -s /var/spool/squid_ssldb
33 chown -R squid:squid /var/spool/squid_ssldb
34
35 # uncomment to regenerate certificates for SSL bumping if you do
36 # openssl req -new -newkey rsa:1024 -days 1365 -nodes -x509 -key
37 # openssl x509 -in myca.pem -outform DER -out myca.der
38 # then copy certificates
39 # cp myca.pem /etc/opt/quintolabs/qlproxy/
40 # cp myca.der /etc/opt/quintolabs/qlproxy/
41
42 # make squid autostart after reboot
43 chkconfig squid on
Step 7. Integrate Squid with Diladele Web Safety

Integrate Squid and Diladele Web Safety by running the following script.
1 #!/bin/bash
2
3 # stop on any error
4 set -e
5
6 # integration should be done as root
6 # integration should be done as root
9 exit 1
10 fi
11
12 # allow web ui read-only access to squid configuration file
13 chmod o+r /etc/squid/squid.conf
14
15 # perform integration by replacing squid.conf file
16 mv /etc/squid/squid.conf /etc/squid/squid.conf.original &&am
17
18 # parse the resulting config just to be sure
19 /usr/sbin/squid -k parse
20
21 # restart squid to load all config
22 /sbin/service squid restart
Step 8. Transparently Redirect HTTPS Traffic to Squid

Transparent filter for HTTP and HTTPS traffic will be implemented by redirecting traffic to
ports 80 and 443 to Squid using iptables. This implies that the box with Squid acts as
default gateway for your LAN. Please note this is only one way to implementing transparent
filtering. Other possible solutions are described in Squid’s Wiki.
1 #!/bin/bash
2
3 # firewall setup should be done as root
6 exit 1
7 fi
8
9 # check kernel forwarding is enabled
9 # check kernel forwarding is enabled
10 enabled=`cat /proc/sys/net/ipv4/ip_forward`
11 if [[ $enabled -ne 1 ]]; then
12 echo "Kernel forwarding seems to be disabled, enable it
13 exit 1
14 fi
15
16 # set the default policy to accept first (not to lock ourselves
17 iptables -P INPUT ACCEPT
18
19 # flush all current rules from iptables
20 iptables -F
21
22 # allow pings from eth0 and eth1 for debugging purposes
23 iptables -A INPUT -p icmp -j ACCEPT
24
25 # allow access for localhost
26 iptables -A INPUT -i lo -j ACCEPT
27
28 # accept packets belonging to established and related connection
29 iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
30
31 # allow ssh connections to tcp port 22 from eth0 and eth1
32 iptables -A INPUT -p tcp --dport 22 -j ACCEPT
33
34 # allow connection from LAN to ports 3126, 3127 and 3128 squid i
35 iptables -A INPUT -i eth0 -p tcp --dport 3126 -j ACCEPT
38
39 # redirect all HTTP(tcp:80) traffic coming in through eth0 to 31
40 iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -
41
42 # redirect all HTTPS(tcp:443) traffic coming in through eth0 to
43 iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443
44
44
45 # configure forwarding rules
46 iptables -A FORWARD -i eth0 -o eth1 -p tcp --dport 22 -j ACCEPT
47 iptables -A FORWARD -i eth1 -o eth0 -p tcp --sport 22 -j ACCEPT
48 iptables -A FORWARD -p icmp -j ACCEPT
50 iptables -A FORWARD -i eth1 -o eth0 -p tcp --sport 80 -j ACCEPT
52 iptables -A FORWARD -i eth0 -o eth1 -p udp --dport 53 -j ACCEPT
53 iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCE
54 iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
55
56 # enable NAT for clients within LAN
57 iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
58
59 # set default policies for INPUT, FORWARD (drop) and OUTPUT (acc
60 iptables -P INPUT DROP
61 iptables -P FORWARD DROP
62 iptables -P OUTPUT ACCEPT
63
64 # list created rules
65 iptables -L -v
66
67 # save the rules so that after reboot they are automatically res
68 /sbin/service iptables save
69
70 # enable the firewall
71 chkconfig iptables on
72
73 # and reboot machine
74 reboot
Check if HTTPS is Transparently Filtered

Please note, in order for HTTPS filtering to function correctly, we must install the proxy
certificate from /etc/opt/quintolabs/qlproxy/myca.der into Trusted Root Certification on all
workstations in our network. The following screenshots show that HTTPS requests were
decrypted and filtered transparently.
Browsing to Google and searching for an adult term (e.g. NSFW), we get the HTTPS request
filtered and blocked transparently.
Resume
We now have the default gateway in our network capable of transparently filtering HTTP and
HTTPS traffic. All workstations in our network trust the root certificate from proxy, and thus
get their HTTPS request decrypted and filtered. Browsing environment in our network
became much safer.
Links
Archive with all scripts mentioned in this HOWTO
Online documentation of Diladele Web Safety
Squid proxy wiki
Download this article as ad-free PDF (made possible by your kind donation):
Subscribe to Xmodulo
Do you want to receive Linux FAQs,
detailed tutorials and tips published
at Xmodulo? Enter your email address
below, and we will deliver our Linux
posts straight to your email box, for
free. Delivery powered by Google
Feedburner.
Your email address
Subscribe
Support Xmodulo
Did you find this tutorial helpful? Then please be generous and support Xmodulo!
Bio Latest Posts
Rafael Akchurin
Diladele Web Safety for Squid Proxy Server is an ICAP server that
integrates with existing Squid proxy server and provides rich content and
web filtering functionality to sanitize Internet traffic passing into internal
home/enterprise network. It may be used to block illegal or potentially
malicious file downloads, remove annoying advertisements, prevent
access to various categories of the web sites and block resources with
explicit content.
Share 29 Tweet Share 4 subm it
Related FAQs:
How to set up Squid as a transparent web proxy on CentOS or RHEL
How to analyze Squid logs with SARG log analyzer on CentOS
How to set up MailScanner, Clam Antivirus and SpamAssassin in CentOS mail
server
How to close an open DNS resolver
How to block network traffic by country on Linux
Categories: CentOS, Networking, Security, System Administration

Tags: https, iptables, proxy, squid
12 thoughts on “How to set up a transparent HTTPS filtering proxy on

CentOS”
krot on April 8, 2014 at 9:33 am said: Reply
Don't forget to create your own ssl certificate and install it on all your
LAN hosts.
If you have problem with your generated certificates try this tip: Rebuild ssldb
sudo service squid3 stop
sudo rm -R /var/spool/squid3_ssldb
sudo mkdir /var/spool/squid3_ssldb
sudo /bin/ssl_crtd -c -s /var/spool/squid3_ssldb
sudo chown -R proxy:proxy /var/spool/squid3_ssldb
sudo /usr/sbin/squid3 -k parse
who on April 8, 2014 at 11:37 am said: Reply
Is this just not a MITM attack?

You will be giving "Diladele Web Safety" all your employee's web banking
details too, along with all passwords, etc. etc.
Raf on April 8, 2014 at 8:09 pm said: Reply
The SSL bump is only appropriate in cases when you are the sole
owner of a network (like home for example). In any case doing SSL bump may
be fully illegal in some countries so consult your lawyer first when planning to implement it.
This article describes only technical means of performing HTTPS filtering which may be
required for some deployments (think schools, libraries etc). More info can be found on
the Squid's web site (search for Squid SSL Bump Wiki).
Really? on April 17, 2014 at 6:30 am said: Reply
MITM is definitely illegal in plenty of countries especially given that you

will likely pick up banking details and plenty of passwords. The best you can
legally do for HTTPS is to just block based on destination.
That's right on January 14, 2016 at 11:06 pm said: Reply
MITM is only an attack if the implementer/administrators deem harm

on the users. Perhaps employees should stick to banking on their own
network.
Reality on April 8, 2014 at 10:23 pm said: Reply
Why should we trust you to know security setting appropriately if your

page is littered with uncorrected HTML / XML entities?
This line:
echo "This script must be run as root" 1>&2
Won't work very well if just blindly copied and pasted (as you seem to be suggesting users
do)
Raf on April 10, 2014 at 6:57 am said: Reply
Probably some "safe" parsing of wordpress on this site - there is

downloadable archive attached to this article - it contains all the scripts for just
running - no need to copy paste.
Dan Nanni on April 11, 2014 at 8:43 pm said: Reply
They were just due to the HTML encoding problem of our site. Not to
blame the author.
They are all fixed now.
jfdesir on April 27, 2014 at 4:04 pm said: Reply
Hi everybody,
The post is great. But i've got something that's not OK.
Transparent HTTP is OK but when it is HTTPS, squid crashed.
i've got that logs:
2014/04/27 11:55:26 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' processes

2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
2014/04/27 11:55:26 kid1| Logfile: opening log daemon:/opt/squid3/var/logs/access.log
2014/04/27 11:55:26 kid1| Logfile Daemon: opening log /opt/squid3/var/logs/access.log
2014/04/27 11:55:26 kid1| Store logging disabled
2014/04/27 11:55:26 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects
2014/04/27 11:55:26 kid1| Target number of buckets: 1008
2014/04/27 11:55:26 kid1| Using 8192 Store buckets
2014/04/27 11:55:26 kid1| Max Mem size: 262144 KB
2014/04/27 11:55:26 kid1| Max Swap size: 0 KB
2014/04/27 11:55:26 kid1| Using Least Load store dir selection
2014/04/27 11:55:26 kid1| Set Current Directory to /opt/squid3/var/cache/squid
2014/04/27 11:55:26 kid1| Finished loading MIME types and icons.
2014/04/27 11:55:26 kid1| HTCP Disabled.
2014/04/27 11:55:26 kid1| Squid plugin modules loaded: 0
2014/04/27 11:55:26 kid1| Adaptation support is on
2014/04/27 11:55:26 kid1| Accepting NAT intercepted HTTP Socket connections at
local=192.168.1.254:3126 remote=[::] FD 21 flags=41
2014/04/27 11:55:26 kid1| Accepting SSL bumped HTTP Socket connections at
local=192.168.1.254:3128 remote=[::] FD 22 flags=9
2014/04/27 11:55:26 kid1| Accepting NAT intercepted SSL bumped HTTPS Socket
connections at local=192.168.1.254:3127 remote=[::] FD 23 flags=41
2014/04/27 11:55:27 kid1| storeLateRelease: released 0 objects
2014/04/27 11:55:28 kid1| assertion failed: comm.cc:769: "Comm::IsConnOpen(conn)"
what am I doing wrong?

I've got only one interface on my proxy.
eric777 on May 11, 2014 at 6:25 pm said: Reply
jfdesir did you solve your issue?
jfdesir on October 16, 2014 at 1:31 pm said: Reply
Sorry for that late repy,
Yes i solved my issue.

Yes i solved my issue.
gtcoDave on October 27, 2014 at 12:01 pm said: Reply
This is a fantastic solution.
However I have a query or two:
Am I right in saying that implementing this method, whilst much better for an academic
network (IE School), it will prevent Diladele from being able to log which Active Directory
accounts visit which websites?
If this is true how could one get workstations to Auth somehow and populate the logs with
their account names again?
Leave a comment
Your email address will not be published. Required fields are marked *
Comment
Name *
Email *
Website
Post Comment
Notify me of follow-up comments by email.

Notify me of new posts by email.
« What is good LaTeX editor software on Linux
How to configure Conky with a GUI-based Conky config tool »
Subscribe to daily Linux FAQ tips
Enter your email address Subscribe
Search
Related Linux FAQs
What are available How to install

iptables GNOME desktop on
management tools CentOS
with GUI
How to configure How to monitor
firewall via user login history on
command line on CentOS with
Linux utmpdump
How to verify the How to boot on an

authenticity and ISO image from
integrity of a Grub
downloaded file on
Linux
Ask Xmodulo
How to fix “configure: error: pcre.h not
found”
How to enable and use logging module in
Python
How to remove all network namespaces at
once on Linux
How to plot a bar graph on Gnuplot
How to disable MAC learning in a Linux
bridge
How to enable .htaccess in Apache HTTP
server
How to add bookmarks to a PDF document
on Linux
How to record a particular area of desktop
screen on Linux
How to add proxy exceptions on Ubuntu
desktop
How to set up NFS server and client on
CentOS 7
Xmodulo List
Kryo
Apache Maven
Graphite
Cool Reader
netdata
Anjuta
ClamAV
GNU Octave
Audacity
CodeLite
About Advertise Write for Us Contact
Hosted by Stablehost

Transparent HTTPS Filtering Proxy On CentOS

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Transparent HTTPS Filtering Proxy On CentOS

Caricato da

Copyright:

Formati disponibili

Home About Linux Q & A List Write for Us Advertise Contact

Boost BGP Network Performance by 42% - [Free

How to set up a transparent HTTPS filtering proxy

Linux FAQ categories

Step 4. Install Required Build Tools

Step 5. Build Squid from Source

Step 6. Install Squid

Step 7. Integrate Squid with Diladele Web Safety

Step 8. Transparently Redirect HTTPS Traffic to Squid

Check if HTTPS is Transparently Filtered

Your email address

Bio Latest Posts

Share 29 Tweet Share 4 subm it

Categories: CentOS, Networking, Security, System Administration

12 thoughts on “How to set up a transparent HTTPS filtering proxy on

who on April 8, 2014 at 11:37 am said: Reply

Is this just not a MITM attack?

Raf on April 8, 2014 at 8:09 pm said: Reply

Really? on April 17, 2014 at 6:30 am said: Reply

MITM is definitely illegal in plenty of countries especially given that you

MITM is only an attack if the implementer/administrators deem harm

Reality on April 8, 2014 at 10:23 pm said: Reply

Why should we trust you to know security setting appropriately if your

Raf on April 10, 2014 at 6:57 am said: Reply

Probably some "safe" parsing of wordpress on this site - there is

Dan Nanni on April 11, 2014 at 8:43 pm said: Reply

jfdesir on April 27, 2014 at 4:04 pm said: Reply

2014/04/27 11:55:26 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' processes

what am I doing wrong?

eric777 on May 11, 2014 at 6:25 pm said: Reply

jfdesir did you solve your issue?

jfdesir on October 16, 2014 at 1:31 pm said: Reply

Sorry for that late repy,

Yes i solved my issue.

gtcoDave on October 27, 2014 at 12:01 pm said: Reply

This is a fantastic solution.

However I have a query or two:

Notify me of follow-up comments by email.

« What is good LaTeX editor software on Linux

How to configure Conky with a GUI-based Conky config tool »

Subscribe to daily Linux FAQ tips

Enter your email address Subscribe

Related Linux FAQs

What are available How to install

How to verify the How to boot on an

About Advertise Write for Us Contact

Potrebbero piacerti anche