How to install and configure Varnish cache on Ubuntu Linux 16.04 LTS
H
ow do I install Varnish Cache server to improve the performance of my existing Apache web server especially caching dynamic content on Ubuntu Linux 16.04 LTS server?
Varnish Cache is a web application accelerator. You install it in front of any web-server that speaks HTTP and configure it to cache the contents. It act as a web application accelerator focusing on optimizing caching and compression It is really fast and used by high traffic websites. Let us see how to configure Varnish cache version 4.x on Ubuntu Linux 16.04 LTS server.
Our setup
Fig.01: Set up Varnish cache with various options
Install Varnish
Type the following apt-get command/apt command to install Varnish cache software:
$ sudo apt update
$ sudo apt upgrade
$ apt search Varnish
$ sudo apt install varnish
Sample outputs:
Reading state information… Done
The following additional packages will be installed:
binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0
libubsan0 libvarnishapi1
Suggested packages:
binutils-doc cpp-doc gcc-5-locales gcc-multilib make autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg libitm1-dbg
libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg varnish-doc
The following NEW packages will be installed:
binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0
libubsan0 libvarnishapi1 varnish
0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded.
Need to get 23.1 MB of archives.
After this operation, 79.7 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://in.archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB]
…..
…
..
Setting up libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) …
Setting up libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) …
Setting up gcc-5 (5.4.0-6ubuntu1~16.04.4) …
Setting up gcc (4:5.3.1-1ubuntu1) …
Setting up libvarnishapi1 (4.1.1-1) …
Setting up libjemalloc1 (3.6.0-9ubuntu1) …
Setting up varnish (4.1.1-1) …
Processing triggers for libc-bin (2.23-0ubuntu5) …
Processing triggers for systemd (229-4ubuntu16) …
Processing triggers for ureadahead (0.100.0-19) …
Varnish installed and it is configured to listen on port 6081. This is default. You can verify it with the following simple netstat command:
$ netstat -tulpn | grep varnishd
Sample outputs:
Fig.02: Varnish is configured to listen on port TCP IPv4/IPv6 port # 6081
Varnish configuration
Ubuntu Linux 16.04 LTS uses systemd based config files as follows:
/lib/systemd/system/varnish.service – Default config file
TCP 6081 port – Varnish cache port
TCP 6082 port – Varnish management port
/etc/varnish/ – Varnish configuration files including VCLs
Let us see how to configure as per setup #1 (see fig.01).
Step 1 – Put Varnish cache on port 80
Type the following command to edit varnish cache file and create a new config file /etc/systemd/system/varnish.service.d/:
$ sudo systemctl edit varnish.service
Add/append the following config options:
[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a 202.54.2.50:80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,1024m
Save and close the file. Where,
202.54.2.50:80 – Listen to public IP address on port 80.
1024m – Increase RAM usage for Varnish from default 256m to 1024m
Type the following command to reload changes:
$ sudo systemctl daemon-reload
Step 2 – Configure Varnish to use our Apache server as a backend
Next, you will configure Varnish to use our Apache server as a backend which is running on IP address 192.168.1.6 port 80. Edit the /etc/varnish/default.vcl, enter:
$ sudo vi /etc/varnish/default.vcl
Set the following to point to your content server (Apache/Lighttpd/PHP/Python app server) running on 192.168.1.6 port 80.
backend default {
.host = 192.168.1.6 ;
.port = 80 ;
}
Basically, you need to set the values of host and port match your LAMP server private IP address and listening port. Save and close the file.
Step 3 – Configure Apache to use run on IP address 192.168.1.6 port 80
Type the following command to edit the /etc/apache2/ports.conf:
$ sudo vi /etc/apache2/ports.conf
Edit/add/append Listen directive:
Listen 192.168.1.6:80
Save and close the file. Edit /etc/apache2/sites-enabled/000-default.conf, enter:
$ vi /etc/apache2/sites-enabled/000-default.conf
The Virtual Host should be updated as follows :
<VirtualHost 192.168.1.6:80>
Save and close the file. Finally, restart the Apache web-server, run:
$ sudo systemctl restart apache2.service
At this stage you need to restart the Varnish cache too, enter:
$ sudo systemctl restart varnish.service
Step 4 – Configure ufw to open port 80
Type the following command to open port 80 using ufw:
$ sudo ufw allow 80
Sample outputs:
Rule added
Rule added (v6)
Step 5- Test it
Fire your webbrowser and type the url or IP address:
http://your-ip-here/
http://202.54.2.50/
http://your-domain-name/
Sample outputs:
Fig.03: Varnish working on my setup
Another option is to use the curl command to see http headers:
$ url -I http://202.54.2.50/
Sample outputs:
HTTP/1.1 200 OK
Date: Wed, 22 Feb 2017 11:59:38 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Wed, 22 Feb 2017 11:42:57 GMT
Vary: Accept-Encoding
Content-Type: text/html
X-Varnish: 32773 32771
Age: 4
Via: 1.1 varnish-v4
ETag: W/ 2c39-5491cff7c5c13-gzip
Accept-Ranges: bytes
Connection: keep-alive
The value of Age: indicate that pages has been cached by Varnish server.
How do I see the Varnish log and other info?
To see Varnish log entry ranking, enter:
$ sudo varnishtop
Sample outputs:
Fig.04: varnishtop in action
To see log file cd to /var/log/varnish
$ cd /var/log/varnish
$ sudo tail -f varnish.log
$ sudo tail -f varnishncsa.log
Sample outputs (provided that you configured PHP/Python with Apache you should see something as follows):
151.1.xx.yy – – [22/Feb/2017:12:03:48 +0000] GET http://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/ HTTP/1.0 200 60944 https://www.google.co.in/ Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0
111.z.xx.yy – – [22/Feb/2017:12:03:48 +0000] GET http://www.cyberciti.biz/faq/linux-disk-format/ HTTP/1.0 200 94830 – Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
xxx.ttt.zz.tt – – [22/Feb/2017:12:03:48 +0000] GET http://www.cyberciti.biz/faq/wp-content/plugins/contextual-related-posts/default.png HTTP/1.0 200 15454 https://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/ Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0
aa.bb.cc.dd – – [22/Feb/2017:12:03:48 +0000] GET http://www.cyberciti.biz/faq/find-linux-distribution-name-version-number/ HTTP/1.0 200 92470 https://www.google.co.in/ Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
See the Varnish server cache statistics
Type the following command:
$ sudo varnishstat
$ sudo varnishstat -1
$ sudo varnishstat -1 | grep -i cache
Sample outputs:
MAIN.cache_hit 334059 6.33 Cache hits
MAIN.cache_hitpass 239 0.00 Cache hits for pass
MAIN.cache_miss 94300 1.79 Cache misses
How do I purge content from the Varnish cache?
First you need to define ACL. I have the following config in my /etc/varnish/default.vcl file:
# Who is allowed to purge?
acl purge {
localhost ;
127.0.0.1 ;
72.14.190.12 ;
192.168.1.6 ;
}
sub vcl_recv {
# allow PURGE from localhost,72.14.190.12 and 192.168.1.6
if (req.method == PURGE ) {
if (!client.ip ~ purge) {
return(synth(405, Not allowed. ));
}
return (purge);
}
}
Use the following command to invalidate https://www.cyberciti.biz/faq/bash-for-loop/ page I would call out to Varnish like this:
$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret ban req.http.host == www.cyberciti.biz && req.url == /faq/bash-for-loop/
To remove/purge all .html pages from cache, enter:
$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret ban req.http.host ~ www.cyberciti.biz && req.url ~ .html
If you want to purge everything just restart the Varnish cache:
$ sudo systemctl restart varnish.service
See HTTP Purging for more info. Stay tunned for more Nginx SSL config and creating custom VCL for caching pages.
See also
9 Awesome Open Source Web Performance Software For Linux and Unix-like Systems