How to install and configure Varnish cache on Ubuntu Linux 16.04 LTS

H

ow do I install Varnish Cache server to improve the performance of my existing Apache web server especially caching dynamic content on Ubuntu Linux 16.04 LTS server?

 

Varnish Cache is a web application accelerator. You install it in front of any web-server that speaks HTTP and configure it to cache the contents. It act as a web application accelerator focusing on optimizing caching and compression It is really fast and used by high traffic websites. Let us see how to configure Varnish cache version 4.x on Ubuntu Linux 16.04 LTS server.

Our setup

Fig.01: Set up Varnish cache with various options

Install Varnish

Type the following apt-get command/apt command to install Varnish cache software:

$ sudo apt update

$ sudo apt upgrade

$ apt search Varnish

$ sudo apt install varnish

 

Sample outputs:

Reading state information… Done

The following additional packages will be installed:

binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0

libubsan0 libvarnishapi1

Suggested packages:

binutils-doc cpp-doc gcc-5-locales gcc-multilib make autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg libitm1-dbg

libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg varnish-doc

The following NEW packages will be installed:

binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0

libubsan0 libvarnishapi1 varnish

0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded.

Need to get 23.1 MB of archives.

After this operation, 79.7 MB of additional disk space will be used.

Do you want to continue? [Y/n] y

Get:1 http://in.archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB]

…..

..

Setting up libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) …

Setting up libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) …

Setting up gcc-5 (5.4.0-6ubuntu1~16.04.4) …

Setting up gcc (4:5.3.1-1ubuntu1) …

Setting up libvarnishapi1 (4.1.1-1) …

Setting up libjemalloc1 (3.6.0-9ubuntu1) …

Setting up varnish (4.1.1-1) …

Processing triggers for libc-bin (2.23-0ubuntu5) …

Processing triggers for systemd (229-4ubuntu16) …

Processing triggers for ureadahead (0.100.0-19) …

Varnish installed and it is configured to listen on port 6081. This is default. You can verify it with the following simple netstat command:

$ netstat -tulpn | grep varnishd

 

Sample outputs:

Fig.02: Varnish is configured to listen on port TCP IPv4/IPv6 port # 6081

Varnish configuration

Ubuntu Linux 16.04 LTS uses systemd based config files as follows:

/lib/systemd/system/varnish.service – Default config file

TCP 6081 port – Varnish cache port

TCP 6082 port – Varnish management port

/etc/varnish/ – Varnish configuration files including VCLs

Let us see how to configure as per setup #1 (see fig.01).

Step 1 – Put Varnish cache on port 80

Type the following command to edit varnish cache file and create a new config file /etc/systemd/system/varnish.service.d/:

$ sudo systemctl edit varnish.service

 

Add/append the following config options:

[Service]

ExecStart=

ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a 202.54.2.50:80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,1024m

Save and close the file. Where,

202.54.2.50:80 – Listen to public IP address on port 80.

1024m – Increase RAM usage for Varnish from default 256m to 1024m

Type the following command to reload changes:

$ sudo systemctl daemon-reload

Step 2 – Configure Varnish to use our Apache server as a backend

Next, you will configure Varnish to use our Apache server as a backend which is running on IP address 192.168.1.6 port 80. Edit the /etc/varnish/default.vcl, enter:

$ sudo vi /etc/varnish/default.vcl

 

Set the following to point to your content server (Apache/Lighttpd/PHP/Python app server) running on 192.168.1.6 port 80.

backend default {

.host =  192.168.1.6 ;

.port =  80 ;

}

Basically, you need to set the values of host and port match your LAMP server private IP address and listening port. Save and close the file.

Step 3 – Configure Apache to use run on IP address 192.168.1.6 port 80

Type the following command to edit the /etc/apache2/ports.conf:

$ sudo vi /etc/apache2/ports.conf

 

Edit/add/append Listen directive:

Listen 192.168.1.6:80

Save and close the file. Edit /etc/apache2/sites-enabled/000-default.conf, enter:

$ vi /etc/apache2/sites-enabled/000-default.conf

 

The Virtual Host should be updated as follows :

<VirtualHost 192.168.1.6:80>

Save and close the file. Finally, restart the Apache web-server, run:

$ sudo systemctl restart apache2.service

 

At this stage you need to restart the Varnish cache too, enter:

$ sudo systemctl restart varnish.service

Step 4 – Configure ufw to open port 80

Type the following command to open port 80 using ufw:

$ sudo ufw allow 80

 

Sample outputs:

Rule added

Rule added (v6)

Step 5- Test it

Fire your webbrowser and type the url or IP address:

http://your-ip-here/

http://202.54.2.50/

http://your-domain-name/

Sample outputs:

Fig.03: Varnish working on my setup

 

Another option is to use the curl command to see http headers:

$ url -I http://202.54.2.50/

 

Sample outputs:

HTTP/1.1 200 OK

Date: Wed, 22 Feb 2017 11:59:38 GMT

Server: Apache/2.4.18 (Ubuntu)

Last-Modified: Wed, 22 Feb 2017 11:42:57 GMT

Vary: Accept-Encoding

Content-Type: text/html

X-Varnish: 32773 32771

Age: 4

Via: 1.1 varnish-v4

ETag: W/ 2c39-5491cff7c5c13-gzip

Accept-Ranges: bytes

Connection: keep-alive

The value of Age: indicate that pages has been cached by Varnish server.

How do I see the Varnish log and other info?

To see Varnish log entry ranking, enter:

$ sudo varnishtop

 

Sample outputs:

Fig.04: varnishtop in action

 

To see log file cd to /var/log/varnish

$ cd /var/log/varnish

$ sudo tail -f varnish.log

$ sudo tail -f varnishncsa.log

 

Sample outputs (provided that you configured PHP/Python with Apache you should see something as follows):

151.1.xx.yy – – [22/Feb/2017:12:03:48 +0000]  GET http://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/ HTTP/1.0  200 60944  https://www.google.co.in/   Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0

111.z.xx.yy  – – [22/Feb/2017:12:03:48 +0000]  GET http://www.cyberciti.biz/faq/linux-disk-format/ HTTP/1.0  200 94830  –   Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

xxx.ttt.zz.tt – – [22/Feb/2017:12:03:48 +0000]  GET http://www.cyberciti.biz/faq/wp-content/plugins/contextual-related-posts/default.png HTTP/1.0  200 15454  https://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/   Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0

aa.bb.cc.dd  – – [22/Feb/2017:12:03:48 +0000]  GET http://www.cyberciti.biz/faq/find-linux-distribution-name-version-number/ HTTP/1.0  200 92470  https://www.google.co.in/   Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0

See the Varnish server cache statistics

Type the following command:

$ sudo varnishstat

$ sudo varnishstat -1

$ sudo varnishstat -1 | grep -i cache

 

Sample outputs:

MAIN.cache_hit            334059         6.33 Cache hits

MAIN.cache_hitpass           239         0.00 Cache hits for pass

MAIN.cache_miss            94300         1.79 Cache misses

How do I purge content from the Varnish cache?

First you need to define ACL. I have the following config in my /etc/varnish/default.vcl file:

# Who is allowed to purge?

acl purge {

localhost ;

127.0.0.1 ;

72.14.190.12 ;

192.168.1.6 ;

}

sub vcl_recv {

# allow PURGE from localhost,72.14.190.12 and 192.168.1.6

if (req.method ==  PURGE ) {

if (!client.ip ~ purge) {

return(synth(405, Not allowed. ));

}

return (purge);

}

}

Use the following command to invalidate https://www.cyberciti.biz/faq/bash-for-loop/ page I would call out to Varnish like this:

$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret ban  req.http.host == www.cyberciti.biz && req.url == /faq/bash-for-loop/

 

To remove/purge all .html pages from cache, enter:

$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret  ban req.http.host ~ www.cyberciti.biz && req.url ~ .html

 

If you want to purge everything just restart the Varnish cache:

$ sudo systemctl restart varnish.service

 

See HTTP Purging for more info. Stay tunned for more Nginx SSL config and creating custom VCL for caching pages.

See also

9 Awesome Open Source Web Performance Software For Linux and Unix-like Systems

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *