Monday, June 19, 2017

Restricting Access at in Varnish Server Wide to Admin Areas

There are lots of ways to implement access controls and additional security on your hosting. I recently implemented a very restrictive but easy to manage implementation for our FreeBSD server that runs everything through Varnish. I could have placed this restriction in Apache via .HTACCESS files or global rules but placing them in Varnish they execute earlier and have less server overhead and ensure there are no caching related flaws.

To set this up I added the following to the top of my /usr/local/etc/varnish.vcl file.
# Whitelist of IP Address's or Ranges that are allowd to access restricted administration pages.
acl admin {
    "127.0.0.1";
    "localhost";
    "10.1.0.0"/16; # Local Network Class B
    "192.168.1.0"/16; # Local Network Class C
    "1.1.1.1"; # Sample
    # INSERT IPS #
}

Next inside the vcl_recv sub, or add one if needed include this before other rules. The bold text is doing URL matching to determine what scripts or directories to require approval for.  This could also be applied to specific domains but in this case it is server wide.
# Optional feature to only allow access to matched pages if client is on a whitelist.
    if (req.url ~ "^/wp-(login|admin|cron|json)" && client.ip !~ admin) {
        # Whitelist for all admin pages
        return (synth(403, "IP address not authorized, please request access from AdminEmail" ));
    }

Now you can reload varnish and do some testing by adding and removing your local network or IP and ensure its working. You can stop here but this isn't very convenient to manage or update form offsite.
sudo service varnishd reload

Add a custom script to grant access to IP's.

We have a custom script hidden our our server that it not indexed or published anywhere that our administrator can use to add new addressed on the fly. It asks for a name, location, and IP Address and will update the varnish.vcl file and reload the configuration when submitted.

You can post this script to a secret location on the web server.  You may want to add a password to it for extra security and don't link to it anywhere. You will also need to run the following commands.

// Make writable by www user.
sudo chmod g+w /usr/local/etc/varnish.vcl
sudo chgrp www /usr/local/etc/varnish.vcl
// Add custom script to reload varnish.
echo "#!/usr/local/bin/bash
service varnishd reload" > ~/varnish-reload
// Move script and make it executable
sudo cp ~/varnish-reload /usr/local/bin/varnish-reload
rm ~/varnish-reload
sudo chmod +x /usr/local/bin/varnish-reload

Read XML Sitemap to a CSV File

You may need to process a sitemap for redirects or get a list of all your pages for some testing or monitoring process.  Here is a simple script you can run to convert your sitemap to a CSV file.  It even supports multi-file sitemaps.

Simply save this script sitemap-to-csv.pl to your computer. You will need Perl to run in.

Then you can execute it from the command line with the following format:
// This will automatically append /sitemap.xml
./sitemap-to-csv.pl http://domain.com

// Or specify the exact file.
perl sitemap-to-csv.pl http://domain.com

// With SSL domain authentication override.
PERL_LWP_SSL_VERIFY_HOSTNAME=0 ./sitemap-to-csv.pl http://domain.com