Sleep tight Ruby Roo.

Today’s been a difficult day. Ruby, the company figurehead and our beloved dog, was released from the pain of crippling arthritis that has blighted her recently. She’s been on all the drugs of late but as her mobility steadily decreased to the point where she’s been largely restricted to a single room of the house, her quality of life was clearly suffering.

We’ve been giving her every treat we can over the last week, in the knowledge this day was coming. She’s had Chinese crispy duck twice and I’ve had to deal with the consequences of the terribly rich diet she’s been having.

This morning I took her to the beach for one last look at the sea and fed her chicken breast and liver pate til she could eat no more. That’s a first for Ruby, not being able to eat another morsel.

You’ll be sorely missed, Ruby Roo. We love you.

31/10/2003 – 24/3/2018

New t-shirts/mugs “Qubits”

raclassic_teex2100101010-01c5ca27c6front-c3551317501000-bgf8f8f8-lite-1u1I set myself a brief to design a t-shirt that might be worn by the character Sheldon Cooper from The Big Bang Theory. I settled on Quantum Computing as the subject, specifically the quantum bit, or qubit. In traditional computing a bit is the smallest piece of information you can store, and each bit represents either a 1 or a 0.

A qubit, given the spooky powers of quantum physics exists as both 1 and 0 at the same time. The design I came up with forms the letter Q as a 3 dimensional object composed of pixels. From one side it would look like a 0 and from the other, a 1.

T-shirts and mugs now for sale on Redbubble and Society6.

WordPress and Google Pagespeed optimisation of blocking javascript

When you include scripts in your WordPress theme using the prescribed functions wp_register_script and wp_enqueue_script you can use the fifth parameter to force your script into the footer. Putting your script into the footer just before the closing body tag prevents it from blocking the page rendering.

Google’s PageSpeed Insights tool will advise you of any scripts that might be blocking rendering because they appear within the <head></head> element.
You may well do the right thing, but unfortunately with WordPress, you’re at the mercy of every other developer who’s plugins you may use. Should one of your plugins include its script in the head, and should that script rely on a function library, such as jQuery then WordPress will helpfully pull jQuery up into the head so the script doesn’t fail. Only now you have the jQuery library blocking rendering of your page. Even if you specified that jQuery be inserted into the footer, any errant plugin can ruin your plans.

So, what can you do? You can try installing another plugin to try and fix it for you (I’ve tried this with limited success) ,or you can hack the plugin. The problem with the latter approach is you’d have to redo it every time the plugin was updated, so I’ve worked out my own solution.

The wp_register_script and wp_enqueue_script functions both accept the $in_footer parameter and if you look at the code for those functions you’ll see it calls an internal WP method that adds some meta data to the script handle.

Using this method it’s possible to add the same meta data to any script handle you like, whether its in a plugin or not.

Step 1

Assuming you’re using jQuery dependent scripts, and you’ve set jQuery to be in the footer, but it’s not, examine the script tags in your head element and find the first one that isn’t part of the jQuery library. Search for the script filename within your site’s codebase.

Step 2

Having found where the script is registered or enqueued, the handle will reveal itself as the first parameter of the function call. e.g. foo_script

Step 3

Create a new function in your functions.php file, I’m calling mine “rr_force_plugin_js_to_footer”. Then use add_action to call your function. I’m not sure which is the most appropriate action on which to hang the function call but init seems to work. Your code should look like this


add_action( 'init', 'rr_force_plugin_js_to_footer' );

function rr_force_plugin_js_to_footer(){
$wps = wp_scripts();
$wps->add_data(‘foo_script’, ‘group’, 1); // repeat this line per script
}

Step 4

Work your way through each of the blocking scripts and add a new line to the function, just changing the handle you find each time. You’ll know when you’re done when the jQuery library scripts drop to the bottom of the page where you want them.

YMMV.

“Easy” install of free SSL certs from letsencrypt.org for Debian 8 (Jessie)

Letsencrypt.org is a new certificate authority that provides FREE SSL certificates for web hosting, but even better than the fact they are open, is the fact they’re entirely automated. No creating a CSR, no verifying your identity to a company 8,000 miles away or even waiting for an email. No downtime either. I’m running a Debian 8 server and I tried following their instructions, and those found at certbot.eff.org, but repeatedly hit dependency issues. Whether their documentation needs updating or not, if I ever need to do this again I need a set of instructions that actually work, so here it is.

You may still need to add the backports repository to your list of sources for apt, in which case:

%sudo nano /etc/apt/sources.list.d/backports-for-certbot.eff.org.list

and add the following line:

deb http://ftp.debian.org/debian jessie-backports main

With that done, the following lines should get certbot installed with the apache plugin which you’ll need to auto-configure each domain you want to serve securely.

%sudo apt-get -t jessie-backports install certbot
%sudo apt-get install libaugeas0
%sudo apt-get install python-certbot-apache

%sudo certbot --plugins

Output:

* apache
Description: Apache Web Server - Alpha
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.configurator:ApacheConfigurator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator

* standalone
Description: Automatically use a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator

That confirms that the apache plugin is installed. Now you just have to issue one command to install SSL for any domain on your server. It may be obvious to some, but I had Cloudflare services enabled for my domain and that prevented the install procedure from working so if you’re using Cloudflare you’ll have to turn it off for now.

*NOW* it’s easy.

    %sudo certbot --apache

It’ll ask you which domains to install for and present a list of all those you have in your virtualhosts:

    [*] example.com
    [*] www.example.com
    [*] another-example.com

It will install for all domains marked with an asterisk. To remove an asterisk you just use the cursor keys to move up and down the list, and hit space to change the setting.

Answer the questions and… job done.

WordPress filter to add HAVING clause to SQL

There’s posts_fields, posts_join, posts_where, posts_groupby but strangely there’s no WordPress filter named posts_having and it took many Google searches to find anything which might present a solution, and when I did find something it had precisely one upvote and hadn’t even been accepted as the answer.

It’s not necessarily a robust solution, it depends on your circumstances. Obviously to even use a HAVING clause you have to be grouping and have specific fields to compare. It seems a bit hacky, but what WordPress solution doesn’t begin with the words “it’s a bit of a hack but…”.

The answer is to add a SECOND posts_groupby filter and ensure it comes last so you can add the HAVING term manually.

add_filter('posts_fields', 'my_fields', 10);
add_filter('posts_groupby', 'my_groupby', 10);
add_filter('posts_groupby', 'my_having', 1000000);

function my_fields($sql){
    $sql.=' , MIN(`my_field`) AS `my_field` ';
    return $sql;
}

function my_groupby($sql){
    $sql.= (strlen($sql)?',':'').' some.id ';
    return $sql;
}

function my_having($sql){
    $sql.= (strpos($sql, 'HAVING') === false ? ' HAVING ' : ' AND ') . ' my_field >= 0 ';
    return $sql;
}

If you need more than one HAVING condition then just add another posts_groupby filter with a number one higher than the previous and repeat the check for HAVING in the filter function to ensure you only have one instance.

Obscure fix for AJAX requests that queue

I just found and fixed a major performance bottleneck in a web application I’ve been working on. The application consists of a single page populated with various dynamic graphs, the data for which is loaded via around 10 separate ajax requests which fire periodically to update the graphs and statistics.

It appeared, watching the network tab in Google Chrome developer tools, that each AJAX request was queueing behind the previous one. There was a clear progression in the response graphs with latency getting longer with each request (The green bit in the cart below).

Screen Shot 2015-10-15 at 20.50.26

All the requests were fired at the same time but returned sequentially, no matter how much work was being done to build each response. Another symptom of the issue is that it seemed impossible to navigate away from the page, or refresh the page without waiting for the last pending ajax request to complete. Since one of the ajax responses took up to 10 seconds to return, it meant a significant delay in moving away from the page and when testing by refreshing the page. I amended the javascript side of the app to abort all ajax requests on beforeunload but it seemed to make little difference to the latency issue.

The answer was to be found buried in an unaccepted answer with only 1 upvote (until now) on StackOverflow. What was happening was the PHP session was being locked by the first AJAX request. The PHP side of the app wouldn’t even accept the subsequent AJAX requests until the first had completed and freed up the session. And this happens for every single request, forcing responses to come back in the same sequence in which they sent.

This also meant that navigating to another page on the same site, or refreshing the page, also had to wait. If I were navigating away from the site entirely it wouldn’t have been an issue, but because the new page I was now requesting utilised the same PHP session it had to wait until all pending requests to that session were complete. This problem would not be evident on a site that didn’t use sessions to maintain state.

The solution is to free up the session as soon as you can during the responding PHP script by using

session_write_close();

After adding this line my network responses looked like this:

Screen Shot 2015-10-15 at 20.58.53

You can see now that the limiting factor is the number of permitted concurrent connections. The latency for each request is now only dependent on the time taken to compile each response, and later requests complete before earlier, time-consuming, ones. This has also had the effect of taking two whole seconds off the apparent loading time of the page and navigation across site has dramatically increased in speed.

The graph above also points towards a further optimisation that could be made. The thin grey bars in the chart occur when the request is waiting for a free connection to the server. Note that the length of the grey bars at positions 7 & 8 correspond to the latency of requests 3 & 2 respectively. If 7 and 8 were moved up to fire first then later requests that are awaiting connection would have grey bars equivalent in length to the green portions of these requests taking nearly 2 seconds off the apparent page load time.

Coda Plugin for WordPress Hook completion

WordPress hides a lot of functionality behind its filter/action hooks. Unfortunately these are generally inaccessible to many an IDE via code completion because they’re held within opaque strings. Coda, my preferred IDE, is affected, but since it’s extensible there are many plugins available to provide additional functionality.

As I was unable to find a plugin to perform this functionality I had a bash at writing one myself. I picked apart how this one by Matthew Woodard works and found a list of all the hooks here. You can download it by clicking the image below.

Screen Shot 2015-09-26 at 15.56.14It may be of limited use since as soon as you type the first ‘(‘ the autocomplete list disappears, so you can only ever choose from the full list rather than it converging on your typing as you go, but at least you can see what’s available. If I can figure out a better way to do this then I’ll update the plugin.

Yet another way in which your server might prevent file uploads

So, you’ve created a form for your application which allows file submissions. You’ve checked your php.ini file and made sure that post_max_size, upload_max_filesize and memory_limit have all been set large enough to accommodate your expected filesize.

You’ve got a hidden input field named MAX_FILE_SIZE and correctly set the value in bytes, but your file uploads are still failing. Well here are a couple more server settings to check which may be preventing your form from functioning.

Apache’s LimitRequestBody directive accepts a value in bytes. It *may* be possible to set this in your .htaccess file if your host will allow it.

Your host may also have restrictive settings within Apache’s mod_security config. Among others there is the SecRequestBodyLimit which will prevent http request exceeding a certain size.

It should be noted though, that modifying any of these settings in isolation may increase the attack surface area of your server and application. This is just a pointer to some possible causes of PHP file upload errors.