Proof of Global Warming
Ladies and Gentlemen, I present to you irrefutable evidence of global warming.
See this map of the world, created prior to global warming, Greenland and Antarctica are huge!
Now I present this more modern map, both Greenland and Antarctica have melted away to remnants of their former glory.
For the future of Greenland and Antarctica, please, do your part to stop global warming.
An Automattic Adventure
Today I officially begin my role as a TransCoder1 at Automattic. As a TransCoder, I’ll be doing new and exciting things with VideoPress, our video hosting service.
For the last 4 years or so, I’ve been working at MySQL (which was swallowed by Sun, which in turn was consumed by Oracle), based 100% out of my home office. The thread of working as part a distributed company is something I’m very happy to be continuing at Automattic.
What else is cool?
Impact is cool. Automattic is around 100 people, but WordPress.com is one of the top 20 websites in the world, getting millions of visitors a day. Look at that employee:visitor ratio in the table over there. Isn’t that a beautiful thing?
Getting things done is cool. The smart people at Automattic2 have done a great job of making continuous deployment scale as they’ve grown, so engineers can rapidly develop, launch and iterate features. There’s nothing quite like launching a new feature, seeing it run millions of times an hour, then tweaking it based on how people are using it.
Working with awesome people is cool. I’ve had the privilege of getting to know the folks at Automattic recently, and even meeting a few of them. When work turns from the usual day-to-day, into hanging out with friends and doing interesting things, you know you’re onto something.
The Automattic Creed
I don’t think I’ve ever worked for a company that has a creed, but I like this one.
I will never stop learning. I won’t just work on things that are assigned to me. I know there’s no such thing as a status quo. I will build our business sustainably through passionate and loyal customers. I will never pass up an opportunity to help out a colleague, and I’ll remember the days before I knew everything. I am more motivated by impact than money, and I know that Open Source is one of the most powerful ideas of our generation. I will communicate as much as possible, because it’s the oxygen of a distributed company. I am in a marathon, not a sprint, and no matter how far away the goal is, the only way to get there is by putting one foot in front of another every day. Given time, there is no problem that’s insurmountable.
What will happen to my other projects?
PonyEdit development is continuing strongly. We’re working on a bunch of serious performance improvements, expect to see it out soon.
Job Manager is slowly waking up from the long winter of bug-fix releases only. With a little luck, I’ll be able to start making some serious inroads on the next feature release.
I also have some ideas I want to play around with, and now sounds like a good time to see what works. Stayed tuned. 
One more thing…
If you want to join in the fun, go and check out our Work With Us page, and apply! You may end up becoming my TransCoding sidekick (we’ll get superhero costumes, and our own theme music).
Avoiding HTC Locations
I recently updated my HTC Desire HD to Android 2.3, which is quite nice.
Unfortunately, HTC has used this upgrade to force its Locations app into the system – if you select an address in Calendar, then Locations will open, even if you have another mapping application set to default.
Happily, if this is something that bothers you, there’s an easy workaround for this.
- If you haven’t already, install a file browser app – I use ASTRO File Manager.
- If you haven’t already, register an account on Android Forums.
- Go to this post, and download the attached file, com.google.android.calendar.apk. This is the default Android Calendar app.
- Plug your phone into your computer, so it appears as an attached drive, and copy com.google.android.calendar.apk to your phone. Once it has finished copying, disconnect your phone.
- Open ASTRO File Manager, and locate com.google.android.calendar.apk. Select it, then use the “Open App Manager” option.
- Install it.
- Go to your Apps, and open the Calendar app. The first time you open it, it’ll crash – you can just Force Close it.
- Open the Calendar app again, and open an event – it will ask you which Calendar app you want to use as default, select the one with the Blue calendar icon.
Okay, so there are a few steps involved, but if you’re familiar with side-loading apps, it shouldn’t take you more than a few minutes.
After this, individual events will open in Google Calendar, which does use your default mapping application. It won’t disable HTC Locations completely, but it’s the best option we have.
Finally, an open letter:
Dear HTC,
Respect system defaults.
Love,
Android users everywhere.
PonyEdit: It’s really fast
Over the past few months, a friend and I have been hard at work on a new breed of text editor that we call PonyEdit. If you’ve ever found yourself cursing over the lag of working on remote cloud servers, this is the editor for you.
It’s not just another SFTP editor…
Reading and writing files over SFTP is nothing new; dozens of text editors can do it. But it’s always slow, clunky and feels like the feature was bolted on as an afterthought. You’ll find yourself using separate shortcuts to open files locally vs remotely, and dealing with sometimes painful save times with every edit, no matter how minor.
PonyEdit gets rid of this terribly slow method of working by connecting over SSH, and using edit streaming to push changes to the server in the background as-you-type.
Head on over to PonyEdit.com to download a free trial, and let me know what you think! Oh, and…
Stand by to have your mind blown.
Preventing Users From Accessing wp-admin
If you have a WordPress site that you allow people to sign up for, you often don’t want them to be able to access wp-admin. It’s not that there are any security issues, you just want to ensure that your users are accessing your site in a predictable manner.
To block non-admin users from getting into wp-admin, you just need to add the following code to your functions.php, or somewhere similar:
add_action( 'init', 'blockusers_init' ); function blockusers_init() { if ( is_admin() && ! current_user_can( 'administrator' ) ) { wp_redirect( home_url() ); exit; } }
Ta-da! Now, only administrator users can access wp-admin, everyone else will be re-directed to the homepage.
Partitioning the WordPress Comments Table
WordPress sites can get big. Really big. When you’re looking at a site of Cheezburger, Engadget or Techcrunch proportions, you get hundreds of comments per post, on dozens of posts per day, which adds up to millions of comments per year.
In order to keep your site running in top condition, you don’t want to be running queries against tables with lots of rarely accessed rows, which is what happens with most comments – after the post drops off the front page, readership drops, so the comments are viewed much less frequently. So, what we want to do is remove these old comments from the primary comment table, but keep them handy, for when people read the archives.
Enter partitioning.
The idea of MySQL partitioning is that it splits tables up into multiple logical tablespaces, based on your criteria. Running a query on a single partition of a large table is much faster than running it across the entire table, even with appropriate indexes.
In the case of the WordPress comments table, splitting it up by the `comment_post_ID` seems to be the most appropriate . This should keep the partitions to a reasonable size, and ensure that there’s minimal cross-over between partitions.
First off, we need to add the `comment_post_ID` column to the Primary Key. This can be a slow process if you already have a massive `wp_comments` table, so you may need to schedule some downtime to handle this. Alternatively, there many methods for making schema changes with no downtime, such as judicious use of Replication, Facebook’s Online Schema Change Tool, or the currently-in-development mk-online-schema-change, for Maatkit.
ALTER TABLE wp_comments DROP PRIMARY KEY, ADD PRIMARY KEY (comment_ID, comment_post_ID);
Now that we’ve altered this index, we can define the partitions. For this example, we’ll say we want the comments for 1000 posts per partition. This query can take a long time to run, if you already have many comments in your system.
ALTER TABLE wp_comments PARTITION BY RANGE(comment_post_ID) ( PARTITION p0 VALUES LESS THAN (1000), PARTITION p1 VALUES LESS THAN (2000), PARTITION p2 VALUES LESS THAN (3000), PARTITION p3 VALUES LESS THAN (4000), PARTITION p4 VALUES LESS THAN (5000), PARTITION p5 VALUES LESS THAN (6000), PARTITION p6 VALUES LESS THAN MAXVALUE );
When you’re approaching the next partition divider value, adding a new partition is simple. For example, you’d run this query around post 6000.
ALTER TABLE wp_comments REORGANIZE PARTITION p6 INTO ( PARTITION p6 VALUES LESS THAN (7000), PARTITION p7 VALUES LESS THAN MAXVALUE );
Naturally, this process is most useful for very large WordPress sites. If you’re starting a new site with big plans, however, you may just want to factor this into your architecture.
UPDATE: Changed the partition definition to better reflect how WordPress uses the wp_comments table, per Giuseppe’s comments.
Job Manager HOWTO: Customizing the Job List
A common problem I hear people having with Job Manager is that they don’t know how to customize how the Job List looks. In this post, I’ll try to break down how to write your own Job List templates, so you can get it looking pretty.
First of all, open your wp-admin, and go to Job Manger->Settings->Display Settings. Scroll down to the box called Job Templates. Like many other plugins allow in Posts and Pages, Job Manager templates use shortcodes. All of the shortcodes are described in this text box, but I’ll only be using a handful of them here to demonstrate.
The default template shows an expanded view of each job, but many sites only want a summary table, so people can click through to the job. Here’s an example of that:
To start with, decide which fields you want to show in the table. From the default fields, you might decide to use the Job Title, the Location, and the Start Date. On a default installation of Job Manager, these have the shortcodes [job_title], [job_field4] and [job_field2], respectively.
Now we can start writing our template. Start with the table header:
<table>
<tr>
<th>Title</th>
<th>[job_field4_label]</th>
<th>[job_field2_label]</th>
</tr>Notice that we’ve used the job_fieldi_label shortcodes. These will be replaced by the Field Labels you can define in Job Form Settings.
Next up, we need to define the Job Loop. [job_loop] is a special tag, because it has a start tag ([job_loop]) and an end tag ([/job_loop]). Everything between those tags will be repeated for every job being displayed, which means we only have to write the HTML for one table row, like so:
[job_loop]
<tr>
<td>[job_link][job_title][/job_link]</td>
<td>[job_field4]</td>
<td>[job_field2]</td>
</tr>
[/job_loop]Notice that we’ve also used the [job_link] tag, which will make the Job Title link to the full Job description.
Finally, we just need to close the table.
</table>
Naturally, you might choose to include extra things in your template, this is a very cut-down example. But hopefully it should give you some ideas for you how can expand the templating system to create wonderful new layouts!
What I’m Listening To
That 1 Guy – Packs A Wallop!
Heavy metallic beats, rocking guitar, gravel filled voice – there’s no mistaking this for anything other than That 1 Guy‘s latest album.
Unlike his previous albums, which consisted of songs that had been performed and refined in front of audiences, Packs A Wallop! was composed and constructed entirely in the studio, “I went in there with zero – nothing finished – and I lived there for a month. It was really hard work, really long days, and I forced myself to craft some stuff.”
As it turns out, this was a good plan. It opens with the heavy Modern Man, and meanders its way through various styles, eventually winding up at Stones Throw, one of the chillest tracks I think I’ve ever heard. When I copied the CD to my iPod, the genre field gave an excellent one word description of this album – Unclassifiable. That’s always been That 1 Guy’s modus operandi, and I’m happy to hear it continuing.
He’s also renowned for putting on a great show, and just happens to be in the middle of an Australian tour right now, playing Melbourne tonight and Sydney tomorrow. You should probably check him out.
Brandon Flowers – Flamingo
As the distinctive voice of Brandon Flowers wafts around the room – a little bit nasal, a little bit clean, strangely hypnotic – one is forced to consider how much of The Killers personality is made by Flowers alone. The style is very much like where The Killers having been heading (not that this is a bad thing).
In case you forgot (hardly likely), Flowers is a Las Vegas boy, he reminds us all with the opening track, Welcome to Fabulous Las Vegas. This sets the tone for the entire album, the glitz of Vegas is almost overwhelming.
I enjoyed this album immensely, as will any fan of The Killers. If I were allowed one complaint, it would be that this album is a little too Killers-esque, I really would have liked to hear Flowers experiment some more. Certainly, his voice ties him strongly to The Killers, but that doesn’t mean he can’t mix things up – compare Bernard Fanning‘s solo album Tea & Sympathy to his work as lead singer of Powderfinger.
Finally, the album art: if you need a reminder that Flowers is quite a pretty man, you need look no further than here. Consisting of photos in and around Vegas, Flowers looks at home in a 70′s themed hotel, surrounded by showgirls, or out in the desert. You have to hand it to a guy who can pull off the “double denim” look without looking like someone who escaped the purge of early 90′s fashion.
Billy Corgan – The Future Embrace
Billy Corgan – his name can’t be mentioned without bringing up certain associations. Brilliant musician. Artist. Egomaniac. Bald. The Future Embrace is a few years old now, I just happened to have gotten around to picking it up recently. I’m happy to report, it ain’t half bad.
Like Brandon Flowers, Corgan struggles to separate his style from that of The Smashing Pumpkins (We’ll pretend that Zwan never existed). That said, his cover of To Love Somebody is worth the price of admission alone.
On a mostly unrelated note, Billy Corgan is also the source of one of my greatest regrets. Allow me to construct a scene for you:
You’re sitting at a table in a smoky, dimly lit bar. In a nearby corner, a small stage is illuminated by a single spot light, a high stool and a microphone stand all that’s on display. You cough, and take a sip of your drink as a shadowy figure approaches the stage and seats himself, starting pensively at the floor for a moment. The beret stops most of the glare of the light reflecting off the bald head underneath. Slowly, he looks up, and launches into the most pretentious poetry you’ve ever heard in your life.
And now for the back story: whilst walking around Bologna one day, I noticed a poster on a wall for a poetry reading by a Mr B. Corgan, his unmistakable dome hiding behind the large title text. Unfortunately, in a daze of imagination, conjuring images of how wonderfully terrible it would be, I wandered off forgetting to note down the date and venue. When I went back the next day the poster was gone, no doubt souvenired by an enthusiast fan. To this day, I’ve been left wondering what it would have been like to listen to a recital of the poetry of B. Corgan.
Grinderman – Grinderman 2
Oh, Nick Cave. I try hard to like your music, I really do. But I just can’t get into it. Don’t get me wrong, I like the stuff that ends up on radio, and Into My Arms playing at the end of He Died With a Fellafel In His Hand is one of my most memorable movie moments. But every time I listen to one of your albums, it just grates against me. It’s cool, I still like you – I just didn’t really like this album.
The Key of Sea
This is an album of pairs. Australian music and World music. Popular Australian musicians and unknown musicians from around the world, who’ve come to live in Australia. A great CD and a great cause.
Amongst many others, there are two styles of music I like: modern Australian indie, and world music. As luck would have it, the good folks behind The Key of Sea project were kind of enough to provide us all with exactly that combination. The idea of this project was to pair “artists that you already know and artists who have brought their incredible musical skills and cultures to Australia. It represents the efforts of an entire community who joined together in search of a common goal.”
From Australian music main stayers like Tim Rogers, Blue King Brown and The Cat Empire, through to some of the newest popular bands like Oh Mercy, Urthboy and Philadelphia Grand Jury, this album has something for everyone. Combined with the talent of international artists like Group 120, Diafrix and Yousif Aziz, you get a bit of a strange mix that just happens to work really well.
For this album, I can only recommend that you go out and buy it – there’s great music, and you’re supporting an excellent cause.
Welcome, SkySQL!
It seems the SkySQL website just went live, which I hope will breath some life back into the MySQL ecosphere – it’s been a while since there’s been some new competition, especially in the style of classic MySQL services.
For those too lazy to read the SkySQL site, the services offered are similar to what you’d be familiar with from Oracle:
- SkySQL’s Consulting and Training are pretty much the same as Oracle’s existing offerings, though a bit more limited. I expect this to grow as SkySQL grows, however.
- SkySQL’s Support has been simplified slightly, removing the equivalent of MySQL Enterprise Gold support.
- They are offering monitoring and query editing, (which Oracle offers with MySQL Enterprise Monitor, and MySQL Workbench) through what I can only assume are branded versions of Webyog‘s MONyog and SQLyog. A smart move on their part – rather than having to develop something in house on a startup budget and timeframe, they can offer a mature product off the bat.
Sadly, prices aren’t listed, so we can’t really compare that.
I do hope to see SkySQL evolve further – I count many of SkySQL’s founding employees as friends, and I know they won’t stop at just offering the same services as Oracle. I’m always a fan of a bit of friendly competition! 
Lycka till, SkySQL!
Below the Line Voting Data: Roll Your Own Analysis
I’ve had a few people ask where I got the data for my last couple of posts, so I thought I’d put together a quick how-to for performing the same analysis, or running your own queries. I’ve used MySQL for this, but you should be able to easily convert these statements to work with your data store of choice. (Feel free to post them in the comments!)
First of all, you’re going to need to get the raw data. The AEC provides this as a bunch of CSV files, available for download on their site. The 2010 data can be found here, the 2007 data can be found here.1 Grab the National list of Candidates, and the State Below the Line Preferences files.
Next up, create some tables to hold this data. Here’s the table definition for the candidates table:
CREATE TABLE `2010_candidates` ( `state` varchar(3) NOT NULL, `party` varchar(5) NOT NULL, `party_name` text NOT NULL, `candidate_id` bigint(20) NOT NULL, `surname` text NOT NULL, `name` text NOT NULL, `elected` varchar(1) NOT NULL, `historic_elected` varchar(1) NOT NULL, KEY `party` (`party`), KEY `candidate_state` (`candidate_id`,`state`) ) ENGINE=InnoDB;
You’ll also need to create a table for each state. You could put it in one big table, but I was doing all of this on my local machine with a limited amount of RAM, so I wanted to keep the tables small:
CREATE TABLE `2010_prefs_nsw` ( `candidate_id` bigint(20) NOT NULL, `preference` int(11) NOT NULL, `batch` int(11) NOT NULL, `paper` int(11) NOT NULL, KEY `candidate_pref` (`candidate_id`,`preference`) ) ENGINE=InnoDB;
Importing the data is pretty easy. Just run this query for the Candidate file:
LOAD DATA LOCAL INFILE "/path/to/SenateCandidatesDownload-15508.csv" INTO TABLE 2010_candidates FIELDS TERMINATED BY ',' IGNORE 2 LINES;
Similarly, run this query for each state (modifying for the correct file name and table name, of course).
LOAD DATA LOCAL INFILE "/path/to/SenateStateBTLPreferences-15508-NSW.csv" INTO TABLE 2010_prefs_nsw FIELDS TERMINATED BY ',' IGNORE 2 LINES;
You now have all of the raw data. I preferred to make a summary table to hold the preference count for each candidate:
CREATE TABLE `2010_prefs_summary` ( `candidate_id` bigint(20) NOT NULL, `preference` int(11) NOT NULL, `preference_total` bigint(20) NOT NULL, KEY `candidate_pref` (`candidate_id`,`preference`) ) ENGINE=InnoDB;
Populating this table is pretty easy. Just repeat this query for each state (note the reference to the 2010_prefs_nsw table, and the “NSW” WHERE clause):
INSERT INTO 2010_prefs_summary ( SELECT c.candidate_id, p.preference, count( p.preference ) AS preference_total FROM 2010_candidates c LEFT JOIN 2010_prefs_nsw p ON c.candidate_id=p.candidate_id WHERE p.state="NSW" );
Now, grabbing the preference data for any candidate is quite easy, you just need to know their candidate_id (look it up in the 2010_candidates table):
SELECT * FROM 2010_prefs_summary WHERE candidate_id=12345;
Have fun! If you come up with anything interesting, let me know on Twitter, I’d love to hear about it!
- I’ve contacted the AEC, but it seems they don’t have data available from before the 2007 federal election – I assume they just didn’t store the raw counting data. ↩
Gary Pendergast, noun: Gamer and developer. MySQL, WordPress, web and music geek. Makes interesting things at Automattic. Can be observed in his natural habit at pento.net, or usually in Melbourne, Australia.
My Photo Blog
