The Back Burner

The change of software in my last post meant that not only the domain, but all the page names and the entire structure of my recipe site changed — a big no-no for search engine purposes! On the other hand, the traffic this application gets isn’t actually worth anything to me in monetary terms — quite the opposite, it just uses bandwidth — so I wasn’t hugely bothered if it lost rank in Google for a while. In fact it was a golden opportunity to test the best way of handling this situation.

I started off by buying a brand new domain name, setting up MovableType, importing all the data, fixing up the design, and testing it to see that it all worked OK. At this stage it was a secret from Google. Once I was happy, I wondered just how I was going to keep all that precious traffic to the old site. I didn’t want to devote a huge amount of time to it, so I decided I’d just use 301 redirects (this is a header sent by the web server that tells the page requester that the page has “moved permanently”; there is also a “temporary redirect” status).

Thank goodness for dynamic pages! The old system used a single page called ShowRecipe.asp which took the id of the recipe and displayed it. So I just recoded this page to take the id, look up the recipe title, and convert it into the new URL (MovableType uses “search-friendly” URLs constructed from the recipe title). Then it does a 301 redirect to the URL on the new site. I removed / renamed the few other pages on the old site.

I added the new site to my Google webmaster tools page and monitored the results. Google came and spidered a few pages from the site within a couple of days, and I was soon seeing almost the level of traffic I’d had on the old site. Within a week there were 23 pages in Google’s index, which seemed not bad for a brand-new domain registered less than a fortnight before and with no incoming links yet, apart from our own site. As of now, Google has spidered again and picked up over 140 pages.

The process is evidently not finished yet though, as old redirected pages are still showing high up in Google (#6 for “pitta bread recipe” for example). Of course if anyone clicks on those results, they will automatically end up on the new site anyway, because of the redirect.

I’m probably not going to bother going after links to the old site and asking people to change them, but I would if this was a commercial site. Otherwise old linked-to pages seem to hang around in Google forever.

So at present just doing the 301 redirect looks as if it is going to work pretty well. If you completely change the structure of your site it’s well worth doing a specific 301 redirect for every page I think, for the benefit of human visitors rather than search engines.

OK, so this blog doesn’t get updated very often, but I have three blogs to look after now! Currently it’s all change on the blogging front. When I logged into the Back Burner just now, Google insisted that I upgrade to the new Blogger. The process appeared to be painless, except that the post entry screen now refused to accept any input — I typed happily away only to realise my words were disappearing into the ether. Yet another application that wasn’t tested in Opera . I fiddled in the settings and switched off “Compose” mode, and all is well now. It’s a shame that none of these WYSIWYG editing tools works in Opera; I don’t know whether it’s Opera’s fault, or just developers who don’t care about this small segment of the browser market.

In the meantime, I decided recently that it was time to revamp the recipe database which is currently responsible for much of the traffic to our company site (see Embarrassed by Success). When we created this demonstration CMS back in 1999, database-driven sites were a lot less common than they are now. It seemed inappropriate to keep this demonstration module on our site when it no longer reflects what we are capable of (we are much cleverer than that now!). It also used an Access database, which is hardly state-of-the-art. On the other hand I still find it a very useful system for storing my favourite recipes, and it gets a fair amount of traffic.

But time to completely redevelop the system (in Ruby on Rails for example) was lacking. So rather than reinventing the wheel again, I thought I might as well simply extract the content from the database and pour it into a blog. I experimented briefly with the latest blogging poster child, WordPress, before deciding I felt more comfortable with my old favourite Movable Type. This surprised me slightly, since Wordpress is written in PHP, which I am much more comfortable with than the “old-technology” Perl behind Movable Type. But The Movable Type administration interface feels much more polished and flexible than Wordpress’s and its templating system is easy to understand and manipulate. Not only that, but the Movable Type architecture allows you to generate PHP pages, so you can have the best of both worlds by incorporating PHP into your templates to provide dynamic elements.

Movable Type has moved on since I last installed it back in the days of version 2.66. It now incorporates a whole new plugin architecture, allows you to tag posts as well as assign them to multiple categories, has improved comment handling, and includes a style switcher so you can easily apply a theme without having to dabble in templates. It also easily handles multiple blogs and authors. Its major downsides compared to WordPress are that it doesn’t cater sensibly for static pages outside the blog structure, and its image handling is pretty rudimentary. Oh — and some parts of the admin interface don’t work properly in Opera!

Anyway, having installed Movable Type, grabbed a theme from MT’s style contest, and done a lot of cutting-and-pasting of content, I decided that the shiny new version of my recipe database deserved its very own domain. To my amazement, both larecettedujour.org and recettedujour.org were available, so I snapped them both up, and pointed them temporarily at a subdirectory of our company site. So … drumroll … here is La Recette du Jour in all its glory!

The task: provide a neat hierarchical tree as a web interface to a project management system where any project might have a variable number of sub-projects. A tree structure in other words. So I ended up with this:

Nice and neat… the only problem is that on the face of it, relational databases aren’t very good at handling trees, and after a bit of experimentation I was ready to write a whole lot of procedural code in ASP to take a recordset and build a hierarchy from it. But of course I googled first. The initial responses included Joe Celko’s (in)famous nested-set tree structure which was far too complicated for my feeble mind to grasp and didn’t look as if it would provide a simple interface for users to add new projects.

And then I came across Philip Greenspun’s interpretation of the classic problem of an employee table in which some employees are managed by others who in turn are managed by others, all the way up to the big boss. He used the Oracle-specific CONNECT BY PRIOR which I’d come across elsewhere, but explained it in a way I could actually understand. Just the ticket! So with a single SQL statement I now had my neat hierarchical recordset.

Next: how to present it. Serendipitously, I happened to be looking at Matt Kruse’s Javascript Toolbox for another reason and noticed a DHTML Tree. I like his stuff, so I had a look. Magic! A simple bulleted list is transformed into an expanding and collapsing tree with a few lines of CSS and a sprinkling of Javascript. The nice thing is that in non-CSS and non-Javascript browsers it degrades to a simple list so it’s accessible too. My brain steamed lightly as I wrestled with the logic to get the opening and closing list tags in the right places as I read through the recordset, but a couple of hours’ work had given me an ideal solution and in another half hour I had a simple form for adding new projects at any level in the structure. So humble thanks to Philip and Matt for being far cleverer than I will ever be!

When I used Windows I had a host of small, unobtrusive utilities that made my life easier. I’m sorry to say that in most cases I haven’t found Linux replacements which work as well. But they might work for you, so here they are:

Time tracking

Even if you don’t bill by the hour, if you are selling your services you need to keep track of billable hours to make sure you are estimating/charging appropriately. I like to use a stopwatch-style time tracker that simply runs in the background. On Windows, I used an excellent little utility called Allnetic Time Tracker. It sat discreetly in the system tray and with a right-click I could quickly start, stop, and switch tasks. The program has configurable idle-time detection, so if you forget to stop it and go away, you’ll be greeted on your return with a dialog box asking whether you want to discard the idle time or log it (the latter being useful if the reason you went away was to go to a meeting or do some non-computer-based task that still needs logging).

You can set up projects with tasks and sub-tasks, and everything is displayed in a neat three-paned window, making it easy to get an overview of total time at project and task level. Each logged time period can have a note attached to it, useful for detailing what you did. Allnetic also has very comprehensive reporting options — you could theoretically generate invoices directly, although I never did this.

Well, there’s nothing like this for Linux as far as I can tell. I tried a few applications and ended up with the default KDE tool, KArm. It does basically the same thing as Allnetic Time Tracker, but in a less sophisticated way; all projects are displayed in a single long list, which makes it rather unwieldy if you have a lot of tasks. It too sits in the system tray, but to start, stop or switch tasks you need to restore the window rather than being able to do everything with a right-click. It only keeps totals of hours for each day, not the actual times that you worked, and reporting is very limited; all you can do is copy either the totals or history between two dates to the clipboard, in text format (which is not suitable for loading into a spreadsheet for further manipulation). Still, it does the basic job that I need.

To-Do list

I don’t like bloated Personal Information Manager software — all I want is something that will let me keep an easily-accessible list of to-dos (including recurring ones), with due dates, priorities, and configurable reminders, and Quick To-Do Pro did it for me. I think it’s expensive for what it does (I see it’s now $39.95) but it fitted my needs so perfectly that I grudgingly paid up. Again I couldn’t find anything that worked that well for Linux — so I ended up using KDE’s Personal Information Manager. It’s full of other stuff I don’t want, and the To-Do list is basic, but it’s better than nothing.

Clipboard enhancement

Who can live with a clipboard that only stores the last item you copied? I can’t, and a lot of people obviously feel the same, because there are loads of clipboard enhancers for Windows, most of them suffering from feature bloat. I used a small, free utility called CLCL. The usual cut/copy/paste shortcuts work as expected, but a simple Alt-C in any application pops up a menu of all the items on the clipboard (not just text, but images and objects as well), so you can select the one you want, and you can also set up permanent entries for frequently-used text.

There isn’t so much choice on Linux, but KDE comes with its own utility, Klipper, so that’s what I ended up with. It works, but the clipboard doesn’t always contain what I expect, and there’s no facility for permanent items.

Accented characters

Not essential for everyone, but I do enough typing in French on an English keyboard for it to be an issue for me — there’s no way I want to type some convoluted code or use Windows’ character map every time I want to type a word with an accent in it. At last, this is something that Linux does better than Windows: you can configure a trigger key that works the way the Compose key on old DEC terminals did — press the selected key (e.g. Alt), then type a two-key mnemonic, e.g. typing , and c will give you ç. It’s simple to guess what the mnemonic will be, even for rarely used characters.

Windows doesn’t have anything like this built in, but we have used the free utility Allchars for years — it implements the Compose key functionality, but it also supports macros so you can set up mnemonics for longer pieces of text or key sequences (if you’re not already using CLCL for that …).

I’ve been developing websites with PHP for about 5 years now and I blush to confess that I have only just discovered I can use bind parameters with SQL, instead of painstakingly building up my SQL statements by concatenating strings, while remembering to deal appropriately with all potentially dangerous characters to avoid the risk of crashes or hacks.

Bind parameters are an unreservedly Good Thing. Not only do they make your SQL more readable, maintainable, and portable, but they also protect against the dreaded SQL injection which is the scourge of badly coded PHP websites developed by people who don’t know any better. And as if that wasn’t enough they can improve performance too.

I am no fan of reinventing the wheel, so since I first started using PHP I have used the open-source ADODB database abstraction library for database access. I like it because it’s fast and lightweight compared to its better-known competitor PEAR::DB. Tutorials are thin on the ground though, and I confess I only discovered its support for bind parameters recently, by reading between the lines of the documentation while looking up something else. Since then I’ve been gradually refactoring existing code — whenever I need to edit some code for some other reason I have a look for any embedded SQL and add some data-binding goodness to it. This is simple to do with ADODB. For example:

Before:

$SQL = "SELECT column1, column2, column3 FROM Tablename WHERE column1 = '$myvalue'";
$Result = $DB->GetRow($SQL);

having first carefully cleaned the contents of $myvalue of course!

After:

$SQL = "SELECT column1, column2, column3 FROM Tablename WHERE column1 = ?";
$Result = $DB->GetRow($SQL, array($myvalue));

Note I didn’t need to quote the string in the SQL statement, and the database library deals with any escaping necessary, in the appropriate style for the database being used — no MySQL-specific code here! In addition the SQL statement is pre-compiled — if you are going to execute the same statement a number of times with different values each time (e.g. when doing multiple inserts), this can really boost performance.

So what’s not to like? I don’t know why this technique isn’t better known among PHP developers. For example here is a lengthy thread discussing SQL injection, involving some apparently quite knowledgeable developers, and yet bind parameters are never mentioned; everyone pins their hopes on the stupidly named PHP function mysql_real_escape_string. Perhaps this will change now that support for binding is built into the PDO database access layer in PHP 5. Even if you are still using PHP 4.x you owe it to yourself to try a database abstraction library instead of endlessly hard-coding for one specific database engine.

Of course I should add that none of this means you can dispense with proper data validation before writing it to the database: the golden rule still remains “Never trust user input”!

These days practically every website has a “contact us” form allowing visitors to send email to the site owner without exposing the recipient’s email address to the world.

Lately I’ve been investigating ways that spammers can hijack these forms to send spam without the site owner’s knowledge. The fact is that it’s alarmingly easy to use an insecure form as a spamming engine, using a technique known as “header injection” — and the resulting spams will be clearly identified as coming from your site, resulting in possible blacklisting of the server, or even having your site shut down by your webhost for spamming.

The good news is that once you know exactly how the spammers can exploit the holes, it’s quite easy to guard against them. My researches soon turned up a page with a technical explanation of the topic. And from there I moved on to sites with examples of techniques for addressing vulnerabilities. Armed with this information it didn’t take me long to knock up a PHP function for processing all input from forms to strip out anything potentially dangerous. And knowing what I know now, I’d be reluctant to use any of the many free “form to mail” scripts without carefully checking whether they are vulnerable to this exploit.

I should stress that although the examples I’ve used are PHP, this problem is not a weakness of any particular scripting language or web server software — it’s simply a result of the fact that when standards for email headers were developed, the Internet was a kinder, gentler place than it is now, and spam wasn’t even a blip on the horizon.

Every January, assorted pundits claim, “This is the year Linux will conquer the desktop!” Every year, they are proved wrong, and Windows continues to power 95% of desktop PCs (web servers are another matter; a very sizable majority of those run some flavour of Unix).

Well, my experience so far suggests that Linux is ready for the desktop — kind of. Using Xandros for everyday tasks is barely distinguishable from using Windows, and many mainstream PC users would be quite happy with it. I had no trouble at all finding replacements for mainstream Windows applications; Xandros comes with pretty much everything you need out of the box. For someone who makes fairly basic use of an office suite, as I do, the free Open Office is a more than adequate replacement for Microsoft Office, and it will save in Microsoft formats, so you can continue to exchange documents with MS Office users.

As for web browsers, you are spoilt for choice — the only unavailable choice, of course, being Microsoft’s very own Internet Explorer! I carried on using Opera, just as I did on Windows, but I could have chosen Firefox or Konqueror. Email was slightly more difficult, since Eudora, the indispensable email client I’ve been using since 1996, is only available for Windows and Mac. I tried Thunderbird, the companion email program to Firefox, didn’t like it, and ended up using Opera’s built-in email facility. It’s idiosyncratic but it has some interesting features, including threading and excellent search and filtering capabilities. For those wedded to Outlook, Evolution is said to be a good alternative.

I’ve already covered text editors in a previous post — Quanta Plus is simply the best editor I’ve ever used, on any platform. Linux has a built-in firewall, and as for anti-virus — I don’t bother!

All of those cover about 90% of my daily work. But what I find I really miss from Windows is the selection of handy little utilities I’d accumulated over the years, that chugged unobtrusively away in the background, helping me manage my workload. It’s turned out to be very hard to find good open-source replacements for these. Of course I could run them using Windows emulation software, but this seems to me to defeat the object of switching. Coming up next: my review of utilities I’ve discovered so far.

A few years ago, when dynamic, interactive websites were not as common as they are now, we decided to showcase our database skills by setting up a simple demonstration website that would allow site visitors to edit the content online. Looking around for some suitable application I hit on my small collection of recipes that I’d been emailing to friends, and the Archetype recipe database was born.

I created two versions — the “play” versions, that visitors could edit, and a live uneditable version that contained myoriginal recipes. I’ve since found it a genuinely useful application — when I find a recipe in a magazine or on a website that I particularly like, I add it to the website and I can then easily find it again, no matter where I am.

But rather embarrassingly, it seems a lot of other people find it useful too. Our website is supposed to promote our web design services, but the recipe pages have become by far the most popular on the site — without bringing in hordes of prospects eager for us to make them a dynamic website too! Looking at the stats for search terms used to find our site is revealing:

… and so it goes on. I’m sure it can’t do our site much good in the search engines, in terms of ranking highly for web design!

But perhaps I should see this as an opportunity. When I realised how popular these pages were, I thought I might as well put some Amazon ads on them, but those have earned me about one cookbook every three years. Now I’ve decided to try putting Google AdSense ads on them — we’ll see if those do any better. So it’s turned out to be a useful space for experimentation with revenue models, and it helps me get aninsight into the economics of web publishing.

More generally, looking at the terms people use to find your site can be a useful exercise. It may alert you to new services you could offer them, if they are searching for products and services related to your business, but which you don’t currently offer. Hmm, on this evidence maybe we should start bottling and selling lemon butter sauce …

A well-known fact about the history of the microcomputer is that it took a “killer application” to make it worth buying a PC: something so useful that it justified the cost of the computer. Back in the early 1980s, that killer app was Visicalc, the first-ever spreadsheet application, which had Apple II computers pouring off the shelves, snapped up by eager accountants.

In a similar way, my Xandros PC sat on the spare desk, providing a development web server and little else, until I found my very own killer app which justified moving that PC onto my desk and dumping the Windows PC on the spare one. The application that did this for me was Quanta Plus — a web editor.

A web editor? But there are dozens of great web editors for Windows! Surely you don’t need Linux just to run an editor? Well, it’s not quite as simple as that. I spend nearly all my working hours editing program code in a text editor — no fancy WYSIWYG tools for me. Since 1993 and Windows 3.1, that editor has been TextPad, which had got welded into the core of my being. I tried other, allegedly superior text editors from time to time but somehow I just couldn’t get along with them and always found myself back with simple, reliable, uncluttered TextPad. It wasn’t perfect, further development seemed to have ground to a halt, but I knew every quirk and feature and it was tuned for just one job — editing text files quickly and efficiently. In fact it was one reason the Linux box was still on the spare desk — I tried half a dozen Linux editors and didn’t like any of them.

Then in October a student on an Open University course I teach said that he couldn’t imagine creating web pages without Quanta. Out of curiosity I visited the site and thought it was worth a try.

I was just starting work on a large PHP website, so it seemed sensible to try Quanta for this. I started it up, had a quick scan through the help files, and set up my first project. Two hours later I was completely hooked. It was one of those rare occasions when you find an application that thinks exactly the way you do. It was the “tipping point” … and within days the Linux box was on my desk with Windows relegated to the outer darkness.

It is interesting to see how something you never knew you needed can prove itself indispensable!

I’ve had a spare computer by my side running Linux for a couple of years now. I started out with a very old, slow PC that struggled to run an ancient version of Red Hat Linux. It was unusable as a desktop machine, but did the job as a development web server for testing sites before unleashing them onto a live server.

When I upgraded my desktop PC (or rather Steve bought a new PC and I got his cast-off, the normal pattern in our office) I decided it was a good opportunity to try Linux on a more realistic platform: my discarded 733 MHz Pentium III. I didn’t want to spend days fiddling about trying to get things working though, so after due research online I plumped for recently-released Xandros, trumpeted as a newbie-friendly release.

It certainly did what it said on the box: I popped the Xandros CD into the drive, clicked a few buttons, and then left it to install. Within half an hour I had a working system — it was easier to install than Windows! I was favourably impressed by the fact that everything worked immediately: all devices recognised, Internet connection working, even file-sharing with the Windows PCs on the network was easily achieved. And it looked very nice too, with a desktop environment barely distinguishable from Windows.

After that, I installed XAMPP, a one-stop way of getting Apache, PHP and MySQL running so that I could continue to use the PC as a development server. That too went smoothly … it seemed Linux was truly “ready for the desktop” — but Xandros stayed on the spare desk and I continued to work on my trusty Windows 2000 PC.

Next instalment: the application that convinced me to push the Windows PC aside and put the Linux box on my desk!