Running as Root

Today's weird problem had to do with a browser cookie not keeping the value I gave it. The cookie's value is an encrypted string. I noticed in the debugger that what was written out was not what was read in later. The decryption failed because the encrypted value retrieved from the cookie just didn't make sense. Apparently cookies are sent as HTTP headers and can only use US-ASCII characters minus whatever other special characters don't work (you can wade through several RFCs if you like). In my case it was trailing "="s that were being eaten.

The easiest solution (besides just not using those characters) is to encode the values. Something like base64 or using the URL encode/decode utility classes in Java would easily do the trick.

The more interesting thing is that we use other cookies to store all kinds of strings, sometimes internationalized funky character containing strings. Those Unicode characters also don't get handled very well in cookie values so there is a general need to encode/decode them. Maybe it's just safest to always encode on the way out and decode on the way in.

Chalk up another one for things I probably should have known but didn't. Yay!

Here's a quick note on migrating data from MS SQL Server to MySQL. I found out in a white paper entitled A Practical Guide to Migrating From Microsoft SQL Server to MySQL (free registration required) that the MySQL install (on Windows at least) includes a tool called the MySQL Migration Toolkit.

It has a simple wizard that steps you through all of the options and you're ready to migrate. You can even choose to migrate "live" or save the scripts to files. One minor annoyance is that you can't pick the name of the target database/schema. When migrating from a SQL Server database named "blahblah" with default schema of "dbo", the migration tool will create "blahblah_dbo" in MySQL. I thought I would just rename it only to find that support for RENAME DATABASE has been dropped.

Instead, I decided to just do a backup and restore said backup to a new schema. I then ran into a problem with one of our badly designed tables that has large (not huge) amounts of BLOB data. The restore died with an error saying "MySQL Server has gone away." I finally tracked down the issue which can be fixed with a small configuration change. Since I was testing this in MySQL for Windows, I just added max_allowed_packet=12M to the my.ini under the server directory and bounced the service. Everything worked nicely after that. Your size on that configuration option may vary, of course.

Yesterday at work I got a question from a co-worker about doing something under AIX. I don't really work with AIX, but I managed to answer the question. This is for another project that I don't work on. It got me to wondering if I could run AIX in a VM on PC hardware, probably using a PPC emulator. I'll save you the suspense and tell you that I couldn't find a way to do it. There are a couple of projects that are sort of trying to do it but they haven't done it successfully that I could find. Those would be PearPC and QEMU. I'm sure one of these projects will get it working someday, as soon as some really capable programmer wants it badly enough. I briefly thought about suggesting purchasing an RS6000 from eBay but decided it wasn't a project I needed to poke my nose into.

This is not a rant against AIX, although in a world gone mad with a billion distros of Linux, OpenSolaris, and PC hardware that is criminally cheap and available I don't feel the need to brush up on that incredibly rusty skill set. No, this is more about that fact that I don't think I would ever choose to work with any operating system and/or software product that I can't run in a VM on PC hardware–it's just too damn handy these days, especially when it comes to QA. That of course includes any non-hacked up version of MacOS, not that they'll miss my business. Also, sadly, I don't always get to pick what I want to work with. Still, it's definitely something to consider when picking your stack and deployment/hosting environment.

I ran across a post in my RSS feeds today that referenced a paper on Bypassing Web Authentication and Authorization with HTTP Verb Tampering. What an awesome title. I'm immediately adding "verb tampering" to the list of things I randomly exclaim in meetings. The short version of the paper (that's represented pretty well by the other blog post) is:

• some developers secure URLs in their web application by URL and method (POST, GET, etc). Everything else is allowed (for some strange reason)
• some servers when they receive an invalid HTTP method or often HEAD will perform a GET and then just discard the body of the response. This is fine and is part of the RFC apparently, since the headers have to match between the two. I just wasn't aware of the fact
• there are still programmers that have non-idempotent GETs in their applications

The scenario is you find these applications / servers and do something like send a HEAD to the URL "deleteUser?userId=27″ and then the server does it, despite the fact that you're not logged in.

I'm amazed that this is a problem, for multiple reasons. Who are these people that still don't understand that you don't use GET to do things like delete records from your system? I'd hate to see what one of those crazy internet spiders could do to these guys.

This is also a reason why I'm a big advocate of pushing your security checks as close to the data as you can comfortably stand. Certainly you should have protection at a service or DAO layer to prevent users with inadequate permissions (unauthenticated users fall into this category) from performing most operations in your system. This is also a good practice to ensure that different types of potential front ends don't accidentally grant access to the wrong users. The URL level of security is just icing and fluffery to make the application a little more user friendly.

Of course, that being said I work on an application that relies on mostly on URL level restrictions (I didn't do it and I'm working on changing it) and, if I remember correctly, so does my favorite Java web stack. I should point out that neither of these suffer from the problem described in the paper.

Sure, I could have named the post something like "Laying Pipe" but that's a bit obvious, isn't it? As my two blog readers have no doubt noticed (with some annoyance), I've been playing around with Bitstrips and throwing together the occasional comic (complete with my first comment directed at how shitty my "artwork" is). One problem with Bitstrips (other than the fact they co-own everything you do) is that they don't have an RSS feed for either your comics or your series. I opened a bug on the matter but rather than sit back and wait for it, I decided it might be time to have some more fun with Yahoo! Pipes.

The Plan

Bitstrips provides a link to a pseudo feed for each series. The series ID is passed in as a parameter and you get a paginated view of all of the episodes. I figured I'd just grab that page in pipes, create a title off of the episode title, grab the associated URL for the episode, grab the image of the strip for the episode, and throw the whole thing together to make a feed for my series.

Execution

Numerous problems occurred after I start implementing that simple plan–most of the problems being in the Bitstrips layout. I won't go into detail about it, but here's a summary of the steps I went through to get my feed:

• Get the series "feed" page - http://bitstrips.com/feed.php?feed=s_9713 in this case
• Find the URL of whatever page is associated with the "go to last" button - The feed page outputs the series in order. For the feed, I want the most recent stuff. There is no option to just go to the last page, so I need to find that button, grab its associated URL, and then retrieve that URL as my new starting point.
• Regex some fields - I use the regular expression module in Pipes to pull out values for the title field, the publication date, and the link for the item.
• Clean up the date - The date isn't in the proper format, so I need to use the Date Formatter and re-assign the value to "item.y:published". This is how you get a pubDate into an RSS feed from Pipes (thank you Yahoo! discussion forums). This all happens inside of a loop.
• Sort - I want to include this feed into a spliced super feed, so I want to sort it by publication date. However, the publish date in the comic has no time stamp. Luckily, the URL for the comic uses an auto-incrementing ID, so I just sort by that.
• Get the image for the comic - Now we follow the link that we got earlier that goes to the individual episodes. We do this inside of a loop to get images for each item. After that loop, we use a regular expression to get rid of all the extra HTML around the image so we're just left with a nice img element.
• Ship it! - We now have the title, a link to the comic, and the image of the comic. We're done.

Results

Here's what the pipe looks like from way up here:

You can see the results of the pipe here or even examine the "source code" a little more closely by editing it (requires a Yahoo! login and you can't actually edit my version, so there).

Of course, this is a very brittle and ill-advised way of doing things. As soon as Bitstrips changes their site my pipe will burst into flames and spew all sorts of errors into the feed. There is also probably an easier way to do it either in or out of Yahoo! Pipes, but I had fun doing it this way. I haven't decided if I want to put this feed into a jumbo super feed yet or not, but that is most likely the way I'll do it. For now, subscribe if you want but it may go away or be duplicated in a feed to be named later.

I'm enamored with the idea of Google Android. I briefly looked at the development kit and was very impressed by it. And then I did nothing with it. Part of the problem is that there is no Android capable phone at the moment. That takes some of the sex appeal out of developing for it. More than that is the fact that I don't have any ideas for applications that I'm all that passionate about.

This weekend I heard that Google acquired Grand Central. They're a company I've never heard of but they've got an impressive list of features. Grand Central is a "web-based voice communications platform". It'll let you do all sorts of cool things with your phones via a single number.

This got me to thinking about cool things you could do with Grand Central and Android. Where I work, we use a product called Contactual to route calls for our support number to any other phone number. You can also use their web interface to place an outgoing call and have the resulting call sent to your phone. This is in place so support technicians don't inadvertently give their personal phone number to a customer via caller ID. Once a customer gets a personal number they think that they have a buddy in technical support that can't help them out whenever they run into a snag, rather than going through the proper channels.

What would be interesting would be to see an Android/Grand Central application that would make a similar scenario as easy as dialing an ordinary number. Perhaps allowing you to select any of your phone numbers from a drop down when making the call.

Many of the other features listed on Grand Central's page also look like they'd be an excellent fit with an Android phone.

• Call Record - Use the phone to signal to the server that you want to record your current call
• Block Callers - Why not use Android to let you mark an incoming call's number (or current call) as a blocked number?
• Call Switch - Use your phone to transfer your current call to any of your other numbers.

All of these things are possible with Grand Central and become that much cooler if you can use your phone to do them more directly. It seems like a great tie in for an upcoming product and a recent acquisition all in the same related problem space. Of course, I'm far too lazy and stupid to code any of it, but it still sounds cool.

I bought Lisa a new laptop a couple of months ago and the only OS option was Vista. I can't get her onto the Linux bandwagon unfortunately. Yesterday she was trying to print something only to find out that I hadn't set up her printer on the new laptop.

We have an HP Photosmart 1215 which sadly doesn't have (and apparently never will have) Vista drivers. I tried the XP driver only to find that it didn't quite work. Luckily, I found this post saying that the drivers for the 7200 should work. I tried them and they did. Yay, internet!

I might point out that I had no problem getting the printer working with Linux–you know, the hard to use operating system. Gee, I don't know why people are hating Vista so darn much.

I read a post recently about how someone found a password via a search for its MD5 hash on Google. Within the comments someone mentioned a site that had a database and search engine for hashes. The part I found clever was that the database of hashes self populates whenever someone uses the site to calculate a hash off of a string of plain text. I'm sure they populate from another source as well, but the last just struck me as interesting.

I added The Linux Action Show! Podcast to my listening list today and got immediate benefits. I nearly stopped listening within the first minute since the two guys try and do the fake "radio guy" voice a little too much. Luckily they leveled off a short bit later and began to sound pretty much like any two random tech people I know. One of the items in their news section mentioned that a new version of VirtualBox is available.

I'd never heard of VirtualBox before, but I'm a big fan of virtualization and I liked the features they were mentioning. It came at a great time for me as I have been debating buying VMWare Workstation for home use, replacing the free VMWare Player (I also use VMWare server on a dedicated machine). I'd been getting a little irritated at the lack of snapshots in Player lately.

I'm on Ubuntu and I use a Windows VM for assorted applications that don't work under Linux (and are flaky under Wine). Occasionally I feel like trying a random Windows program out. I'd like the option to roll back the VM to a known good state in case anything happens. VirtualBox has an open source version and an enterprise version that is free for personal use–both have snapshot support. I tried out the latter, which also has an iSCSI feature I want to play with later (which is supported by FreeNAS). My initial impressions are very good. It supports snapshots, automatic guest OS desktop resizing, and automatic keyboard and mouse capture and release. Plus it's free for me. Those are the big things I want in a desktop VM application. The performance is also somewhat snappier for several of the programs I use regularly under the Windows VM.

I'll probably still need VMWare player for some more exotic operating systems that might not work 100% with VirtualBox, but that's fine. I'll be sure to update this if anything starts to go badly wrong, but for now it looks like I've got a new VM solution at home. Color me happy.

I ran out of space on my home file server about 4 or 5 months ago. Since drives have become so cheap I went out and got a pair of 500GB Seagates with the intention of running a software RAID (level 1, two drive mirroring–let's have the 4/5 debate some other time). As long as you're not doing a software RAID using Windows, my understanding is that it'll be stable and reasonably performant. Besides, I'm not an IT shop, just a home network and I also didn't feel like purchasing and dicking around with new hardware for this project. Of course, being lazy I just now got around to actually doing the work.

My first idea was to just put good old Ubuntu server on there and set up Samba, SSH, etc myself. It's not all that difficult to do. Then, I ran across a project calls FreeNAS. It's a FreeBSD-ish network attached storage project that borrows some of the configuration pages / scripts from m0n0wall (another project I started using instead of Freesco or Coyote Linux for my routing and firewall needs).

After deciding to give FreeNAS a try I pulled down the live CD, currently version 0.685RC1, and burned a CD. I slapped the drives into the old box (400Mhz Pentium leftover), threw in a CD and booted up. Running in this configuration I would boot from the CD each time, read the specific configuration from a floppy, and use the full space of both drives for data. It took a few minutes to figure out the correct order of doing things (add drive to configuration, format drive, create raid volume, format raid, create mount point, configure CIFS, configure SSH, add users and groups) but the whole thing took well under an hour. You have the option to use the web interface to configure everything, back up your config floppy, and even add and rebuild new drives in the RAID.

Once I was done with all of that I had a file server I could access from both Windows and Linux. I moved some files out there and played around testing the performance. Everything seems good. Yay. Finally a home IT project that wasn't a huge pain in the ass. If nothing breaks I'm a convert. I'll just have to move over the old files at some point and I'll be fully done.