Archive for the 'Programming' Category

Cookies and Funky Characters

Thursday, July 24th, 2008

Today's weird problem had to do with a browser cookie not keeping the value I gave it. The cookie's value is an encrypted string. I noticed in the debugger that what was written out was not what was read in later. The decryption failed because the encrypted value retrieved from the cookie just didn't make sense. Apparently cookies are sent as HTTP headers and can only use US-ASCII characters minus whatever other special characters don't work (you can wade through several RFCs if you like). In my case it was trailing "="s that were being eaten.

The easiest solution (besides just not using those characters) is to encode the values. Something like base64 or using the URL encode/decode utility classes in Java would easily do the trick.

The more interesting thing is that we use other cookies to store all kinds of strings, sometimes internationalized funky character containing strings. Those Unicode characters also don't get handled very well in cookie values so there is a general need to encode/decode them. Maybe it's just safest to always encode on the way out and decode on the way in.

Chalk up another one for things I probably should have known but didn't. Yay!

  • Share/Bookmark

Migrating SQL Server 2005 to MySQL

Tuesday, July 22nd, 2008

Here's a quick note on migrating data from MS SQL Server to MySQL. I found out in a white paper entitled A Practical Guide to Migrating From Microsoft SQL Server to MySQL (free registration required) that the MySQL install (on Windows at least) includes a tool called the MySQL Migration Toolkit.

It has a simple wizard that steps you through all of the options and you're ready to migrate. You can even choose to migrate "live" or save the scripts to files. One minor annoyance is that you can't pick the name of the target database/schema. When migrating from a SQL Server database named "blahblah" with default schema of "dbo", the migration tool will create "blahblah_dbo" in MySQL. I thought I would just rename it only to find that support for RENAME DATABASE has been dropped.

Instead, I decided to just do a backup and restore said backup to a new schema. I then ran into a problem with one of our badly designed tables that has large (not huge) amounts of BLOB data. The restore died with an error saying "MySQL Server has gone away." I finally tracked down the issue which can be fixed with a small configuration change. Since I was testing this in MySQL for Windows, I just added max_allowed_packet=12M to the my.ini under the server directory and bounced the service. Everything worked nicely after that. Your size on that configuration option may vary, of course.

  • Share/Bookmark

Virtualization as Adoption Criteria

Wednesday, June 4th, 2008

Yesterday at work I got a question from a co-worker about doing something under AIX. I don't really work with AIX, but I managed to answer the question. This is for another project that I don't work on. It got me to wondering if I could run AIX in a VM on PC hardware, probably using a PPC emulator. I'll save you the suspense and tell you that I couldn't find a way to do it. There are a couple of projects that are sort of trying to do it but they haven't done it successfully that I could find. Those would be PearPC and QEMU. I'm sure one of these projects will get it working someday, as soon as some really capable programmer wants it badly enough. I briefly thought about suggesting purchasing an RS6000 from eBay but decided it wasn't a project I needed to poke my nose into.

This is not a rant against AIX, although in a world gone mad with a billion distros of Linux, OpenSolaris, and PC hardware that is criminally cheap and available I don't feel the need to brush up on that incredibly rusty skill set. No, this is more about that fact that I don't think I would ever choose to work with any operating system and/or software product that I can't run in a VM on PC hardware–it's just too damn handy these days, especially when it comes to QA. That of course includes any non-hacked up version of MacOS, not that they'll miss my business. Also, sadly, I don't always get to pick what I want to work with. Still, it's definitely something to consider when picking your stack and deployment/hosting environment.

  • Share/Bookmark

Who Pays These People to Code?

Wednesday, May 28th, 2008

I ran across a post in my RSS feeds today that referenced a paper on Bypassing Web Authentication and Authorization with HTTP Verb Tampering. What an awesome title. I'm immediately adding "verb tampering" to the list of things I randomly exclaim in meetings. The short version of the paper (that's represented pretty well by the other blog post) is:

  • some developers secure URLs in their web application by URL and method (POST, GET, etc). Everything else is allowed (for some strange reason)
  • some servers when they receive an invalid HTTP method or often HEAD will perform a GET and then just discard the body of the response. This is fine and is part of the RFC apparently, since the headers have to match between the two. I just wasn't aware of the fact
  • there are still programmers that have non-idempotent GETs in their applications

The scenario is you find these applications / servers and do something like send a HEAD to the URL "deleteUser?userId=27" and then the server does it, despite the fact that you're not logged in.

I'm amazed that this is a problem, for multiple reasons. Who are these people that still don't understand that you don't use GET to do things like delete records from your system? I'd hate to see what one of those crazy internet spiders could do to these guys.

This is also a reason why I'm a big advocate of pushing your security checks as close to the data as you can comfortably stand. Certainly you should have protection at a service or DAO layer to prevent users with inadequate permissions (unauthenticated users fall into this category) from performing most operations in your system. This is also a good practice to ensure that different types of potential front ends don't accidentally grant access to the wrong users. The URL level of security is just icing and fluffery to make the application a little more user friendly.

Of course, that being said I work on an application that relies on mostly on URL level restrictions (I didn't do it and I'm working on changing it) and, if I remember correctly, so does my favorite Java web stack. I should point out that neither of these suffer from the problem described in the paper.

  • Share/Bookmark

Google Android and Grand Central

Monday, April 7th, 2008

I'm enamored with the idea of Google Android. I briefly looked at the development kit and was very impressed by it. And then I did nothing with it. Part of the problem is that there is no Android capable phone at the moment. That takes some of the sex appeal out of developing for it. More than that is the fact that I don't have any ideas for applications that I'm all that passionate about.

This weekend I heard that Google acquired Grand Central. They're a company I've never heard of but they've got an impressive list of features. Grand Central is a "web-based voice communications platform". It'll let you do all sorts of cool things with your phones via a single number.

This got me to thinking about cool things you could do with Grand Central and Android. Where I work, we use a product called Contactual to route calls for our support number to any other phone number. You can also use their web interface to place an outgoing call and have the resulting call sent to your phone. This is in place so support technicians don't inadvertently give their personal phone number to a customer via caller ID. Once a customer gets a personal number they think that they have a buddy in technical support that can't help them out whenever they run into a snag, rather than going through the proper channels.

What would be interesting would be to see an Android/Grand Central application that would make a similar scenario as easy as dialing an ordinary number. Perhaps allowing you to select any of your phone numbers from a drop down when making the call.

Many of the other features listed on Grand Central's page also look like they'd be an excellent fit with an Android phone.

  • Call Record – Use the phone to signal to the server that you want to record your current call
  • Block Callers – Why not use Android to let you mark an incoming call's number (or current call) as a blocked number?
  • Call Switch – Use your phone to transfer your current call to any of your other numbers.

All of these things are possible with Grand Central and become that much cooler if you can use your phone to do them more directly. It seems like a great tie in for an upcoming product and a recent acquisition all in the same related problem space. Of course, I'm far too lazy and stupid to code any of it, but it still sounds cool.

  • Share/Bookmark

MD5 Hash Database Population

Tuesday, November 27th, 2007

I read a post recently about how someone found a password via a search for its MD5 hash on Google. Within the comments someone mentioned a site that had a database and search engine for hashes. The part I found clever was that the database of hashes self populates whenever someone uses the site to calculate a hash off of a string of plain text. I'm sure they populate from another source as well, but the last just struck me as interesting.

  • Share/Bookmark

On the Telephone!?

Sunday, September 16th, 2007

In the Future, Everything Will Be Glue

The entry title is from Weird Science, by the way:

Gary: I was crazy for this little eighth grade bitch.
Wyatt: Crazy, Insane!
Gary: I was dedicated to this girl, I called her every damn night!
Old Pimp Dude in the Bar: You called her every night? On the telephone?
Gary: On the telephone? What's he mean on the telephone, course it was on the telephone!

In my work life, the team I'm on has been working for quite some time fixing the architecture for an existing product. One of the things we've been doing while we've been fixing things is to add a REST API. You can save the REST vs WS-* debate for some other time and place.

Meet the New Product, Same as the Old Product

Unfortunately, there was a distinct lack of excitement about the new stuff we'd been doing from other areas of the company. Programmers seem to immediately "get" why exposing all of the CRUD functionality of your system through a web API is the sheer sweetness in terms of application integration. Sales and executive staff don't always see the possibilities. They saw the new version as architecture changes that improved performance and scalability while maintaining functional parity with the shipping version. That didn't scream sexy.

The programmers, ever fearing being a cost center with a very killable project, decided to start creating proof of concept integration examples that would create internal demand for the nearly finished version.

Pretty State Machine

As background, our system has an operational state that changes the way almost everything behaves. The change in this state is typically user driven. Someone using the software would decide to change the operational state due to an external event such as a Martian invasion. The software has a web interface but we had the idea that it'd be much cooler to change the operational state via a telephone.

We can change our operational state via the REST API by putting (as in HTTP PUT) an XML document representing the system status to the server. The system status has an element that represents the operational state. Now, our IT guy is always telling me the crazy stuff he can do with his home VOIP system. He runs a virtual machine with Asterisk and other assorted software an combination with an IP phone. I ran the idea of executing a shell script from a phone menu by him and he assured me it should be easy to do.

I was working under a deadline (of course) so I created three versions of the XML (one for each state I would be changing to) and got a copy of cURL installed in the Asterisk VM to PUT each document to our server, hopefully in response to someone pressing a button on a phone. After hitting a very serious dead end and wasting several hours trying to get an IVR working (that's what phone people call those voice menus) I finally found an example of an Asterisk iTunes controller. To use it you dial an extension, hear a beep, then press a button to do something. The script in Asterisk then calls a shell script in response to your button press. Five minutes after finding it I had a working example of changing operational state in our product via a telephone (actually a softphone but who's really keeping score at this point). Booyah! Sure it only took me 10 hours discover how to do an hour or so worth of work but then that's the plight of the knowledge worker.

Demo Day

The next day we invited additional people to our iteration demo meeting (really it's just the people that were supposed to be going all along but weren't) and showed the phone demo as our finale. Suddenly, people seemed as interested in our new version of the product as the developers are. There was a palpable sense of excitement. There were tears of joy. The developers were hoisted up onto the shoulders of sales and management and carried out into the streets (think about the scene in Dragon: The Bruce Lee Story).

I'm exaggerating a little bit perhaps. Of course there are all sorts of negatives preceding and following this event, but can't we just bask in the glory of this one little success for just a few moments? On the telephone.

  • Share/Bookmark

Black, Blue, and Purple

Thursday, June 21st, 2007

How I waste my free time

In between trying to get a simple Arduino project thrown together and farting around with the Yahoo! UI Library, I took a timeout to play around with Firefox add-on (back in my day they were extensions) named Stylish.

The subject came up on Jyte that black text on a white background wasn't the friendliest design for some people. Unfortunately, most sites don't support user specific skins so if the site's design causes you problems, from a vision related impairment for instance, then you're just shit out of luck. That's where Stylish comes in. It lets you create custom CSS overrides on a per site basis.

Can you see me now?

Yet another thing to do when you're bored: see if you can completely change the color scheme of a site to make it hideously ugly for no good reason. It took a little work, but I managed to make a really nasty looking blue and purple on black style for Jyte.

blackjyte

Most of the work was just finding all the nasty little inherited background colors, background images, etc. One interesting problem came up because Jyte has a set of menus that are images of text. They have a white highlight built into them and look like complete crap on the new black background. I thought it wouldn't be a problem to turn links with images into links with the image alt attribute as the link text and then hide the image.

Let the CSS fun begin

I thought maybe some sweet CSS action like this would work:

.nav_item a img:before {
  content:attr(alt);
}

.nav_item a img {
  visibility:hidden;
  width:0;
}

Oh, how lovely that would be. Apparently it'll work in some builds of Firefox but got turned off in later versions. I think it was a feature request specifically to make my life more difficult. :before and :after pseudo elements for other elements work fine, just not images.

So, with a lot of trial and error I wound up with the less than ideal:

.nav_item a[href="/home"]:before {
  content:"Home";
}

.nav_item a[href="/home"] img {
  visibility:hidden;
  width:0;
}

Since I no longer had access to the alt attribute on the image I had to manually insert the correct link text for each menu item. I did this for each menu by using an attribute selector to find each link by its href attribute, then hard coded the content to the correct text and finally hid the image.

You nasty, nasty boy

Oh, it's nasty all right, but it works. The only problem is I was trying to make the CSS overrides generic and there's a dynamic menu that includes the user name. Luckily there's only one such dynamic menu so I just made an entry to handle all such menus as the dynamic one and let the more specific rules clean up after the fact. Ta da! A custom, site specific CSS override that no one will ever use because it looks like it was beaten with a bag of hammers and makes your eyes hurt to read.

The interesting thing is that there are now ways for users to make the sites they frequent more appealing and possibly more accessible via add-ons like Stylish and Greasemonkey. It's not ideal for non-technical users, but sites like Userscripts and Userstyles can help by acting as repositories with aids created by the more technical members of the community.

  • Share/Bookmark

Blockbuster, Tivo, and Greasemonkey

Friday, March 2nd, 2007

Someone asked for a Blockbuster version of the Netflix Greasemonkey script I wrote. I signed up for the free two week trial and got it working, I think. It's not very different from the original Netflix version, just some stripping of extra characters like "|WS|" or "|Unrated|" that Blockbuster adds to the title. Feel free to download it from here.

gm_blockbuster_tivo

Update: BlockBuster made some minor changes that broke that version of the script. I've made some minor changes to fix things. I updated the link in the post to point to the new one. If you can't be bothered to find that link, you can get it here.

Update: I updated URLs to which the script applies and added quotes around the search title to get more exact matches. As always, you can get it here.

Update: Blockbuster started including the year of the disc in parentheses which would throw off the search so I stripped that information out before hitting the TiVo search site. I also changed the link to the TiVo site so that it would open in a new window/tab. I found myself always shift clicking the link so I just put it in the script.

  • Share/Bookmark

Netflix, Tivo, and Greasemonkey

Monday, February 26th, 2007

I wasn't very satisfied with my last attempt at merging Netflix and Tivo. I'm sure it can be done using just Pipes eventually but I think it'll take a while for Pipes to mature enough.

Barring that, I got curious this weekend and decided to see if I could do the same kind of thing directly on the Netflix queue page via a Greasemonkey script. So I wrote a quick little script over the weekend:

gm_netflix_tivo

The quick overview is that the script takes each movie title in your queue and searches for the text on TiVo's site. If it finds a match it puts a link after the movie to TiVo's search page. I used a link so you can open the matches in other tabs. This is especially handy since it takes a while to search TiVo for all the movies in your queue. I also tried to make the link stand out a little so it would be easy to spot while scrolling through the page. Besides being a slow loader, the other downside is that any match will cause the link to appear. This is a little bit annoying but seems bearable. In theory you would then schedule the recording online via TiVo's site.

If you're interested, go install Greasemonkey followed by this script. Feel free to modify the hell out of it if you like. I realize it's still not a perfect solution, but seems a bit useful for now. Also, please excuse the crudity of the code. Like I said, it was a quickie and Javascript is not my forte.

Update: Here's a Blockbuster version.

Update: I added the quotes to the embedded search. I no longer have a Netflix account, so it's not the easiest thing for me to test out. Let me know if there are any problems. The script is now hosted at userscripts.org.

  • Share/Bookmark