Running as Root

Sure, I could have named the post something like "Laying Pipe" but that's a bit obvious, isn't it? As my two blog readers have no doubt noticed (with some annoyance), I've been playing around with Bitstrips and throwing together the occasional comic (complete with my first comment directed at how shitty my "artwork" is). One problem with Bitstrips (other than the fact they co-own everything you do) is that they don't have an RSS feed for either your comics or your series. I opened a bug on the matter but rather than sit back and wait for it, I decided it might be time to have some more fun with Yahoo! Pipes.

The Plan

Bitstrips provides a link to a pseudo feed for each series. The series ID is passed in as a parameter and you get a paginated view of all of the episodes. I figured I'd just grab that page in pipes, create a title off of the episode title, grab the associated URL for the episode, grab the image of the strip for the episode, and throw the whole thing together to make a feed for my series.

Execution

Numerous problems occurred after I start implementing that simple plan–most of the problems being in the Bitstrips layout. I won't go into detail about it, but here's a summary of the steps I went through to get my feed:

• Get the series "feed" page - http://bitstrips.com/feed.php?feed=s_9713 in this case
• Find the URL of whatever page is associated with the "go to last" button - The feed page outputs the series in order. For the feed, I want the most recent stuff. There is no option to just go to the last page, so I need to find that button, grab its associated URL, and then retrieve that URL as my new starting point.
• Regex some fields - I use the regular expression module in Pipes to pull out values for the title field, the publication date, and the link for the item.
• Clean up the date - The date isn't in the proper format, so I need to use the Date Formatter and re-assign the value to "item.y:published". This is how you get a pubDate into an RSS feed from Pipes (thank you Yahoo! discussion forums). This all happens inside of a loop.
• Sort - I want to include this feed into a spliced super feed, so I want to sort it by publication date. However, the publish date in the comic has no time stamp. Luckily, the URL for the comic uses an auto-incrementing ID, so I just sort by that.
• Get the image for the comic - Now we follow the link that we got earlier that goes to the individual episodes. We do this inside of a loop to get images for each item. After that loop, we use a regular expression to get rid of all the extra HTML around the image so we're just left with a nice img element.
• Ship it! - We now have the title, a link to the comic, and the image of the comic. We're done.

Results

Here's what the pipe looks like from way up here:

You can see the results of the pipe here or even examine the "source code" a little more closely by editing it (requires a Yahoo! login and you can't actually edit my version, so there).

Of course, this is a very brittle and ill-advised way of doing things. As soon as Bitstrips changes their site my pipe will burst into flames and spew all sorts of errors into the feed. There is also probably an easier way to do it either in or out of Yahoo! Pipes, but I had fun doing it this way. I haven't decided if I want to put this feed into a jumbo super feed yet or not, but that is most likely the way I'll do it. For now, subscribe if you want but it may go away or be duplicated in a feed to be named later.

This entry was posted on Tuesday, April 8th, 2008 at 1:51 pm and is filed under Technical. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.