Graphical Programming with Yahoo Pipes

Let us start with the need. I found a tool to allow me to scroll RSS feeds across my desktop, just like every news channel.  This is a neat thing, but I found that I didn't like the way it did RSS feeds. Instead of cycling through them, it kept using one. So, I set about solving that by combining feeds.

The method I chose was Yahoo Pipes. When I decided to check Craigslist for items of certain types (Fender guitars, especially) in local areas (where I decided I could reasonably drive, should the catch of a lifetime come around), I used Pipes, mostly as an excuse to learn it. With Google Reader being the recipient of those feeds, I didn't need to massage them too much. The news I fed News was twitter feeds from news sites I wanted to follow, mostly local news and weather.

To the right, we see the "code" I used to combine seven news sources, (mostly) remove redundancies, do some slight editing to the content and ensure that only the day's news shows up. It's all Twitter RSS, so there's one date format, but that format is not usably sortable. I format today's date in the same format and filter out if it isn't there, but the better solution is to take a Unix timestamp of the current time and date (representing seconds since Jan. 1, 1970), subtract 24*24*60 (number of seconds in a day) and drop out entries that were not published during that time. That will take a slightly larger amount of cleverness to do that, but I think I'll crack that nut eventually.

The thing I notice is, while a graphical solution seems tailor-made to mimic flowcharts, there's no splitting here. It's all very linear, with minor loops and filters. I suppose I could get a more funnel-looking program if I handled the inputs separately. The more I play with Yahoo Pipes, the more I like it.