2014/04/18

The Graphics Interchange Format in Conversation

  • For me, it's a solved issue. When the inventor of a thing says it's pronounced a way, I give that weight.



    Similarly:



  • For me, it's a shibboleth. I have been involved in web development since 1996, and if you pronounce it "Gif" instead of "Jif", that's a sure sign that you're a noob. There's nothing wrong with being a noob, but there's certainly nothing right.
  • For me, it's also not a big thing. If you pronounce it the "wrong" way in a conversation with me, I won't correct you or start an argument about it. Starting an argument over things that don't matter, is dickery.



    Continuing one
    , on the other hand, isn't so bad.
  • The "It's not gramatical" argument cuts no ice with me. Saying the G has to be a hard G will always have me wondering how you'd pronounce "a German Giraffe", and while you say "But it comes from graphical so it must be a hard G", I flash to Police Academy 4: Citizens On Patrol, notice that "Citizens On Patrol" acronyms to COPs, and wonder if you would have me pronounce it "sops".
  • Yeah, I know going back to a Police Lobotomy movie as an argument is not too strong, but my mind has better things to do than develop and parse through a list of acronyms. Believe me, I've tried, and my mind has said "Let's watch YouTube videos. We could browse the web for pictures of smiling dogs. We could consider why your SQL tables are stupid. We could think about Steve Gutenberg movies. Really, anything but the acronym list thing." 

2014/04/11

About Me


I've been working on my resume recently. No reason, really. (If you have questions, ask elsewhere. I'm fairly easy to find.) A recent response to my resume tells me it sucks at telling the story of me. Not too surprised; the form sucked before, and has become the gameboard for competative Reverse Buzzword Bingo, where if you find the right combination, you get the job, which makes it counterproductive to even try to tell your narrative.

So, this is the story of me:


My first degree was a BS in Journalism and Mass Communication. My greatest experience was as copy editor of my student paper, where I did full-page layout of the State-and-Nation wire news section to our paper's style standards, collected the weather forecast (which I cribbed from the regional newspaper's front page by calling a friend who worked in a gas station) and read every dang word in the paper. I got this position by copy-editing a copy of one week's front page, where I not only found the errors they introduced to test me, but also errors that made it into print that week. The biggest lessons I learned were that I was really good at finding miniscule errors in large blocks of text, I was pretty good and making an attractive and readable layout if I knew what sort of look we were shooting for, and if you're really physically and mentally exhausted with more work left to do, fruit juice is better than soda.

Also, I learned that when people say "If everyone from Dan Rather to the guy who writes obit for the local fish-wrap were to die tomorrow, there would not be enough positions opened up to employ all the people who graduate with journalism degrees this year", that's not just a punchline, that's a suggestion that you change majors as soon as this conversation ends.

xkcd
After not finding work in journalism, I went back to school, where I received a BS in Computer Science, and worked for the university as a web developer. As with xkcd, my time with Perl has been much more useful in my post-collegiate career than the courses I took.

Also, the things I learned as a j-school student -- finding small errors in large text blocks, making text blocks readable and attractive, and handling exhaustion -- are incredibly useful for CS students and web geeks, too.

During that time, I found that what I loved to do was converting things to other things. If you sent this email address a message with this string in the subject and a URL in the body, it responded with that URL's web page as the body. If you went to a certain web page, you could send an anonymous email. I fell in love early with RSS and wrote many things that read it and many other things that wrote it. I wrote a thing for my home page that displayed one guitar if I had new mail waiting, and another guitar if I didn't. I once helped write a tool that scrolled RSS headlines across the top of a web page. My goal and hope was to be a developer who turned things into other things, preferably with web technologies on a Unix/Linux platform while contributing to Open Source projects, in an organization whose primary work is creating software.

A thing I didn't really learn is that, in the last year of your college career, your primary goal is finding something to do with your life after graduation. Preferably, something where you are paid. Successfully completing your courses and graduating is necessary but secondary. So, I found myself as a VMS admin and occasional Microsoft-focused web developer for a medical clinic. I often made things out of other things: I recall parsing the four-digit long-distance records by long-distance code and sending a list of calls to the user of record for each code. It was okay, but not in line with my desired workplace as above.

After that, I came to work at the university again, for a research lab. We do gene sequencing; I do full-stack web development. I did some of the stuff I like to do: I turned web forms into database updates. I turned database tables into web tables. I turned CSV into plots with R. I used Perl and Linux and Unix and eventually R. I coded  daily.

For a while, I worked through a temp service with a defense contractor. Mostly I maintained a lab. On occasion, I set up specialty PCs. I used a visual pre-processor that eventually made C code, which wasn't fun. The pay was somewhat better, but of the things in the goals-and-hopes bit above, I did nothing.

So, I came back to the lab again. I gained more responsibility, and continue to make things from other things. I turn database dumps into XML-filled ZIP files that our instruments use to handle the sequences we give them. I used Sikuli and Jython to script a Windows application to turn a bunch of XML files that held UUencoded BLOBs into searchable XML files. I turned database dumps into JSON, and turned JSON into web page tables and Javascript-created plots. I write libraries that make it easy turn things on various machines into messages on my Linux box, on my Windows box, on my phone.

My one-true-language is Perl. Most everything I write, I first try in Perl, and only go away from it if I can't make it work there. I code in JavaScript: Perl doesn't run on the web browser, so I have to. I'm trying Node.js, and am slowly finding my way toward usefulness. I'm writing more and more R, because it's the shortcut for making plots.

I dislike Python. On first hearing about it, I thought, "What's the point? It sits in the same niche as Perl, but isn't Perl. Why use it?" My first experience with it was some code which displayed available machines in an on-campus computer lab. It has been handed to me as "this works", but it was indented with tabs, and somewhere along the line, a space had been inserted before a tab, so it didn't work. I get readable code, and my journalism degree forced me to appreciate whitespace for readability, but here something I couldn't see ruined my day. My Sikuli project was written with Jython, the Java implementation of Python, because that's the only choice, and my FitBit Tools are in Python because I couldn't easily get OAuth working with Perl. I could see using it more, but I write Perl in a Perl shop right now.

The Greyjoys of the Iron Islands Do Not Sow; I Do Not Make Web Pages. I make tools. I make things that turn things into other things. This can be web, but it will be dynamic, either at the server side or the client side. If you want a web page -- something static, a billboard for the web -- you will be happier and I will be happier if someone not me made this page. For design, I can get you 50-75% of the way to an attractive and useful web tool, and probably more if given a standard to align to, but design is not where my strengths lie and are not where my interests lie.

This is me. This is the tools I like to use, the tasks I like to complete, the environment I like to be in. Right now, I am a developer, working with web technologies in a Unix/Linux environment, doing more but not enough to contribute to Open Source projects. I'm part of an organization that's (writ large) education and/or (smaller) assisting biological research. This means I'm not quite where I want to be, but better than I was for many years. So, I'm looking, but not looking too hard.

2014/03/14

My Body Plans For 2014: March and April, plus Status Report

First, my status report:

Caffeine:

The darker, the more cups recorded. I drank coffee on weekends. I often had more than 1 cup. I sometimes had more than two cups. I didn't drink it late, but I drank more of it. I consider this sort of a fail.

Weights: In February, I participated in a month-long triathlon. I rode lots of bike. I swam miles like it ain't no thang. I did not lift much. So, right now, I'm struggling to throw around the 20lb dumbbells, much less throw more reps of the 25lb dumbbells. But I did get the t-shirt.

Feet: I've started to wear my inserts again. I haven't developed a the series of range-of-motion exercises, much less committed to doing them daily. I have been walking a lot, which means I've been getting the activity without so much the stomping on my ankles.
Blue: max/week. Green: min/week. Orange: avg/week. Yellow: AHA goal of 10,000 steps.  Purple: Personal goal of 6000 steps.

Tools: I wrote a tool, more working against MongoDB than anything else, that checks and stores my steps per day from FitBit, and tell me if I haven't been taking more than 100 steps per hour.

Weight: I have made a new all-time-low weight, 208 lbs, but that was when I was out sick. Otherwise, I'm seeing a winter-month plateau. So, while I am not where I want to be, and see more jiggle than I want to see, I am not depressed about my progress.

Endurance: By the rules of the month-long triathlon, all steps counted and I knew that, by my normal walking, I could easily handle the distance. I have been doing other endurance exercises, but I haven't been doing anything to push myself in walking into running.

I think I have been consistent with my goals but not so much my plans. I think I haven't been doing it as good as I could, but I've been OK.

In part, I think I don't push myself nearly enough when I'm walking, so I'm starting back to the treadmill, and while the action will continue to be "walk on the treadmill until I hit my 10,000 steps" when I show up, I'll start trying to push the speed and elevation to keep my heart rate raised.

I have a pointer to a weather API to make the "Bike Today, Dag-nabbit!" tool.

That's enough for now.

2014/03/06

It started out with wanting to take in mass quantities of "ruin my day". I wrote a program that would traverse a directory tree and run Perl::Critic on everything with pl and pm suffixes. This would bring out all the depression in all the code I've written in the last five years.

It got to ~/bin/TESTING and crashed. This the program it crashed on:


Slight, isn't it. You wouldn't figure it out at first.

. This code works; you can have Unicode characters in variable names in Perl.

What you can't do is have Unicode characters in variable names in PPI, which means you can't have Perl::Critic analyze Perl programs with Unicode characters in variable names.

I have reported this bug to Perl::Critic. I don't expect them to be able to fix it, because it isn't in their code.

I have also reported this bug to PPI. Making everything Unicode-safe will be difficult, but considering how so many billions of people don't use languages based on the Roman alphabet, allowing them to write code in as much of their language as possible is a good thing. I can also imagine mathematicians liking $π instead of $pi, $θ instead of $theta, and so on, but the world population of mathematician programmers is smaller than the world population of non-western-european-language-speaking programmers, to be sure.

ETA: It is being fixed.

I don't really know how to make Perl code Unicode-safe, but I might pull down the source to PPI::Token::Word and figure it out.

2014/02/22

You Don't Want Another Computer! On Dash-Top PCs, Set-Top PCs and Pocket Supercomputers

A decade ago, I started to get obsessed with the idea of hooking a computer to my car. I thought about storing and displaying diagnostic and status information, about storing music and podcasts, about having it handle navigation and all sorts of stuff. I hit the point where I considered the trade-off between storage media — solid state would be more survivable, but I could get literally hundreds of much bigger spinning hard drives for the cost of an SSD — and decided to keep it a mental exercise.

I'm certainly not the only one who considered putting computers in cars. And, eventually it became easy, because the iPhone came around, phones all had data and GPS, so the media and navigation parts of the equation were solved, and with Bluetooth and even 1/8" jacks, cars became stereos with four wheels. The only parts remaining are diagnostics and auto behavior modification. You can get the Garmin ecoRoute for $100 or a Bluetooth-talking ELM327 OBDII dongle for much less, plus many free apps for your phone. To my knowledge, chipping isn't dynamic yet — you can't remap your engine control unit's behavior on-the-fly — but I'm sure it's coming. Andy Greenberg wrote in Forbes about all the things two security researchers could do to pwn a Toyota Prius, and they were looking at capabilities, not attack vectors.

Point I'm trying to make is, for all my dreaming of putting a car into my computer, I now have one every time I sit down in my car, and even better, it comes back out every time I get out, so I don't have to worry about it when I go to the office, and it isn't locked in there if I have an accident. Things like Ford's Sync work best when they realize they're just an interface between your phone and your car.

(I now have an OBDII-to-USB cable and a Raspberry Pi, so I will have a computer in my car, but I'm going to explore the API and try to do more of the controlling-the-car things than a phone app would do. Certainly I'll still listen to podcasts while driving with my phone, not my Pi. I'll be sure to blog about it later.)

But I am not here to talk about car PCs.

I'm here to talk about home theater PCs.

I have one. It's an old HP that was handed down to me because it had connection issues I have never been able to replicate. The CMOS battery had died and corroded, too. Now it runs Windows 8.1 — will the indignities never end? — and I have it in my bedroom connected to the VGA port of my TV. I don't often use it, because when I start throwing pixels through the video card, the fan starts up and I have to turn it up to hear anything. I use it when I want to play with Windows 8, so not a lot.

When I want to consume media these days, I pull out my phone or tablet and point it at the Chromecast. More and more apps are coming — Google's opened the ChromeCast API — so the sweet spot between "essentially a wireless HDMI cable" and "smart thing that holds on to your credentials and streams you stuff" is being filled with more and more things.

I have a BluRay player. It comes with a wired ethernet jack in the back, and I have on occasion had it set up so I could use the networking portions, but when you're searching for YouTube videos or trying to type in passwords, the remote is a poor interface. Updates to YouTube's TV interface and others now bring up a URL you browse to that sets up the connection between your box and your account, but that's an update nobody ever pushed to my BluRay player; basically, once they make the next device, they don't care about the old one, so nothing cool is ever going to come back to my several-year-old player. This is what I fear is the fate of proprietary smart TV interfaces: the next cool thing in streaming apps will come next year, while I'll hold on to any TV I buy for several years, which means I'd have an ugly interface I hate for a decade after the company doesn't care.

We just got a Roku; I haven't even used it yet, and have only touched the remote once. It makes sense for the living room TV, which is old enough that it doesn't have an HDMI port. There is a Roku app for Android, so I'm sure I'll use it in a similar way to how I use the ChromeCast.

I saw a presentation recently that showed the older Google TV interface next to the Apple TV remote, arguing that every button in the full-keyboard-rethought-as-an-XBox-controller is a delayed design decision, and that the sparseness of the Apple TV is a selling point; I haven't used the Apple ecosystem, and I understand it's really slick, but trying to handle deep, full and varied content through such a limited interface. (Sophie Wong has a fuller discussion with photos of both remotes in a blog post.) The Roku's remote adds to that by having a hardware button to go directly to Blockbuster's app; Blockbuster is defunct and now there's a button always pointing to a dead thing. Software remotes like the Roku app should be easily updated to the sweet spot of usability.

When some people I know have tried the ChromeCast, they've objected, because they want a computer they control, not a second screen for their phone. I recognize that thinking, because really, that's what I was looking for in a car PC, before I realized that it isn't what I really want. Nothing built-in, as little smarts as I need to talk to it, and as much controllable from my phone as possible.

Do you have an argument for set-top boxes I haven't talked through? The only one I can think of is that content providers sometimes allow their content to be viewed in one forum and not others; the video to the left of Marina Shifrin telling her boss she quits through the art of interpretative dance is one of many that is cleared to be shown through TV only. As the use of phones for connectivity dwarfs the use of desktops and laptops, I'm sure that'll fall. If there's another argument, I'm game to engage it.

2014/02/21

Using Fitbit to keep me moving

I have a FitBit Ultra (which I have discussed before) that I use to keep track of my daily activity, trying to force myself into higher and higher numbers, toward the AHA-suggested 10,000 steps per day.

In all honesty, I have dropped my personal goal from 10,000 steps to 6000, because 6000 is an amount that I can adjust my day to get, while I have to spend hours of my evening walking around to get myself up to 10,000. With 6000, I have a lower barrier, so I don't beat myself up with disappointment as easily. I still do, because I still have <3000-step days, but with 6000, I feel I have a fighting chance.

I have a FitBit but I don't pay for the upgraded API, but I can check every hour. And now I do. 

#!/usr/bin/env python

from oauth import oauth
import datetime 
import httplib
import os
import pymongo
import simplejson as json
import time

import fitbit
import locked
import pushover as p

def main():
    checker = locked.locked()   # determines if screen is locked
    if not checker.is_locked():
        from pymongo import MongoClient # connect to mongodb
        client = MongoClient()          # connect to mongodb
        db = client.fitbit              # connect to fitbit DB 
        reads = db.daily_reads          # connect to collection 
        now = datetime.datetime.now()       # what time is now?
        datestr = now.strftime("%Y-%m-%d")  # now in date format
        isostr  = now.isoformat()           # now in ISO format
        fb = fitbit.make_class()                    # connect 
        fbr = fb.activities_date_json( datestr )    # get updates
        summary         = fbr['summary']            # get summary
        steps  = summary['steps']                   # pull out steps 
        new_read = {}               # create new read
        new_read["date"] = datestr     # date == datestr
        new_read["time"] = isostr      # time == ISO
        new_read["steps"] = steps      # steps is steps to date
        
        if 0 == reads.find( { "date" : datestr } ).count(): # if nada 
            for read in reads.find():       # clear out 
                id = read["_id"]
                reads.remove( { "_id" : id } )
            reads.insert( new_read )        # insert 
        else:
            old_read = reads.find_one()
            delta_steps = new_read["steps"] - old_read["steps"]
            if delta_steps < 100:
                msg = "You took " + str(delta_steps) 
                msg = msg + " steps in the last hour. "
                msg = msg + "Go take a walk."
                pu = p.pushover()
                pu.send_message( msg )
            id = old_read["_id"]
            reads.update( { '_id' : id } , new_read )

if __name__ == '__main__':
    main()

There are three modules that are mine: the fitbit module which comes from my fitbit_tools repository; locked module which uses xscreensaver-command to determine if the screen is locked (if I'm not at my desk, my step counter isn't being updates; and pushover, which uses the Pushover service to keep me informed. I could and maybe should have it do speech with Festival or pop to my screens with Notify or Snarl, which also are desktop-centered tools, but this is what I have.

Similarly, perhaps this task is better suited to a key-value store like Redis than MongoDB's document store. At most I'm having one entry per day and the first entry of a day clears out all previous entries.

But, notwithstanding all the perhaps-suboptimal design decisions, this is an example of how coding and quantifying yourself can help you improve your life.

2014/02/15

If ( starting_over() ) { start( this ) } # 'this' undefined

I listened to FLOSS Weekly's podcast on nginx today, and it hit a cap on something that I've been considering for a while. Basically, I have worked in a few technology stacks over the years, but if I was to say what I've spent most of my development time with, I'd have to say Linux machines, Apache web servers, MySQL databases and programs written in Perl. 

This is not exclusive: For a while, I did work with Windows on whatever their default server was, doing Active Server Pages with VBScript connecting to an Oracle DB, while being admin for a VMS system. I've written Python, Bash, R and JavaScript. I've tied hashes to Berkeley DB and used flat-file databases before I learned the next steps. Not that I ever got it into production, but I've written Visual Basic for Embedded to build an app on an iPaq in the days before .NET. And I'm sure there's more that I've forgotten. But the things I write for myself tend to be hosted on Linux, distributed over the web, served by Apache, storing to MySQL and generally written in Perl.

We'll call that my comfort zone.

The comfort zone is a good place. You have a job that works in the comfort zone, you know where to start. You likely have a "framework" in place that you can adapt to what the new thing is. The world can turn quite a lot while you sit in your comfort zone.

Aside: I generally don't use frameworks, because I know my comfort zone well enough that I can make separate things fast enough by adapting existing things that I'd rather do that than spend time trying to learn a framework. With Catalyst, I timed out twice before getting to the point I could do something with it. I'm better with Dancer, which is closer to the level of web programming I'm comfortable with, but the individual-pieces-of-CGI I work with doesn't lend itself to the web-applications world. Our lab still does puppies, not cattle. Consider that part of the comfort zone.

The first thing I missed was Python and Django. I was the last remaining Perl guy when most everyone I knew went to Python in the late 1990s and early 2000s. Then there was Ruby and Rails. I could probably get to iterative and recursive Fibonacci on both without looking too much at the documentation, but I've not spent quality time with either. 

There are two problems with staying in a comfort zone. First is, the world goes on, and eventually nobody values your comfort zone. There are still places where they want Perl people, but other languages are higher in the desired skills list. The other is that it's a good way to burn out. Everything looks like X and you get tired of looking at X

I think the hot spot for web development, the place I'd be if I was starting out right now, is still Linux, still Web, but nginx, MongoDB and Node.js. I like what I've seen of Redis and have used it in a personal project (and really, I've spent enough time with it to REALLY like MySQL; I still think in tables), but I think that MongoDB maps well with JSON (which works well with Perl hashrefs, which is the source of my love for it all, I won't lie) and is my fave so far of all the NoSQL databases I've played with so far.

So, I'm curious: if you were starting over today, knowing the computing environment and your own preferences, what would you start with? Why? Is this stuff you use right now?

2014/02/09

Question about New School Web Authentication and Images

Time was, you authenticated via htaccess, and the server handled the details for you. If you gave the right password, you could access the materials in a directory, and if you didn't, you couldn't.

Now, we're authenticating with OAuth with Google or the like, and the smart stuff you write knows whether you're authenticated or not, but that kinda happens at the client level, and your server doesn't know whether it's authenticated or not.

xkcd
If you're talking to the database to get content, there has to be some code there that says "OK, there's authentication, so we know Alice's content from Bob's, but if there's something saved to file, like an image — You could put your images in a database table, and I've done it, but in practice it's kinda stupid — that means that image is hardcoded, so anyone could browse around your authentication.

So, is the solution to have the images in a non-web-shared directory and have code determine whether it's OK to send the image or not? I've done that, too, but it seems like you stop the server from doing what it's good at. As efficient as you can make a program that takes in a user identifier, determines if it's acceptable, reads in a file and sends either a good image or a "that's not okay" image, that's always going to be slower than letting your server just send that image.

So, do I own that slowness? Is there another way that I just don't know yet? I'd put this on Stack Overflow if I even knew how to pose the question. Any thoughts?

2014/01/31

Trying just MongoDB, and it works!

And it works!

#!/usr/bin/perl

use feature qw{ say state } ;
use strict ;
use warnings ;
use Data::Dumper ;
use MongoDBx::Class ;
use MongoDB ;

my @movies ;
push @movies,
    {
    title           => 'Blade Runner',
    rating          => 'R',
    releaseYear     => '1982',
    hasCreditCookie => 1
    } ;
push @movies,
    {
    title           => 'Thor',
    rating          => 'PG-13',
    releaseYear     => '2011',
    hasCreditCookie => 1
    } ;

{
    my $client     = MongoDB::MongoClient->new( host => 'localhost', port => 27017 ) ;
    my $database   = $client->get_database( 'test' ) ;
    my $collection = $database->get_collection( 'movies' ) ;
    my $movies     = $collection->find( ) ; # READ
    while ( my $movie = $movies->next ) { # would prefer for ( @$movies ) {} but oh well
        my $title = $movie->{ title } ? $movie->{ title } : 'none' ;
        my $releaseYear = $movie->{ releaseYear } ? $movie->{ releaseYear } : 'none' ;
        my $count = $movie->{ count } ? $movie->{ count } : 0 ;
        say qq{$title ($releaseYear) - $count } ;
        my $id = $movie->{ _id } ; # every mongodb record gets an _id
        $collection->remove( { _id => $id } ) if $title eq 'none' ; # DELETE
        $collection->update(
            { title => $title },
            { '$set' => { count => 1 + $count } }
            ) ; # UPDATE
        }
    #LADIES AND GENTLEMEN, WE HAVE CRUD!!!
    }
exit ;
{
    my $client     = MongoDB::MongoClient->new( host => 'localhost', port => 27017 ) ;
    my $database   = $client->get_database( 'test' ) ;
    my $collection = $database->get_collection( 'movies' ) ;
    my $data       = $collection->find( ) ;
    for my $movie ( @movies ) {
        say qq{$movie->{ title } ($movie->{ releaseYear })} ;
        $collection->insert( $movie ) ; # CREATE
        }
    }

The lesson might be Don't Trust ORMs, Do It Yourself, but I hope that there's some reason to use these things.

You Had One Job, MongoDBx::Class Documentation

So, after yesterday, I started back with the command line interface to MongoDB. I found I can do what I want with it, which of course is creating, retrieving, updating and deleting records. So, I try working with Perl, my go-to language, and I install MongoDBx::Class. The following code, especially the $db-&gtlinsert() is closely adapted from the CPAN page.


#!/usr/bin/perl

use feature qw{ say state } ;
use strict ;
use warnings ;
use MongoDBx::Class ;

my $dbx = MongoDBx::Class->new( namespace => 'MyApp::Model::DB' ) ;
my $conn = $dbx->connect(host => 'localhost', port => 27017 , safe => 1 );
my $db = $conn->get_database( 'test' ) ;

my @movies ;
push @movies , {
    title => 'Blade Runner' ,
    rating => 'R' ,
    releaseYear => '1982' ,
    hasCreditCookie => 1
    } ;
push @movies , {
    title => 'Thor' ,
    rating => 'PG-13' ,
    releaseYear => '2011' ,
    hasCreditCookie => 1
    } ;

for my $movie ( @movies ) {
    say qq{$movie->{ title } ($movie->{ releaseYear })};
    $db->insert($movie) ; # How the documentation says it should go
    }