In Praise of Night-Hacking, or at least in hopes to understand

Photo credit: pantulis 

I code every day. It's what I do, and I'm thankful. But there is a problem with coding from 9am to 5pm.

Other people.

Yeah, you're coding for their benefit, and it is well and good and helpful (and secretly, an ego boost at times when you otherwise feel humbled by the tasks in front of you) to serve as tech support for other people, even when and sometimes especially when the task seems otherwise mundane to you.

But, when you have all that state in your head, the last thing you want to do is fix someone else's thing. It has had me nearly to tears before. When there's nobody else around, you don't have to worry about the house of cards you have in your head. That's when you can relax and get into it, man. You know, like a coding machine?

But that's not it. Or at least not all of it.

At least what a geek with a hat has to say. In essence, when you're tired, parts of your brain shut down, and those can be the "squirrel!" aspects that get in the way of the deep concentration you need to build, understand and modify the house of data cards in your head. And, with your focus and the lights of the computer screen, you can keep going until you drop off, and when you do, it's the deep, relaxing sleep that exhaustion brings, not the light, brittle sleep sometimes broken by insomnia you get when you're sleeping because it's time. (I once heard a professor opine that humans were really built for 25-hour days. I do believe that some.)

A big problem is, of course, that if your life doesn't live on hacker's hours, parts of your life fall away when you start living on hackers hours. Something to sleep on, so to speak.

Can You Identify The Bug?

#!/usr/bin/perl
# 2g_run_reversal.pl

use 5.010 ;
use strict ;
use warnings ;
use Carp ;
use Data::Dumper ;
use Getopt::Long ;
use Pod::Usage ;

use lib '/home/ltl/lib' ;
use SecGenTools::Run 'get_run_data' ;
use SecGenTools::Accession 'get_accession_data' ;
use SecGenTools::Request 'get_request_data' ;

my $config      = config() ;
my $run_stub    = get_run_data( $config->{ run_id } ) ;
my $run         = $run_stub->{ $config->{ run_id } } ;
my $run_samples = get_run_sample( $config->{ run_id } ) ;
my %conversion =
    map { $_, $run->{ max_regions } + 1 - $_ } 1 .. $run->{ max_regions } ;

for my $region ( 1 .. $run->{ max_regions } ) {
   my $conversion = $conversion{ $region } ;
   for my $sample_id ( sort keys %$run_samples ) {
       my $sample = $run_samples->{ $sample_id } ;
       next if $sample->{ region } != $region ;
       $sample->{ region } = $conversion ;
       update_run_sample( $sample ) ;
       say join "\n\t" , $sample_id ,
           $sample->{ accession_id } ,
           $region ,
           $conversion ,
           ;
       }
   }

exit 1 ;

This is the core of the code. config() puts all the configuration options into a hash and returns it. get_run_data(), get_request_data(), andget_accession_data(), are wrappers around database calls.

An accession is an individual sample to be analyzed. A request is one or more accessions. A run is one or more requests, assigning the accessions to one of the regions. And, because this is a mapping between a data structure and a physical object, it is possible for the user to start at the wrong end. This program is supposed to take all the requests and reverse the regions. That is, for an 8-region run, all the accessions in region 1 should be in region 8 and vice versa.

This code puts everything in regions 1-4. It took me a morning to figure out why, although part of that was restoring the database to the way it was before.

It took me a while to get it, so stop here if you're still looking.

In a word, persistence.


$sample is a reference to the data in $run_samples. In my mind, my $sample = $run_samples->{ $sample_id } made a local copy, but it didn't. That meant that, when the code hits region 5, it works on the accessions just moved over from region 4, too, and so on.

And, in working around that, I decided that looping on region was useless except to massage the retentive control freak in me, and once I convinced myself of that, things got better. So, like the Monster at the End of the Book, the bug is me.

Lots of Small Rants

I got three things on Black Friday, and the first one shouldn't even count. It's the e-book version of Programming Android from O'Reilly. Time was, when I had a technical question, O'Reilly books were the first place I'd look, but now, my first checks are Google and StackOverflow, so this is a bit of a nostalgia move, but    there's enough of a blank spot in my mind to justify it. Now, just to spend some time and generate something.

The other two are cheap pieces of USB hardware from Newegg. The first is a USB Wifi dongle (shown right), or, as it reads, "802.11b b/g/n Wireless Adatper". I have to say, I like it better so far than the Netgear that has been going back and forth between a few Windows boxes I have. Plus, I like the form. It's nice and small.

The other is also an inexpensive piece of Rosewill kit, a USB dongle connecting to a remote to Windows Media Center. Right now, it's nonsensically connected to the laptop I'm writing this on, but I could easily see myself liking this. I have a wireless Logitech keyboard with gone-AWOL mouse and don't do much with whatever PC I have in the bedroom for lack of pointer, and while I have been using Boxee for bedroom media PC duties, I could see myself accepting Windows Media Center due to this one. Problem is, the PC I needed the USB dongle for is the the same one I needed the remote for, and because it's been repurposed from HTPC to desktop, as long as I need the WiFi dongle, I do not need the remote and vice versa.

I've been doing more with git, as previously mentioned today, and I think have hit the distinction for bare repositories. This has been kicking to most of the day, and it's a bit of duh I just had to work through to get this into my head. I've been making non-bare repositories and wanting them to behave like bare repositories, then tearing my hair out. I'll have to work out aliases or scripts for this stuff.

Book Review - Version Control by Example, or "Thank you, @eric_sink"

For too long, I've worked in environments without version control. There's been backups, either real or virtual, and for the web part there's the Google cache (which saved by butt once as the webmaster for my LUG) but version control has always been something that I know I should be doing, but we've never done. (The one exception was doing temp work at the car parts company. There, we used code reviews and Synergy. Not the good Synergy, which gives you the ability use one keyboard and mouse across several computers in software, but the bad one, the ancient, slow and ponderous configuration management system, which is similar enough to version control that I cannot express the distinction.

So, here's something where I know I should be doing it, but I don't really know how to do it. I've heard enough about Git and have signed up with Github, which put it before Subversion and CVS and the rest of the choices. But, unfortunately, I hadn't heard enough about Git to really know what the heck I should be doing with it.

Then I saw a post by my internet friend Funnel Fiasco about Version Control by Example by Eric Sink, saying how good it was. And, as it turns out, Eric is much more about teaching people about how to not do stupid things than he is about sales, so you can download digital copies, browse through online, and he'll even send you a copy free.

I've had it in my bag for several months without cracking it open, which was stupid of me. Right now, I'm in a decent place at work, with little sitting with a tight deadline, so I have time to go through the book and start  actually learning from it, and wow! I'm getting it! It makes sense. The core of the book is the same examples expressed in Subversion, Mercurial, Veracity and Git, so it serves as a Rosetta Stone, too, so if you know one, you have a step toward using the others if that's what your workplace or project uses.

Thank you, Eric Sink!

Speaking of git

mount.pl, my script for automating the mounting and unmounting of remote file systems via sshfs, is now on github.

Questions about Git

I'm reading Eric Sink's Version Control By Example, starting to hit the examples, and I'm finding that to be a bit of a problem.

Eric's example is one dev in Birmingham, UK and one in Birmingham, AL, writing C code and committing to a server in Cleveland. Right now, the dev team for my office are two coders separated by all of five feet, writing Perl code for one of two servers, one being the web server. I'm easing in to git, as you might guess by the reading material, and like most working environments I've been in, our version control system has been "copy a backup before you muck with the file", which I know is dumb and useless (especially in the lab, where we rely on RAID for data protection and thus don't really have backups). I'm kinda taking the lead on this, having been burned enough to want to protect myself, but I don't really know much of it.

I've been using git so far to keep track of changes locally, just doing git init and git commit within the local file system. Is this enough? Or do we really, truly need a server?

And the code sits in /home/user/web/cgi-bin/foo/bar/blee/quuz.cgi or /home/user/bin/hoge.pl, I'm wondering: should I have those be the directories to run git in? Or should I do the work in /home/varlogrant/dev/hoge.pl/ and copy hoge.pl over to /home/usr/bin when I'm happy with it and it has been committed?

I have a bigger question on how to take a large selection of interconnected Perl modules and make them 1) test driven in the real, chromatic-approved way, 2) working with git in a useful way, and 3) usable from ~/lib on several dissimilar systems. I have a cheap hack on #3, but if I break apart and reassemble the modules, I can probably do it cleaner and smarter.

'cat' considered not useless

Consider this:

cat /path/to/my/file.log |
grep filter |
sed 'something, I don't really use sed a lot' |
grep filter_again |
lp

Many would write this off as "useless use of cat". I don't think so. I mean, functionally, sure. But it isn't all functional.

cat /path/to/my/file.log | # read data from file
grep filter |              # run a filter on file contents
sed 'something, I don't really use sed a lot' | #change file contents
grep filter_again |        # run another filter on contents
lp                         # send to printer

Compare to this:

grep filter cat |          # read data from file AND run a filter on file contents
sed 'something, I don't really use sed a lot' | #change file contents
grep filter_again |        # run another filter on contents
lp                         # send to printer

You have now overloaded grep and if you want to remove it, you now have more editing than reading a line.

Friend of the blog Patrick says that this use of cat is just as useless as comments. I tend to agree.

You Do Not Own Your Phone

That is, unless you root it. Thank you Trevor Eckhart, and no thank you Carrier IQ. And Forbes reports that this could mean millions of violations of wiretap law.