The Frustration of Dealing with Users

This woman -- in honor of anonymity and cryptographic protocols, we'll call her Alice -- working on a startup has a problem. She has changes she wants to make in her website and she has no developer. She had contracted with a international developer, but that had fallen through.

I'm a developer, and I'm looking to expand beyond my primary work, so I offered to help.

Turns out, it's in Django, the Python web framework, although she wants to move it to something else. I'm not a Python guy and I'm not a CMS guy, but I'm willing to give it a shot.

Soon after she gives me access to her GitHub repo, she tells me that the international developer had taken care of the first thing, changing the logo, but those changes were a branch on someone's fork of her code. 

"OK," I thought. "This one's easy. First, she has to have the branch merged into the third-party's code. Then, that person needs to set her code as a pull request for Alice's code, and Alice then needs to merge that pull request. Last, she needs to ssh into her server, use git to fetch the changes and restart." The common thing with all of this? I can do none of it. I have no access to Alice's account. I have no access to the other user's account or repo. I have no access to the server. 

So, I draft an email, explaining the situation and the steps needed to get that code into position. Granted, on my end, I have allowed myself to become overburdened by time constraints. This becomes an issue with the relationship between Alice and myself, and and I own it. My primary goal for this is to justify the addition of the word "Django" to my resume, and anything else was gravy. But I was sure that a set of changes were coming, and it seemed a poor idea to work on the code until at least the changes were merged into Alice's main.

But, after some pushing (and repeated git pulls showed no change in master), I decided to try to get things running. I run a Windows laptop, because it's more about entertainment and ssh than real development, so I try to get Vagrant going and find that VirtualBox is corrupt. This takes some time to handle, time that I can't really call Alice's time, because it isn't her fault my laptop sucks.

And, as a personal failing, if I have nothing good to report -- I'm hitting issues and not getting it, or there are days after days where I'm unable to throw time or mental energy on a project -- I feel really bad about communicating no-good-news, so I go radio silence. Again, my bad. It's an issue I need to work on, and I own it and am trying to work through it. I don't want to tar others with a brush and claim I'm spotless.

Eventually, I get VirtualBox and Vagrant running, install Django and get the repo down, and I go to the "run this" file. I try to run it and it errors because a config file is missing.

So, at this point, this pseudocode is a simplification of my position.
# purpose of this program is to add two numbers

my $x = read_X_from_config_file() ;
my $y = read_Y_from_database() ;
my $output = handle_addition( $x , $y ) ;

sub handle_addition {
    # lots of complexity here.

I'm expected to work on handle_addition(), but I have no idea what $x looks like because I don't have the config file, I have no idea what $y might be because I am denied access to the database and schema, and I really have no idea what changes might occur for handle_addition(). So, I explain my position, saying I'm in position where I'm technically able to proceed, but to really do anything, I need things from her.

The response is, since you don't have much time to contribute, I think it's best if I find someone else to participate in this project.

As I have said, it is not unreasonable for her to drop me from this, but I think she is really missing the boat as to what the issues are with her project. At no point was there any indication that there was any attempt to integrate the other developer's code, despite my suggestions, and at no point was I requested to ignore those coming changes and proceed. I say "You need to do this and tell me when you're done so I can go", and she acts like she's entirely waiting on me.

And, as this was one more side project than I can really spend time on, I don't feel I can, nor do I really want to, really get her to understand the situation.

So, I blog it. 


Leveraging the Forecast.io API

A while ago, I got an API key for Forecast.io, with the intention of writing something that takes a look at the day to say whether I could bike to work without getting weathered on. I have not written that, and I certainly haven't started biking to work on a regular basis.

But now, I have written something that uses that data and presents it via notify-send and Pushover. I use notify-send a lot, but as the sources change, I have to change my scripts.

I've tried to cut things down to just the things that are necessary, but I could do more for that. Carp, for example, is Perl's way of sending errors to STDERR and exiting and such, but I really don't use it. There is a verbose flag that uses Data::Dumper. I'm turning epoch timestamps to times and dates, so I need DateTime (Thank you Dave Rolsky). The API uses JSON to send the weather data, and I use YAML to hold configuration data, so those modules are given. I want to get something from the web, so I use LWP.

So, download, use, adapt, learn, and stay dry!


I get why I was wrong. Re: Database WTF

In a line, date_time NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP. Specifically, ON UPDATE CURRENT_TIMESTAMP. I did not want that. I wanted to default to current timestamp, but I didn't want it to change. Well, in one case, I think. Not here.


A Database WTF

Database table exists, and has three columns worth mentioning:

  • run_id - the key to the table
  • date_time - timestamp created on row creation
  • ack_error - a boolean set to determine if errors are silenced.
This is my test code. It includes some functions I import.
  • db_do - Given SQL statements, does them. Used for INSERT, UPDATE, REPLACE, DELETE.
  • db_arrayref - Given SQL statements, does then and returns the result. Used for SELECT.
  • confirm_error_run - Given a run_id, gives current state of ack_error.
  • ack_run_error - Given a run_id, sets ack_error to true.
  • clear_run_error - Given a run_id, sets ack_error to false. 
    sub check_date {
        my $run_id     = shift ;
        my $check_date = q{
        SELECT date_time date
        FROM sequence_run
        WHERE run_id = ?
        } ;
        my $date = db_arrayref( $check_date, $run_id ) ;
        return $date->[ 0 ]->[ 0 ] ;

    my $SQL = 'UPDATE sequence_run SET ack_error = ? where run_id = ? ' ;

    $DB::Database = 'genomicsdb' ;
    my $run_id = 318 ;
    my $date ;
    my $error ;

    say $run_id ;

    $date = check_date( $run_id ) ;  say $date ;
    $error = confirm_error_run( $run_id ) ; say $error ;

    my $do = db_do( $SQL , '1' , $run_id ) ;
    say $do ;

    say '-' x 40 ;
    $date = check_date( $run_id ) ;  say $date ;
    $error  = confirm_error_run( $run_id ) ; say $error ;

    ack_run_error( $run_id ) ;

    say '-' x 40 ;
    $date = check_date( $run_id ) ;  say $date ;
    $error  = confirm_error_run( $run_id ) ; say $error ;

    clear_run_error( $run_id ) ;

    say '-' x 40 ;
    $date = check_date( $run_id ) ;  say $date ;
    $error  = confirm_error_run( $run_id ) ; say $error ;

Changes to ack_error should be immaterial to date_time, right? Yet...

djacoby@genomics 12:00:44 ~ $ ./test.pl 
2014-07-31 14:36:43
2014-07-31 14:36:43
2014-07-31 14:36:43
2014-08-04 12:22:09

I just don't get it. ack_run_error() and clear_run_error() are essentially like the db_do( $SQL , ... ), and it's somewhat nondeterministic whether my db_do() and ack_run_error() reset the time. Confusing.


2014 Body Goals - Status Report

Just because I haven't been writing about my goals doesn't mean I've forgotten them. In fact, a week ago, I participated in Zoo Run Run, a 5K around Columbian Park to benefit the Columbian Park Zoo, and my time was 34:12, which put me well past my goal for Endurance. My next goal is to come in less than 30 minutes, but I think I'll leave that for next year, rather than sign up for more races this year.

I have started working with the cable setup at the Co-Rec to move toward my goal of one pull-up. I was able to pull about 60 lbs, which is good. I don't really know the next step, but I'm happy with that and will work on a plan for pull-ups.

For the longest time, my personal record on the FitBit was the Independence Day Celebration where I was a volunteer and on my feet all day, exhausting myself but hitting 18K or so steps. Related but separate, I had identified when running that my legs weren't tired, my feet weren't bad, but I was huffing and puffing like a big bad wolf. I've since started to focus on breathing, and I put in a 20K step day on June 20th with the main difference being deep diaphramatic breathing. I haven't hit that PR since, but I have been more consistently hitting the 10K goal. I'm still not hitting it every day -- I wimp out on weekends -- but I've hit 10K at least one day a week every week except one I took as a vacation week. 

I've been trying to put HIIT into my runs. Dale Asberry suggested it in my goal-planning post and it is valid. I guess I see it as a level thing, in that if a person can't jog for 10 minutes, that person would be unwilling and unable to alternate sprints and rests for 4 minutes. I, as my okay-ish 5K time shows, am at a point where I should start doing something like Tabata sprints, which I have tried to start. It went well a few times before the 5K, but since, I haven't been able to really engage. Well, last time, I walked and breathed through 2 of 8 20-second sprint times, which 1) is more than I had the time before, and 2) is less than I wanted. 

For weight, I hit 202 lbs recently, but was 206 when I weighed this morning. I'm a little discouraged by the recent jump up, and I think this means I have other habits to engage, but I'm liking this neighborhood and feeling good about it.

I plan to generate a plan to engage my remaining fitness goals 


Name Things Like You, or "Do you ever feel like an idiot?"

I was preparing a talk on developing against the FitBit API yesterday, which lead me to look at the FitBit website, and I found something called "Alertness Checker", which I didn't recognize. The others, like TicTrac and FaceBook and IFTTT, I did, but not "Alertness Checker".

So I revoked access.

And I found that "Alertness Tracker" is the name I put for my "app", by which I mean the connected series of programs I collect as fitbit_tools, which is the topic of the talk I'll be giving in a week and a half.

Of course, I felt like an idiot, once I realized the FitBit tools I had built my life around had stopped working. The key one is called fitbit.today.py, and it checks once an hour to see if I've made more than 100 steps in that hour, and chides me if I haven't. It's lots of small things, but when you realize you broke lots of small things, you feel like an idiot.

The first thing is, anybody could name a thing "Alertness Tracker". The specific thing that caused me to develop fitbit_tools is a point where I wore my FitBit Ultra every day for several weeks without noticing I hadn't charged it recently and it was a dead dead thing. Once I had enough to tell me if it hasn't synced in a few days, I knew enough to have it tell me if the battery level was anything but high. I wrote it in Python because I couldn't find any code (besides the Net::Twitter module) that really handled oAuth in Perl, but I did find the code in Python. After that, I decided that I'd like to know more about my trends than the FitBit dashboard can tell me, then put it out on GitHub, but since I had the App registered with FitBit, I stayed with it.

This is the main thing I want to hit. The secondary point is "Don't wallop things without checking", but the main point is that you have a style, and you should go with that style. Stay within the conventions of your project and team, of course, but you should be able to recognize yours as yours. In the lab, Rick and I are both Perl programmers, but our styles are very different, so we can open up libraries and look at subroutines and say "Hey, I don't know when I wrote that, or why I wrote that, or what stupid thing I was thinking when I wrote that, but I certainly know that I wrote that."

I recall, I faked the registration using FitBit's tools rather than writing a "get client tokens" oAuth script. I think I put that in the "I really should do that" category and forgot it there. Now, as I'm going to talk through it, and as I just re-experienced the issue, I have explained it in the README and have received a "round tuit", and so will have to write something up.


Question about versioning with Perl modules

My main two languages are Perl and Javascript, and I've been coding them here for several years. This means that I have a body of code I'm perpetually updating and maintaining old code. And, while I am shamed to admit this, I test in production, I dev in production, and until we change our infrastructure, I don't see that changing any time soon.

In Javascript, there is a capability that I've found I like.

<script src="/lib/bulldada-0.0.1.js">

This means if I want to play around with rewriting bulldata, I can copy and poke at bulldada-0.0.2.js, and change the script tag to call for 0.0.2 when I'm happy with it. This means I am developing with the production server, but not necessarily with production tools. This makes me happy.

Let's think about Perl. In my programs, I can specify a version:

use BullDada 0.01 ;

In my modules, I can set that version:

package BullDada ;
our $VERSION = 0.01 ;

Question is, how to have multiple modules with the same name in the same directory structure? By testing with Perl 5.18, I know that you can't have BullDada_0.1.pm and BullDada_0.2.pm, both starting with package Bulldada, and expect it to work. It seems that if I wanted to keep programs from using BullDada.pm with $VERSION = 0.01, I could specify BullDada 0.02, but I could also delete the old version and never use versioning at all.

Unless I'm doing it wrong. I'm willing to grant I'm doing it wrong. If there is a way to do it right, what is it?


Exercise Before Work

Not meaning as a rule. "Exercise takes priority over coding." My goodness that is so not me.

Rather, today, I went to the CoRec and ran this morning, showered and went to work. Normally, I lay in bed watching YouTube videos for the extra hour. I hate to engage the day rushed, having done more than enough waking up barely in time to prepare myself to get there on time. 

As I think I have mentioned, I use jerry-rigged sunrise alarm clocks (one X10 and controlled from Linux via crontab and one a simple vacation timer) to turn lights on in my bedroom, which are set to go off at 5:30 am, and I'm usually awake between 5:50 and 6:20am. This morning, I think I hit close the later limit, watched a video or two, got dressed, and took the bus to campus, then kitted up to run at about 8am. I got an hour in on the upper track, taking a warm-p lap or two, doing four minutes of Tabata Sprints (20 seconds running as fast as I can, 10 seconds slower rest which I did as walking, for 4 minutes) then filling out the hour with laps. I got to 7343 steps before deciding it was time to clean up and get to work.

Exactly 7343 steps, as my FitBit attests

Walking to the office got me to ~9300 (photo not available) and I'm sure that the rest of my day will get me to my FitBit- and AHA-recommended 10,000 steps. I'm thinking I may do pre- and post-work runs on days when my evenings are not already scheduled. I'm liking this.


The Graphics Interchange Format in Conversation

  • For me, it's a solved issue. When the inventor of a thing says it's pronounced a way, I give that weight.


  • For me, it's a shibboleth. I have been involved in web development since 1996, and if you pronounce it "Gif" instead of "Jif", that's a sure sign that you're a noob. There's nothing wrong with being a noob, but there's certainly nothing right.
  • For me, it's also not a big thing. If you pronounce it the "wrong" way in a conversation with me, I won't correct you or start an argument about it. Starting an argument over things that don't matter, is dickery.

    Continuing one
    , on the other hand, isn't so bad.
  • The "It's not gramatical" argument cuts no ice with me. Saying the G has to be a hard G will always have me wondering how you'd pronounce "a German Giraffe", and while you say "But it comes from graphical so it must be a hard G", I flash to Police Academy 4: Citizens On Patrol, notice that "Citizens On Patrol" acronyms to COPs, and wonder if you would have me pronounce it "sops".
  • Yeah, I know going back to a Police Lobotomy movie as an argument is not too strong, but my mind has better things to do than develop and parse through a list of acronyms. Believe me, I've tried, and my mind has said "Let's watch YouTube videos. We could browse the web for pictures of smiling dogs. We could consider why your SQL tables are stupid. We could think about Steve Gutenberg movies. Really, anything but the acronym list thing." 


About Me

I've been working on my resume recently. No reason, really. (If you have questions, ask elsewhere. I'm fairly easy to find.) A recent response to my resume tells me it sucks at telling the story of me. Not too surprised; the form sucked before, and has become the gameboard for competative Reverse Buzzword Bingo, where if you find the right combination, you get the job, which makes it counterproductive to even try to tell your narrative.

So, this is the story of me:

My first degree was a BS in Journalism and Mass Communication. My greatest experience was as copy editor of my student paper, where I did full-page layout of the State-and-Nation wire news section to our paper's style standards, collected the weather forecast (which I cribbed from the regional newspaper's front page by calling a friend who worked in a gas station) and read every dang word in the paper. I got this position by copy-editing a copy of one week's front page, where I not only found the errors they introduced to test me, but also errors that made it into print that week. The biggest lessons I learned were that I was really good at finding miniscule errors in large blocks of text, I was pretty good and making an attractive and readable layout if I knew what sort of look we were shooting for, and if you're really physically and mentally exhausted with more work left to do, fruit juice is better than soda.

Also, I learned that when people say "If everyone from Dan Rather to the guy who writes obit for the local fish-wrap were to die tomorrow, there would not be enough positions opened up to employ all the people who graduate with journalism degrees this year", that's not just a punchline, that's a suggestion that you change majors as soon as this conversation ends.

After not finding work in journalism, I went back to school, where I received a BS in Computer Science, and worked for the university as a web developer. As with xkcd, my time with Perl has been much more useful in my post-collegiate career than the courses I took.

Also, the things I learned as a j-school student -- finding small errors in large text blocks, making text blocks readable and attractive, and handling exhaustion -- are incredibly useful for CS students and web geeks, too.

During that time, I found that what I loved to do was converting things to other things. If you sent this email address a message with this string in the subject and a URL in the body, it responded with that URL's web page as the body. If you went to a certain web page, you could send an anonymous email. I fell in love early with RSS and wrote many things that read it and many other things that wrote it. I wrote a thing for my home page that displayed one guitar if I had new mail waiting, and another guitar if I didn't. I once helped write a tool that scrolled RSS headlines across the top of a web page. My goal and hope was to be a developer who turned things into other things, preferably with web technologies on a Unix/Linux platform while contributing to Open Source projects, in an organization whose primary work is creating software.

A thing I didn't really learn is that, in the last year of your college career, your primary goal is finding something to do with your life after graduation. Preferably, something where you are paid. Successfully completing your courses and graduating is necessary but secondary. So, I found myself as a VMS admin and occasional Microsoft-focused web developer for a medical clinic. I often made things out of other things: I recall parsing the four-digit long-distance records by long-distance code and sending a list of calls to the user of record for each code. It was okay, but not in line with my desired workplace as above.

After that, I came to work at the university again, for a research lab. We do gene sequencing; I do full-stack web development. I did some of the stuff I like to do: I turned web forms into database updates. I turned database tables into web tables. I turned CSV into plots with R. I used Perl and Linux and Unix and eventually R. I coded  daily.

For a while, I worked through a temp service with a defense contractor. Mostly I maintained a lab. On occasion, I set up specialty PCs. I used a visual pre-processor that eventually made C code, which wasn't fun. The pay was somewhat better, but of the things in the goals-and-hopes bit above, I did nothing.

So, I came back to the lab again. I gained more responsibility, and continue to make things from other things. I turn database dumps into XML-filled ZIP files that our instruments use to handle the sequences we give them. I used Sikuli and Jython to script a Windows application to turn a bunch of XML files that held UUencoded BLOBs into searchable XML files. I turned database dumps into JSON, and turned JSON into web page tables and Javascript-created plots. I write libraries that make it easy turn things on various machines into messages on my Linux box, on my Windows box, on my phone.

My one-true-language is Perl. Most everything I write, I first try in Perl, and only go away from it if I can't make it work there. I code in JavaScript: Perl doesn't run on the web browser, so I have to. I'm trying Node.js, and am slowly finding my way toward usefulness. I'm writing more and more R, because it's the shortcut for making plots.

I dislike Python. On first hearing about it, I thought, "What's the point? It sits in the same niche as Perl, but isn't Perl. Why use it?" My first experience with it was some code which displayed available machines in an on-campus computer lab. It has been handed to me as "this works", but it was indented with tabs, and somewhere along the line, a space had been inserted before a tab, so it didn't work. I get readable code, and my journalism degree forced me to appreciate whitespace for readability, but here something I couldn't see ruined my day. My Sikuli project was written with Jython, the Java implementation of Python, because that's the only choice, and my FitBit Tools are in Python because I couldn't easily get OAuth working with Perl. I could see using it more, but I write Perl in a Perl shop right now.

The Greyjoys of the Iron Islands Do Not Sow; I Do Not Make Web Pages. I make tools. I make things that turn things into other things. This can be web, but it will be dynamic, either at the server side or the client side. If you want a web page -- something static, a billboard for the web -- you will be happier and I will be happier if someone not me made this page. For design, I can get you 50-75% of the way to an attractive and useful web tool, and probably more if given a standard to align to, but design is not where my strengths lie and are not where my interests lie.

This is me. This is the tools I like to use, the tasks I like to complete, the environment I like to be in. Right now, I am a developer, working with web technologies in a Unix/Linux environment, doing more but not enough to contribute to Open Source projects. I'm part of an organization that's (writ large) education and/or (smaller) assisting biological research. This means I'm not quite where I want to be, but better than I was for many years. So, I'm looking, but not looking too hard.