Big Three for September 3-7

Here we are, at it again. I'm writing down my Big Three tasks for the coming week. I think I'll admit the progress on the previous Big Three.

• Main Work: This one is actually pretty good. I've had some feature requests and I'll get some more, but I have it in good condition. 
• TEDx Web Development: Institutionally, we're starting to get together, which is good for the team. We're starting to recruit people to take on pieces. This is good. And I have tried to put a basic hello_world.cgi onto the Dev server, and it failed. This is ... well, at least I know. There's a code review process, so I have to start looking into the "who do I ask to get this going" problem to try to get server-side code on this project.
• Food For The Heart: The group has made progress. There is a meeting with a possible funding source. There was a great meeting a week ago where our "Angels" told us what they need from us before they'll cut checks. In terms of institutional setup, I'm happy. But in terms of programmer work, we're not at a point where I can start doing anything.

So, that's my progress. So, what's my next set of goals?

• Main Work: I had been planning on taking a PC and turning it into my Linux desktop, then taking my old Linux desktop and turning it into a Git server. On Thursday, I finally hit the point where I had time, and I made a bootable thumbdrive and everything. And today I find that something on my desktop is kicking out spam. So, this is the justification I needed to move forward. And, I need to look up firewalls and cfengine.
• TEDx Web Development: There will of course be the process of finding the team. My #1 goal, though, is to think through the desired look and functionality.
• Food For The Heart: I need to keep part of the business-building process, because that's the part I most need to learn. But, so I can keep my interest up, I'll probably try to make a location-based app of some sort, just to jump-start my skills on the subject.

What do I do with my weather stuff now?

It started with me wanting to know how to code with R. R is about handling and graphing data, and I didn't really have data of my own. So, what I did was find a source I liked for weather and started putting it into a MySQL database.

That source was Google, specifically the XML used for weather in the soon-to-be-closed iGoogle.

Here's that code.

#!/usr/bin/perl

# $Id: hpsetdisp.pl 11 2006-03-22 01:21:03Z yaakov $

# hpsetdisp.pl
# Connects to a JetDirect equipped HP printer and uses
# HP's control language to set the ready message on the
# LCD display.  Takes an IP address and message on the
# command line. My favorite message is "INSERT COIN".
# Keep in mind the limitations of the display when composing
# your clever verbiage.
#
# THIS PROGRAM IS PROVIDED WITH NO WARRANTY OF ANY KIND EXPRESSED OR IMPLIED
# THE AUTHOR CANNOT BE RESPONSIBLE FOR THE EFFECTS OF THIS PROGRAM
# IF YOU ARE UNCERTAIN ABOUT THE ADVISABILITY OF USING IT, DO NOT!
#
# Yaakov (http://kovaya.com/ )
#
# Modified by Garrett Hoofman
# 10/18/2007
# http://www.visionsofafar.com

# Modified by David Jacoby
# 12/10/2008
# http://varlogrant.blogspot.com/

# Modified by David Jacoby, using notify-send and tweaking Unicode
# 09/08/2008
# http://varlogrant.blogspot.com/

# Modified by David Jacoby, moving from Google to NOAA
# 08/28/2012
# http://varlogrant.blogspot.com/


use strict ;
use warnings ;
use Modern::Perl ;
use DBI ;
use Getopt::Long ;
use IO::Interactive qw{ interactive } ;
use IO::Socket ;
use XML::DOM ;
use Data::Dumper ;

use lib '/home/jacoby/lib' ;
use MyDB 'db_connect' ;

use subs qw{  get_weather notify db_upload } ;

my $zip         = '' ;
my $interactive = '' ;
my $degree = '°' ; #the unicode, straight
   $degree = "\x{00b0}" ; #the unicode, specified
my $rdymsg      = "Ready" ;
my $curr_cond ;

GetOptions(
    'zipcode=s'   => \$zip ,
    ) ;

$zip !~ /\d/ and exit 0 ;
my %weather = get_weather ;
db_upload %weather ;

exit ;

########## ########## ########## ########## ########## ########## ##########
sub get_weather {
    my %output ;
    my $parser = XML::DOM::Parser->new() ;
    my $file   = 'http://www.google.com/ig/api?weather=XXXXX&hl=en' ;
    #my $zip    = shift ;
    $zip =~ s{(\d{5})}{}mx ;
    $zip = $1 ;
    $file =~ s/XXXXX/$zip/mx ;
    my $doc = $parser->parsefile( $file ) ;
    exit if ! defined $doc ;
    $output{zip}        = $zip ;
    $output{city}       = $doc->getElementsByTagName( 'city' )->item( 0 )
                        ->getAttribute('data');
    $output{temp_c}     = $doc->getElementsByTagName( 'temp_c' )->item( 0 )
                        ->getAttribute('data');
    $output{temp_f}     = $doc->getElementsByTagName( 'temp_f' )->item( 0 )
                        ->getAttribute('data');
    $output{humidity}   = $doc->getElementsByTagName( 'humidity' )->item( 0 )
                        ->getAttribute('data');
    $output{wind}       = $doc->getElementsByTagName( 'wind_condition' )->item( 0 )
                        ->getAttribute('data');
    $output{conditions} = $doc->getElementsByTagName( 'condition' )->item( 0 )
                        ->getAttribute('data') ;

    return %output ;
    }
########## ########## ########## ########## ########## ########## ##########

########## ########## ########## ########## ########## ########## ##########
sub db_upload {
    my %cond = @_ ;
    my $dbh = db_connect() ;
    for my $k ( sort keys %cond ) {
        $cond{$k} = $dbh->quote( $cond{$k} ) ;
        }
    my $fields = join ',' , qw{
        city        conditions
        humidity    temp_c
        temp_f      wind
        zip
        } ;

    my $sql ;
    my $rows ;

    $sql = qq{
    INSERT INTO weather_data ( $fields ) VALUES (
        $cond{ city } ,
        $cond{ conditions } ,
        $cond{ humidity } ,
        $cond{ temp_c } ,
        $cond{ temp_f } ,
        $cond{ wind } ,
        $cond{ zip }
        ) } ;
    #$rows = $dbh->do( $sql ) or croak $dbh->errstr;

    $sql = qq{
    INSERT INTO weather ( $fields ) VALUES (
        $cond{ city } ,
        $cond{ conditions } ,
        $cond{ humidity } ,
        $cond{ temp_c } ,
        $cond{ temp_f } ,
        $cond{ wind } ,
        $cond{ zip }
        ) } ;
    $rows = $dbh->do( $sql ) or croak $dbh->errstr;

    return ;
    }
########## ########## ########## ########## ########## ########## ##########

Problem is, Google knows I've been grabbing weather data every 10 minutes and has blocked me.

While I fill in all these, I am not pulling in weather data from multiple cities, so I can hardcode city and zip, and I couldjust drop conditions and wind. All I'm really watching is temp_f, and it's a simple conversion to get temp_c if I wanted it.

I have code that pulls from NOAA. Think I'll try to bring that in here.

Weight Graphs

These are a series of generated graphs. The jagged blue line is the raw data of my more-or-less daily weight measures, and the red line is an average of the previous days' measurements. 

I don't measure every day, because I am human and therefore fallible. I take an average of the measurements of the n previous days, which means if I'm trying to get 20 and I skipped twice, I get 18. I would like to have that number graphed here, but I don't know R that well yet. Here, I'm increasing n by five, going from 5 to 30. My goal is to get to the trend line, to show I'm losing weight in total without getting caught up in the day-to-day jagged differences. After some poking, I decided to go with 20 days. Here, I've run the graph several times with different levels so I can see if there's anything better.

Clearly, the first few weeks will look similar, because a five day average will look like a 30 day average if you only have five days to work with. The trend line approximates 1 lb a week, I think. I should add that as a third line.

Big Three for August 27-31

1. TEDx Web Development: I do web for TEDxPurdueU and so far, that has meant maintenance work. I need to get in front of it, come up with an alternate look, and go from there, so we can toss the old and embrace the new. So, specifically, I want to have a front page that demonstrates my desired look. Plus, clear out backlog, and know to what extent we can run dynamic pages on the server.
2. Food For The Heart: This is the "Do that work" mentioned in the previous. I have taken the visual plan from whiteboard to static HTML prototype. I need to look up and use an icon set, sprite or font, set it up for Bootstrap, and add some features.
3. Main Work: I am convinced that my API design is stupid and needs redesign. I have moved the application code for APR into libraries, so I can change config in the main page and easily move between test and production. I need to move all the output from several pages to one, where the caller object says "operator_list" to the API core to get data rather than looking to ajax_operator_list.cgi

If you see me, feel free to ask me how I'm doing on my Big Three.

My Next Personal Project

I don't like distractions. They're distracting.

I also don't like missing a mail from my boss. They're rare, so when they come, they're often important.

So, I shut down alerts in Thunderbird and wrote a tool called jBiff.

jBiff connects to your mail server via IMAP and finds the new mail (thus biff) that meets the standards you want to watch for, then sends an alert via XMPP, aka Jabber (thus j). As is, I'm mostly searching on sender, and I'm not sure, but that might be all that works.

I suppose I should put this on Github. I'm pretty sure that step's overdue. But there's a problem.

jBiff uses IMAP. There are certain mail servers that don't offer IMAP. This obliges me to start working in POP.

And, there's another possiblity. There are many places I receive mail but never check. There are two primary places I check mail, and if you know me, you know what they are. But having a tool to check the status of the others and notify me weekly or so, that would be good.

Easy Does It!

As previously blogged, my Stapes Easy button has been turned into a switch and connected to a Teensy. The last issue was getting the code worked out. I now have that code worked out.

And released to Github.

// ========= ========= ========= =========
void setup() {
  // make pin 10 an input and turn on the 
  // pullup resistor so it goes high unless
  // connected to ground:
  pinMode(10, INPUT_PULLUP);
  delay(2000);
  Keyboard.begin() ;
  }

// ========= ========= ========= =========
void loop() {
  if ( digitalRead( 10 ) == LOW ) {
    Keyboard.press( KEY_SCROLL_LOCK ) ;
    Keyboard.releaseAll() ;
    delay(300); 
    Keyboard.press( KEY_SCROLL_LOCK ) ;
    Keyboard.releaseAll() ;
    delay(600); // Delay of 6/10 second to keep from repeating.
    } 
  delay(10);
  }

This was my first serious use of Github for Windows, too. Far easier to handle than the previous stuff I did.

Easy Like Sunday Morning

Remember my intention to reimplement the Button?

I have my Easy Button torn down, with the speaker out and the circuit bent. You press the button and there is definitely signal going through. I scratched away at the green and soldered in. It works.

On the other side, I have my Teensy, and man, is it teensy. I'm now 93% sure that the micro-B USB cable I've been carrying arround is charge-only, because I couldn't get it to update the code. The code it has on it right now simply kicks out "Hello World" every 5 seconds. The next step, obviously, is to get the Teensy and the easy together, to code it up so that when I hit the button, it kicks out two scroll-lock characters, and to put that code up on Github.

And, to get a nice long micro-B cable.

My Big Three for August 20-25

Matt Hunckler of VergeIndy sent out a challenge:

What’s big for you this week?

One of my closest friends asks me that question each Monday. The first time she posed the question, it caught me off guard. I can still remember my response.

“Um, my to-do list.”

It took about 2 seconds to identify that my snarky response was a lame attempt dodge the question. I hadn’t defined what was truly BIG for me that week. And I didn’t yet understand how powerful this simple question could be.

My friend and I worked together to identify the 3 biggest things for that week and I wrote them on a post-it. It took less than five minute, but it created a clarity that propelled me through the rest of my workweek. And the best part was that, at the end of the week, the list was complete—a welcomed supplement to the ever-growing list of things that need to get done.

I want to pay it forward. So, here’s the question for you:

What are your 3 Big goals for your business this week?

Write your Big 3 down on a post-it. Tell a friend, a mentor, a random person at a coffee shop—anyone. Heck, if you can’t find someone, email me.

Go ahead and reply with your Big 3 directly to this email. I can’t promise that I can reply to all of them, but I will certainly read each and every one. If there’s some way that I can help, I will.

Sharing this stuff can be scary. Trust me, I get it.

He went on to go through is goals. They are good goals. But I'm about my goals. Which are:

1. Get My Work Status Worked Out: there's a thing I want to get into, startup-wise, but it is essentially working for the U, and I can't work twice for the U. Or something. Need to talk to some people who know, and to HR, about this.
2. Do That Work: This involves a location-connected app, and I want to turn the whiteboard work into something we can put in a proposal.
3. Get APR Stuff Done: This is my non-moonlighting project, for the people who currently pay me. I have it feature-complete, but now they want new features. I want to knock 'em out quickly.

I like this idea, and I will try to keep doing this. I might put outlining Monday's Big Three as Friday's last task. And, as I've stated mine, I should turn it around.

What Are Your Three Big Goals?

"I am deeply shamed by what it looks like"

"Why don't you open source your code?" "I am deeply ashamed of what it looks like" (typical result of formal CS education?)

-- Ehud Lamm (@ehud)

If that's you, raise your hand.

It's kinda me, too.

Programming is like a brain dump, a distilled essence of your skills and knowledge at the time you wrote it, and once you have written something, you learn more and know how wrong your initial understanding of the problem and the language really were. Sometimes, your failings are such that you're not even wrong, that blowing it completely is something you have to aspire to.

I have code in my GitHub account. I think I have to be more explicit on licenses, and I know I have to put more of my stuff up there. But, if you want to know how to make a phone call with Twilio and Perl, or dump your Google Plus account to RSS, take a look.

And if you think it's ugly, by all means, submit a fix full of pretty and elegant code.

How To Wake Up: Am I In The Right?

You might've noticed that I am a reader and fan of Lifehacker. I read the site. I listen to the podcast. I follow many of the writers on Twitter and on Google+.

Yeah, it might be getting to that creepy level.

A recent post covered the subject of getting up in the morning. For a good long time in my life, Morning was a point in my life I dreaded and feared. There was a semester in my life when I'd go to sleep after SNL on Saturdays and not wake up until 5pm Sunday evening, so sleep is a known issue to me. Their suggestions are finding an alarm clock that works for you, putting it in a place where you have to get up to shut it off, and resetting your internal clock. All of these are good techniques. The thing I'm mentally playing with is (by several possible means) setting a light to be on at the time I schedule to be up, so that I'm not met with a dark room and thus more inclined to fall back asleep.

The killer app for me has been Smart Alarm Clock, which finds the point in the sleep cycle where you are sleeping the lightest, then wakes you then. This and seriously backing off on caffeine have

I have been vocal on the G+ list on this subject. Some of the others have been saying "stop being a lazy bastard" and "go to bed earlier". I'm generally against shut up and nut up advice, because it really doesn't give you a mechanism for change. If there is ever a point in the day where it is difficult to exercise your will, it is right when you are supposed to wake up. As for getting to bed earlier, that's all well and good, I guess, but there is a point where you can have too much sleep, which leads to being groggy and tired, which are two reasons you'd want to reach over and slap the snooze button.

I think I have kept my responses at a reasonable and respectful level, but I'm not sure how I'm coming off. If you read this and think I'm either factually wrong or brought out the nasty side of the argument, please respond.

Code I Am Not Proud Of

I have a shell script that uses espeak to tell me the time and temperature, which I crontab to run at the top of every hour. I had been using the Google Weather shadow API, the one that's connected to the doomed iGoogle, but it stopped working on me this week.

This broke my code.

So, I've moved to using XML files from NOAA, which I probably should have used from the first.

sub temperature {
    my $parser = XML::DOM::Parser->new() ;
    my $file = 'http://w1.weather.gov/xml/current_obs/KLAF.xml' ;
    my $doc = $parser->parsefile( $file ) ;
    my $nodes = $doc->getElementsByTagName( "temp_f" ) ;
    my $n = $nodes->getLength ;
    my $temp ;

    for ( my $i = 0 ; $i < $n ; $i++ ) {
        my $node = $nodes->item ($i) ;
        my $str  = $node->toString;
        $temp = sprintf '%d' , ( split /[<>]/ , $str )[2] ;
        }
    return $temp ;
    }

I am fine with most of this. I get a parser. I use it to download the file, and get a document. I get an object full of nodes from that document, and I go through each one. Going through each one is where I hit the goofy.

I can get a node that gives me what I want. What I cannot seem to find is the way to get the value from the object. So, I go to regular expressions. There must be a way to get XML::DOM to do the right thing, but I couldn't find it. From there on, that's OK.

It makes me wish, though, that NOAA exported JSON.

Best Advice I've Ever Received

Lifehacker has a series called How I Work where they detail the apps and tools and environment of how various people do what they do. For two of them, one of the questions is this:

What's the best advice you've ever received?

In about 1996 or so, I was a CS student and was working for Purdue's Engineering Computer Network, and my primary responsibility was keeping track of the documentation library. It was a very paper system, and I decided what we needed was an electronic system. At that point, the World Wide Web was relatively new, as was programming with the Common Gateway Interface (CGI), and about all I knew about programming is what I gained from CS 180. But, I figured that I could do this and it would make my life easier, plus I could use it as an excuse to learn Perl.

At the time, the web lead for our group was Kyler Laird. He showed some interest, gave me some encouragement, and told me the following advice:

"Don't fear flat-file databases."

I know, it doesn't sound like much. But, at the time, I had no access to big database systems on campus nor knowledge of them. Eventually, I used a database that tied to a hash (I think I have some code that still uses this, running via crontab somewhere), but on first pass, I could read a file, separate on pipes (because commas tabs are too common within text, while pipes only show up if you're talking about the UNIX command line) , and write a file, then go on to the next thing. Yes, the right thing is using SQL or, now, NoSQL databases (I still don't get them, but okay), but I could get something going now, then get onto the part where my interest lies and my enthusiasm calls.

So, yes, don't fear flat-file databases. Don't fear not using the "right thing" at some step along the way. Go with what you understand, get something going, and fill in your knowledge later.

I Confess, My APIs Suck

I do web development from SQL to CSS, and I confess that all too often, when I want to interact with the server with AJAX, I create another application.

I confess, I hadn't thought about this, but the more I think about it, the more I know this is the right thing.
Instead of this:

{ data:{abc:123} }

I should have

{ action:"upload_keys",data:{abc:123} }

or whatever I'm doing, and then have my ajax_handler.cgi look for action and handle depending on that value.

I will endeavor. to do better in the future.

So, I taught Javascript on Saturday

I taught Javascript to a class of three young adults and four adults on Saturday, as part of a hackathon for Lafayette Tech. Food was provided in the afternoon by Bruno's Pizza, and it was tasty.

I'd show you my slides, but the value is somewhat slight, because of two issues. First, I get more nervous the more tightly I prepare a presentation, which means that, for me, it is best if I just have a few slides to remind me where I am. Second, I used impress.js and abused the awesome power of 3D visualization. I will use it again, and next presentation will be better thought out.

I found out about jsBin, which I will use again if I ever have a class on Javascript again. Jack Stuckey pointed me to it, and to some code which created a box and moved a ball around it. I made the ball into a ball object and made a ball object maker, pushed the objects into an array, and moved the balls around. I really think I'm not far from turning this into a Javascript implementation of Asteroids.

I think the cool move would be to port this to Processing.js.

Anyway, thanks to all who attended, and I hope you learned much.

I like mustache.js but don't know if it works with jQuery-ui

As I have mentioned, I am currently looking for a templating engine in Javascript which will allow me to create great gobs of HTML things without the bulk of assembling them in jQuery. I couldn't get pure.js to work for me, but from the first with mustache.js, it was working close to how I was thinking.

Here is the specific function I want to generalize and make smoother. It uses jQuery-UI to make a pop-up.

// ========= ========= ========= ========= ========= ========= =========
// creates popup where you enter the accessions desired
// --------- --------- --------- --------- --------- --------- ---------
apr.accession_popup = function () {
    // I do this enough that I should make a specific library to handle
    // this style of popup

    // must investigate reformatting

    $( '#paste_from_spreadsheet' ).remove() ;
    $( '<div id="paste_from_spreadsheet" />' )
        .attr( 'title' , 'Add Accessions' )
        .dialog() ;
    $('.ui-dialog-titlebar-close')
        .click( function () {
            $( '#paste_from_spreadsheet' ).remove()
            } ) ;
    $( '<div/>' )
        .text(
            'Add one or more valid Accession IDs. ' +
            'When done, click "Go" or hit "Enter". '
            )
        .appendTo( '#paste_from_spreadsheet' ) ;
    $( '<textarea/>' )
        .attr( 'id' , 'spreadsheet' )
        .attr( 'name' , 'spreadsheet' )
        .appendTo( '#paste_from_spreadsheet' )
        ;
    $( '<div/>' )
        .attr( 'id' , 'spreadsheet_go' )
        .text( 'Go' )
        .appendTo( '#paste_from_spreadsheet' ) ;
    $('.ui-icon-closethick')
        .text( 'X' ) ;
    $( '#spreadsheet_go' ).click( function () {
        apr.add_accession( $( '#spreadsheet' ).val() ) ;
        $( '#paste_from_spreadsheet' ).remove()
        } ) ;
    $( '#spreadsheet' ).focus() ;
    } ;

And here is the version using mustache.js.

function my_popup () {
    $( '#paste_from_spreadsheet' ).remove() ;

    $( '<div id="paste_from_spreadsheet" />' )
        .attr( 'title' , 'Add Accessions' )
        .dialog() ;
    $('.ui-icon-closethick')
        .text( 'X' ) ;
    $('.ui-dialog-titlebar-close')
        .click( function () {
            $( '#paste_from_spreadsheet' ).remove()
            } ) ;
    var template = '\
    <div>\
        Add one or more valid Accession IDs.\
        When done, click "Go" or hit "Enter". \
    </div>\
    <textarea id="spreadsheet" name="spreadsheet" ></textarea>\
    <div id="spreadsheet_go"> Go </div>\
    ' ;
   var html = Mustache.to_html( template,'' ) ;
    $( '#paste_from_spreadsheet' ).append( html );
    $( '#spreadsheet_go' ).click( function () {
        apr.add_accession( $( '#spreadsheet' ).val() ) ;
        $( '#paste_from_spreadsheet' ).remove()
        } ) ;
    $( '#spreadsheet' ).focus() ;
    }

Notice that there isn't a lot of filling here. I can do filling, but half the time, I'll want to replace that textarea with an input type="text", which means that, if I dive in on this one, I'll have to use the awesome power of mustache.js in some way.

Right now, though, I'm not sure there's enough win there to make it worth my while, or if I could write my jQuery stuff better and make it work like I want it to.

I think Pure.js is out of the running

I have been on the lookout for a templating engine for Javascript. Thanks to the great outpouring of suggestions, I have a number of suggested libraries to test.

And first in the running was Pure.js. There is something to it, and it sounds like the author of the library, BeeBole, is a good programmer with an active user base, and he did try to be helpful to me, sending me this link to where he pulled templates from other pages.

If I could rewrite prompt() such that used a textarea equivalent rather than a popup with input type="text", I'd probably toss the whole thing. But, as this is what I'm looking for, my desire is for a solution that could work on a page that looked like <body></body> and what I see of pure.js is that it is much more aligned with replacing values it finds within existing structures.

I'm thankful for his help, but I'm off to try moustache.js.

Free Outdoor WiFi in Downtown Lafayette

City of Lafayette

"Want to live in a progressive city with 21st century technology?"

That's Lafayette mayor Tony Roswarski's question, and if you live in Lafayette, "Now you do", as the City of Lafayette and Wintek announced the opening of "Free Wi-Fi on Main Street."

I haven't tried it yet, but I expect to do so today. There's a bunch of places I go downtown that are in that map, so I am sure that I'll have a great chance to do so.

As a reminder, though, if you are using free Wi-Fi, there are few controls to keep potential bad guys away from your system. Lifehacker has writeup on how to keep your system safe on free Wi-Fi networks.