2015/05/19

01010100 01101000 01100001 01110100 00100111 01110011 00100000 01110111 01100101 01101001 01110010 01100100 00101110 00101110 00101110

Slight language warning.

The PaleoFuture blog on Gizmodo had a review of Tomorrowland from a Robot from the Future today, and of course it's in binary. I saw that shortly after coming in, and I thought "Hey, that'd be an interesting exercise to use to prime my programming pump, get my head in the right mindset."

So, I wrote code to decode it. I read perldoc -f pack and thought "Nope", so I found Data::Translate, which I now know is a very simple wrapper.

Once I wrote that, I decided the thing to do was to write b2a.pl and a2p.pl, because why not? But I found that this
a2b.pl Bite my shiny metal ass
gave me this
01000010 01101001 01110100 01100101 00100000 01101101 01111001 00100000 01110011 01101000 01101001 01101110 01111001 00100000 01101101 01100101 01110100 01100001 01101100 00100000 01100001 01110011 01110011
but this
b2a.pl 01000010 01101001 01110100 01100101 00100000 01101101 01111001 00100000 01110011 01101000 01101001 01101110 01111001 00100000 01101101 01100101 01110100 01100001 01101100 00100000 01100001 01110011 01110011
gave me this
Bitemyshinymetalass
So, what went wrong? Where did the spaces go?

This will take some time with me trying to grok pack, which is so unrelated to my daily responsibilities that I installed Data::Translate simply so I wouldn't have to learn a thing, but it seems that, if you pass pack the space character in binary (00100000), it returns an empty string. This is a somewhat strange thing, but once you understand the behavior, you can react to it.


(The catchphrase for Bender seemed the most appropriate text to use.)


2015/05/18

Some Wisdom From Twitter

2015/05/14

There are Two Javas

So, this happened...

On the one hand, I wrote this on a day when I woke up after watching this fantastic conference talk on Neo4J and RNeo4J from Nicole White before bed. This had served to stoke the fires of my Big Data / Data Science jones and had moved graph databases from the corner of NoSQL that I just didn't get to something I had the concepts for but needed to play with to understand.

On the other hand, I wrote this on a day where a tech in my lab was upgrading an instrument machine (a computer that controls a data-generating scientific instrument), and in the process, upgraded Java. This lead to that corner of the lab to enter a time of complete shutdown while nothing worked.

I told someone that day, saying "one of my users updated Java", and without any further explanation, he knew exactly what the problem was, exactly how bad it could be. "They didn't", he said.

This is not a case of me hating Java as a language. I mean, I do, but nobody is asking me to write it, at least not in this context. With Neo4J, I there are modules such as RNeo4J, serving as an interface between the database and the language you're doing your work in. I don't care what language MySQL or MongoDB are written in; I just care that they'll store my data and give it back when I need it.

The one thing that distinguishes Java is "write once, run anywhere"; you write it and compile it so it runs in a VM, abstracting away all connections to underlying systems. Except, there are systems created years ago, where the software was written for a version of the JVM that has been replaced and abandoned a long time ago, where they even deploy computers running XP that were built for Windows Vista or even Windows 7. These exist to some extent because for schools of engineering and computer science, you have to teach the basics of computing using something, and somewhere along the line, the go-to something for introductory courses became Java. So, if you're making complex, computer-controlled scientific instruments, you're needing to design for what students know, rather than spend time and money teaching them a new language as well as the interface to your science. And, because once you sell it, you don't get money for upgrading the interface, so you just don't.

That is my biggest fear. That running a big program to run a big program will tend to choke a computer is an issue, but not as big an issue in this case. It'd run on an older computer, but it's a far newer computer than I started to develop my Java hate on, decades ago. I mostly think that, in context of modern, well-backed projects like Neo4J and Cassandra, I wouldn't end up where I can't upgrade the JVM because it'll break my tools and but every time I use the mouse, a popup shows up saying I'm several major versions behind, and I couldn't download and store the "right" version for my software if I wanted to.

Those are the two Javas: The bright shiny one that promises a good future, and the previous version that keeps you stuck in the past.

2015/05/13

Being Open, Being Wrong, Being Corrected, Being Smarter: Response about MongoDB

Just heard a podcast from Developer Tea where Ben Lesh of Netflix explain how he likes it when he makes mistakes, because then he learns, and we should always be learning. (I listened to it when driving, so I didn't write down the quote, and I'm not sure if it's in part 1 or part 2 of the interview.)

Yesterday, I posted on issues I had with MongoDB and Perl. I had done what I had wanted before, I was sure, but my code no longer reflected it, and Shub-Internet was not responding with the answers I desired. If you don't recall:

# This worked to get one value:
my $find_one = $collection->find_one( { date => $today }) ;

# but if I wanted the whole thing, this didn't work:
my $find = $collection->find ;

As it turns out, there's a specific piece of information I did not find, but David Golden of MongoDB set me straight.

# but if I wanted the whole thing, this didn't work:
my $find = $collection->find ;
my @all = $find->all ;
# and in the case I'd want:
my %all =
    map { $_->{ date } => $_ } #map, grep and sort are all-powerful
    $find->all ;
say Dumper \%all ;

This is great. Wonderful, even. I had a failure in understanding, I was open about it, I was corrected, and now I am a better programmer for it.

I do sorta wish I had given Stack Overflow a deeper search first, though....

2015/05/12

Having an issue with MongoDB and Perl

I've learned to love SQL. At first I had lots of functions that did SELECT * FROM table, then I would put everything together at the last moment, with big slow programs. Slowly but surely, I found myself trusting my tools more and more, writing more and more complex SQL queries to give me exactly what I needed, in as close to the right form as I could.

Now that I have some level of expertise with SQL, we have it as a core of our infrastructure at work, so any experimenting with NoSQL databases. It's mostly for my toy bits and administrivia, not really holding any real data, but I am keeping my current FitBit step count in it.

I am currently finding that I can find_one() all I want, but I can't find(). I am sure I used to. I'm also sure that I'm doing something stupid, not the MongoDB Perl module, but I can't tell what I'm doing wrong.

Help?


2015/05/04

Putting Current Temperature into my Bash Prompt

my life as drawn by phdcomics.com
I work in a sub-basement. I've been lead to understand that this is desirable for us because the lab equipment is on the solid cement of the foundation, so that we don't have to worry about footsteps shaking it, but I honestly don't know.

The problem at least for me, is that it I'm well-isolated from the outside world, so that it could just as easily 9° or 90° outside, and I wouldn't know. I've felt apprehension to wander upstairs, only to find that, once I got there, it wasn't all that bad.

With a previous printer, I would write the current conditions to the front-panel LCD, because Perl, but we've moved from HP to Xerox, and I no longer have that ability. I now have it tell the time and temp at the top and bottom of the hour, to keep from being so deep in the zone that I think it's lunchtime when it's really quittin' time. I also had code to generate notify balloons, but a recent switch from Ubuntu to Debian made that unusable.

I've been playing with NoSQL databases and have found that I spend most of my time with MongoDB, but once I started thinking about the weather problem — I need it to be available to any of my n terminals, I don't need structure — I decided that Redis was the way to go.

Each NoSQL database works in a different way. MongoDB works as a Document Store, holding a piece of structured data until you call it back. I have code to record daily status updates and send it to bosses where each day holds an array full of objects holding a timestamp and status report, and at 6pm, if the day's array holds data, it formats that data and sends the mail. This is more structure than I really need in this context.

What I really need is essentially a hash table that I can access from many different programs. If it falls down, that's fine; I don't need to remember what the temperature was at 4am, I just need to know what it is now, or within the last 10 minutes. Redis works like that, and (unless you have it commit to disc), it's only in memory, so fast. (I think; I've only played with Redis, not put production data into it.) I think this feature would be good in, for example, Raspberry Pi uses, where excessive access could prematurely kill your SD card.

Anyway, here's my code to store the temperature, to pull it back out, and a replacement $PS1 to show you how to add it to your prompt. It backends to forecast.io, my most recent go-to for weather information. The API is very useful and easy to use. (I don't put "Powered by Forecast" in my prompt but I do put it in the program that uses notify-osd and Pushover.) Use it in good weather.

2015/05/01

Explaining the Schwartzian Transform

#!/usr/bin/env perl

# what is a schwartzian transform?

# http://en.wikipedia.org/wiki/Schwartzian_transform

use feature qw{ say } ; 

# notice how all of these are not in any order?

my @tedx_feeds = qw{
    tedx
    tedxboston
    tedxindianapolis
    tedxiu
    tedxpurdueu
    tedxlafayette
    tedxohiostateu
    tedxpsu
    tedxuiowa
    } ;

# we'll read this backwards. 
my @sorted = 
    map { $_->[0] } 
    # ... and then we just use the words, jettisoning
    # the lengths and creating the final array.

    sort { $a->[1] <=> $b->[1] }
    # ... which we sort numerically by the length
    
    map { [ $_ , length $_ ] } 
    # ... becomes an array of arrays, being all
    # the words and their lengths, starting 
    # with [ 'tedx' , 4 ] ...

    @tedx_feeds ; # the array in unsorted order ...

# Start here and go up <-----------------<<<<<

say join "\n" , @sorted ;

# output looks like:
# tedx
# tedxiu
# tedxpsu
# tedxuiowa
# tedxboston
# tedxpurdueu
# tedxlafayette
# tedxohiostateu
# tedxindianapolis

It took me quite a long time to get the Transform, and the use of map and grep. But they aren't hard. Once you get them, you'll use them everywhere. Maybe too much....

2015/04/16

Template vs Text::Handlebars

A previous attempt at supporting Handlebars
In the comments of Where To Generate the HTML?, Tom Molesworth points out that, if you're going to be having a template that generates both on the client and server side, having two templates instead of one leads to more maintenance work.

This is most certainly true. In that case, it was just enough code to demonstrate a problem, not something I would expect to maintain. In fact, if users who weren't me were regularly finding that code, I'd be more more likely to delete it than support it. But the broader point; that there are things that I might create on the server side, then regenerate on the client side, and as I do more and more web work, I become more and more likely to want to do that.

Below I show the creation of the table and the creation of output in both Template Toolkit and Text::Handlebars. You will notice that you have a handle for each element in TT ( FOREACH row IN table being akin to for my $row ( @table ) ), while with Handlebars, you get this. The joke about JavaScript is that you want to throw up your hands and scream "this is B.S.!" but you're not sure what this is.

On the other hand, I've always been proud of making good, well-formatted HTML, and I do not like the formatting of the table that TT created. I've tried to make TT format more like I want it to look, and generally I just give up and make sure that it has everything my users, my code and I need of it instead. The Handlebars-generated HTML is exactly like I'd want my code to look.

Plus, Handlebars is cross-platform. I understand there's Jemplates to handle TT in Javascript, so strictly speaking, both are.

So, there's the code and the the rendered output. What do you think? What's your preference?

2015/04/15

We Have The Facts And We're Voting Arduino.cc

You've seen my posts on Arduino before. It's a microcontroller, released with an Open Source license, that finally allowed me to transition from programming bits to programming atoms. OK, I need to gear up and get a servomotor or something, but the point remains.

For years, the place to go for Arduino downloads and documentation was arduino.cc. Things are now changing...

Arduino LLC is the company founded by [Massimo Banzi], [David Cuartielles], [David Mellis], [Tom Igoe] and [Gianluca Martino] in 2009 and is the owner of the Arduino trademark and gave us the designs, software, and community support that’s gotten the Arduino where it is. The boards were manufactured by a spinoff company, Smart Projects Srl, founded by the same [Gianluca Martino]. So far, so good.
Things got ugly in November when [Martino] and new CEO [Federico Musto] renamed Smart Projects to Arduino Srl and registered arduino.org (which is arguably a better domain name than the old arduino.cc). Whether or not this is a trademark infringement is waiting to be heard in the Massachusetts District Court.
According to this Italian Wired article, the cause of the split is that [Banzi] and the other three wanted to internationalize the brand and license production to other firms freely, while [Martino] and [Musto] at the company formerly known as Smart Projects want to list on the stock market and keep all production strictly in the Italian factory.
(quoted from Hackaday)

I'll repeat a line. Whether or not this is a trademark infringement is waiting to be heard in the Massachusetts District Court. It is a matter of law, not up to me. As the boards are Open Source, you are well within your rights to make your own, as we at Lafayettech Labs have considered. (We decided that the availability of inexpensive small boards at SparkFun and AdaFruit make our board redundant.) I have a few boards from non-arduino.cc origins, but I have and am glad to have some that are arduino.cc boards, supporting the project that's inspired me. (And I'm reasonably sure that those boards were actually manufactured by Smart Products Srl).

You are also within your rights to fork a project and make changes, which is how Arduino code got ported to TI boards with Energia, and which is what Arduino.org did.

Using the same name.

GitHub user probonopd reported an issue with the Arduino fork:
Rename this fork and use less confusing versioning
This fork of the Arduino IDE should be renamed something else to avoid confusion, and a version number should be chosen that does not suggest that this fork is "ahead of" the original Arduino IDE when in fact it is not. Naming open source projects should follow community best practices.
As for "arduino-org", one can read on https://github.com/arduino-org that "This organization has no public members. You must be a member to see who's a part of this organization". This is quite telling. The real Arduino team is at https://github.com/orgs/arduino/people as everyone can clearly see.
There is a difference between trying to fork a project and trying to hijack a project, and I think this is clearly what Arduino.org is trying to do. I urge everyone interested in Open Source to urge them to rename and re-version their fork, by making this issue known among their community and by firmly but politely agreeing with probonopd and his, like the over-300 users who already have.

Rename the fork!

Subroutine Signatures and Multimethods in Perl, Oh My!

This is a question I asked about on Google+ a while ago: How do you have multiple subroutines responding to the same name, distinguished only by the subroutine signature? It isn't yet a part of the experimental "signatures" feature that came in for 5.20 (to my understanding).

Turns out, there's a name for it. Multi-methods.

And it turns out there's a module in CPAN already: Class::Multimethods.

And, like so many very strange, very cool modules you don't know you need until you get there, it was written by Damian Conway.

The way I expect I'd handle it, I'd have each test() version handle the input to get what I need as input, then have an internal subroutine handle the munged input. But then again, I'm not sure. My mind is still blown by this, so I don't really know how to use it.

You'll excuse me; it took several years before I felt confident enough to take on the Schwartzian Transform, much less understand and use it, so I'm very happy to get this, even before knowing where or if I'd use it.

(I don't use unicode in the examples, but think that more people should know about the unicode features in newer Perl.)