Configuration: The Devil is in the Details

I work in a lab. The simple fact is that, in the 21st Century, all science is computer science, and that's especially true here. We have several computers whose entire purpose is to control scientific instruments and handle the data that comes from them. But we don't want the data to be on the instrument machine. We want it to be on the server.

This is where FTP, and especially the Perl module Net::FTP, come in. We try to keep it locked down, configured at the firewall so that only the server can FTP into the instrument and even then can only move instrument->server, not the other way around.

One specific machine was set up with Windows 2000 and the FileZilla FTP server. The software, we were told, wanted 2K, and so we ran with that. FileZilla was about the first Google result I hit when I searched on "windows ftp server", and I cannot say there was ever a point where I regretted using FileZilla for all my FTP needs.

Then there was a necessary upgrade that needed Windows XP. The machine had 2 HDs in it, so I installed XP on disc 2 and made it dual boot. And, since it was XP Pro, I had IIS already. I may have had it on the 2K, I never thought to check. Anyway, the machine now spends most of the time under XP. I have had to change the upload-and-munge script to handle the new, and I'm not sure how much is IIS-vs-FileZilla and how much is Instrument-Software 1.3 vs Instrument-Software 2.0.

Where the story threatens to become interesting is this: The person who runs on that instrument, she makes a CSV config. Normally, she copys it, changes it, and seemingly, as long as the Excel file was open, it wouldn't upload. This was the 2K and FZ behavior.

Windows XP and IIS, it seems, FTP anyway, no matter if the Excel file is still open. And I had to clean up after it because of it. Not a problem, to be sure, but something to be aware of.

My first Perl one-liner

jacoby@oz:~$ ls -l *iso  | perl -E 'while (<STDIN>) { $x=(split m{\s+}mx , $_)[4] ; say int $x * 100 / 723488768 , "% done"  } '

I'm downloading Ubuntu 9.10 via FTP. I know the final size. Via ls I know the current file size. if a/b = y/100, where a = the current size, b = the final size and y = the percentage done, this gets the data and solves for y and converts to an integer. (Do I really need it to seven points? No.).

Thank you -E for making command-line Perl so much easier

Running an OddMuse wiki with CKeditor

I admin a few wikis, all using the OddMuse engine. I like Wiki markup, but it is difficult to make it do certain things. One thing we like doing at work is import Excel-style tables, so to do that, we use the FCKeditor.

I have always liked the function. I have not always liked the name. It seems like I'm cussing, which is something I have spent some time trying to stop doing. If I resort to vulgarity, I want it to be because the circumstances have overwhelmed me, not because they reside in the forefront of my vocabulary, and FCK reminds me of a shirt. Never mind the fact that the programmer named it after himself, Frederico Caldeira Knabben.

I must not have been the only person feeling like that, because the new name is CKeditor. But while there are docs for making my wiki engine of choice work with FCKeditor, not so much for the shiny new CKeditor. And it is shiny....

The problem, at the moment, is that I can get it to write wiki markup but I can't get it to write and save HTML, which really is helpful if you want to display tables, which I do.

Will post code once I figure it out

Reading IMAP directories with Perl

I use and like Thunderbird. Not the greatest thing since sliced bread, surely, but better for managing most of my mailboxes than pine and the school's web interface, which might soon be replaced with Gmail. Plus, IMAP!

The thing that sucks is alerting. If a mailing list gets updated, don't tell me. I shunt that stuff off into a subdirectory for a reason. In fact, there are three classes of people whose emails I want alerts about:

• Family — mostly my wife and parents, but also my children, cousins, etc.
• Bosses and Co-Workers — If you're involved in me getting paid, of course I need to know when you contact me
• Friends — this is a low third, which is understandable, I hope.

Everyone else, all the spammers and mailing lists, I'll read your mail when I get around to it.

So, I looked into Perl's Mail::IMAPClient.

#!/usr/bin/perl
use Modern::Perl ;
use IO::Socket::SSL ;
use Mail::IMAPClient ;
use Getopt::Long ;
use Carp ;
use Data::Dumper ;

$Data::Dumper::Indent = 1 ;

my $server = 'imap.gmail.com' ;
my $username = 'me' ;
my $password = 'nope' ;
my @sender ;

GetOptions(
    'sender=s' => \@sender ,
    ) ;

my $sender = join '|' , @sender  ;

my $socket = IO::Socket::SSL->new(
   PeerAddr => $server ,
   PeerPort => 993,
   )
  or die "socket(): $@";

my $client = Mail::IMAPClient->new(
   Socket   => $socket,
   User     => $username ,
   Password => $password ,
   Ignoresizeerrors => 1 ,
   )
  or die "new(): $@";

if ( $client->IsAuthenticated() ) {
    $client->select( 'INBOX' )
    or die "Select 'INBOX' error: ", $client->LastError, "\n";
    #for my $msg ( reverse $client->messages ) {
    for my $msg ( reverse $client->unseen ) {
        #my $string = $client->message_string($msg)
            #or die "Could not message_string: $@\n";
        my $from = $client->get_header( $msg , 'From' ) ;
        my $to   = $client->get_header( $msg , 'To' ) ;
        my $subject = $client->subject($msg)
            or die "Could not subject $@\n";
        if ( $from =~ m{$sender}i ) {
            my $notify = '/usr/bin/notify-send' ;
            my $title = 'New mail from ' . $from ;
            my $body = $subject ;
            my $icon = '-i ~/Pictures/Icons/icon_black_muffin.jpg' ;
            my $time = '-t ' . (1000 * 10 ) ;
            $body  =~ s/(['"])/\\$1/g;
            $title =~ s/(['"])/\\$1/g;
            $title = join q{"} , '' , $title , '' ;
            $body  = join q{"} , '' , $body , '' ;
            qx{ $notify $title $body $icon $time } ;
            }
        }
    $client->logout();
    }

It takes one or more sender substrings. mail_notify.pl -sender '@purdue.edu' , for example, would pop notifications from all mail from Purdue University addresses. And the good thing is, you don't need to change much more than the server name, username and password to make it work on my work server.

TV: Conjecture about White Collar

I didn't intend for this space as a place for me as a place to talk about TV, but it's where I have cut tags to protect people from spoilers. C'est la vie.

White Collar is a show on USA Network. The main character is Neil Caffrey, a con man. He was caught by Peter Burke, an FBI agent. He escapes on the cusp of his release, looking for his wife. Peter catches him again, and, in part inspired by his sense of fairness, springs him to serve as a consultant. There we have the premise.

At one point, Peter is looking for a way to get a warrant to enter a warehouse he is sure but cannot prove is the location of a crime scene. Neil, a criminal not a cop, wonders why they just don't go in. Peter then passes him the Big Book of Warrant Law. Neil reads the Big Book, figures it out, and gets himself and the ankle bracelet location tracker dragged into the warehouse. Peter knows the game when he follows the machine-gun-toting FBI entry team into the warehouse, quoting chapter and verse as to why he can go in without a warrant. I don't know that Peter knew what he was doing when he gave the book, but he certainly knew when he got there.

Consider, for a moment, The Sting. That's the great con movie, and while we're in on great parts of the con, we don't know the final bit until the end. Same with the Ocean's Eleven remake. The audience can't know it all. So, what's the underlying con for White Collar.

We don't know what's going on with Neil's wife, but we know

• Neil knows Peter about as well as Peter knew Neil
• This includes Peter being smart and liking smart, and we mean really smart. He has great ire over the large number of Harvard graduates in his unit who are not nearly as on-the-ball as he is.

He had to know that Peter would come after him if he escaped. If he just waited a month, he would've been a free man, rather being chained to Peter for four years, but now he has a badge. Presumably, he thinks he needs that kind of backing, the kind of backing that can fill a warehouse with armed agents in very little time, to confront whoever has his wife. Which makes an acceptable series-long Big Bad.

Trying Dropbox

I'm trying DropBox. It seems neat. I'm at work. I have two machines running, my main Linux box and my XP netbook. I have a directory in Linux, /home/jacoby/Dropbox/, where I can put things. I put it there, and after some time for uploading and downloading, the items in that directory are placed in C:/Documents and Settings/jacoby/My Documents/My Dropbox/ on my netbook, and the ~/Dropbox/ directory at home.

Isn't that cool?

I can also tell DropBox to share a subdirectory or file with friends who also use DropBox.

This is where you come in.

Sign up and you get to share things with me, and we both get 256MB additions to our default 2GB disk quota.

The thing I'm starting to do is put the images I use a lot, my favorite user icons and background images, in ~/Dropbox/Photos? so that they're effortlessly spread across all the places I'd use them. I'm also considering a ~/Dropbox/bin/ directory that will hold the scripts I go to most often. A poor excuse to not sit down and learn to love git, true, but still kinda neat.

A Few Words on Net Neutrality

I am a user of an ISP. I'm sure you heard of it. I feed my TV through the same wire I feed my home network with. I get 10mbps down and 3 up, more or less, but I share a local network with those people in my neighborhood, so if lots of folks are running some tools (BitTorrent, for example), that could tend to hinder my network usage. This makes me unhappy with my ISP, and this makes my ISP unhappy, so they throttle back the network traffic associated with those tools.

This capability is built into the high-level switches that ISPs use. There's an acronym, QoC for "quality of service", that's all about this kind of thing.

I used to use a service, and I did like it, called Vonage. This provides VoIP, or Voice over IP. My ISP, which also does cable TV, is becoming a competitor to Vonage, handling their own VoIP. It seems like it would be relatively simple to throttle network going to and from Vonage just like they throttle traffic handling BitTorrent packets. This would be technologically easy for them — we've established that they have the technology — but if you, like me, have decided that Vonage is the way you want to go, you get tired of an ISP that sucks like that and move on to one that doesn't suck like that.

This is what I understood Net Neutrality to be about.

But I'm thinking I might be getting to be wrong about that.

I think my statements so far shows my biases. To a certain extent, shaping the traffic of a network to make it more usable is the business of an ISP. But there are points where that is illegitimate. My feeling is that Networks should have Neutrality, but that, with an acceptable level of competition, ISPs that don't mess with you have a competitive advantage over those who do. But, ultimately, that's an issue between the people who provide the network to my house and the people who have things running on their servers. As a programmer, I can see myself eventually working for a start-up, but beyond that, really, there's little direct effect this would have on me personally, and for most people.



Glenn Beck is Glenn Beck. Like him or not. I'm not going to try to argue into accepting him. Skip ahead to 3:55 and you see Van Jones talking about Net Neutrality. Or something that's unrelated to Net Neutrality as I understand it, but going by that name. Net Neutrality meaning free internet to everybody, that doesn't seem ... well, forget "right" or "wrong". It doesn't seem related to the subject at hand.

Let us step back and look sideways. Let us assume that this graph is right. Phones these days have more powerful chips than any computer I had before 2002. And that's Verizon, AT&T, Sprint, T-Mobile — lots of companies that are ISPs. I'm not seeing the problem Van Jones sees, but I can see that Net Neutrality having a generally neutral effect on the population at large.

But what, if anything, am I missing?

Perl IS a community!

Ever have code that doesn't work and you don't know why? I do on occasion, and the error messages didn't tell me anything useful, so I decided to try Perl::Critic on it. This is less a tool to debug your non-functioning program than a program that tells you a better way to have your functioning program function, but I thought it could tell me something.

I generally start all my programs these days with:

use Modern::Perl ;

This takes the place of this:

use 5.010 ;
use strict ;
use warnings ;

This is very nice, in that it gets you to the cool stuff in Perl in one line.

But...

Perl::Critic wants you to run using strict and warnings, and when you use Modern::Perl, it doesn't explicitly say that you are using them, and Critic dinks you for it.

I thought that was interesting.

And, today, when you find something interesting, you tweet it. I also put it on Identi.ca, which is like twitter but mostly programmer types go there. And I got a response. From chromatic on identi.ca and from the Critic folks on Twitter.

That just warms my heart. Not specifically the help to make it work, which is of course great, but that the developers are listening.

And, incidently, making a .perlcritic file with this in it makes it work:

equivalent_modules = Modern::Perl

Modern One-Liner Perl

I have never been much of a fan of Perl one-liners. Take every slur about Perl that exists in the popular imagination, and that's the reality for one-liners.

perl -e 'print 51 * 51 , "\n"'

But now, by going -E instead of -e, you bring in Perl 5.10. This means you get say, which is by far my fave Perl 5.10 addition.

perl -E 'say 51 * 51'

I'm already starting to do one-liners more, now that I have say. Using realias is also helpful with this.

Credit where due.

Avoiding Wallop with Perl

This is a little bit of Perl code that copies a file, but if a file by that name exists, appends a number between the name and the suffix.

#!/usr/bin/perl
use Modern::Perl ;
use File::Copy ;

my $from = '/home/jacoby/test.test.pl' ;
my $to   = '/home/jacoby/target.target.pl' ;

while ( -f $to ) {
    my @to = split m{\.}mx , $to ;
    my $suffix = pop @to ;
    if ( scalar @to == 1 ) {
        my $num = 1 ;
        push @to , $num ;
        push @to , $suffix ;
        }
    elsif ( $to[-1]  !~ m{\D}mx ) {
        my $num = pop @to ;
        $num ++ ;
        push @to , $num ;
        push @to , $suffix ;
        }
    else {
        my $num = 1 ;
        push @to , $num ;
        push @to , $suffix ;
        }
    $to = join '.' , @to ;
    }
say "F\t" . $from ;
say "T\t" . $to ;
copy $from , $to ;