Perl on Ubuntu on Windows: A Solution

I suppose I should've just been more patient.

After a while of waiting and watching and trying to think of a better bug report, one that might get a response, and failing, I got a response.

You can't install the module because File::Find can not recurse directories 
on the file system that is in use by Ubuntu on Windows.

The solution is to edit Config.pm:

sudo vi /usr/lib/perl/5.18.2/Config.pm
Set dont_use_nlink to 'define':

dont_use_nlink => 'define',
Now it's possible to install all modules you want!
(this is a dupliceate of #186)

I haven't made this change yet. I am loathe to change core modules like that, although I have done so in the past. Because I have done so in the past, and it ended up stupid. But I will likely do it.

I was mentally throwing it to the kernel, but was wrong, which is interesting. Makes me think that, rather than running Ubuntu on Windows, doing something with Vagrant might be the better plan.

Net::Twitter Cookbook: Tweetstorm!

I had planned to get into following, lists and blocking, but a wild hair came over me and I decided I wanted to make a tweetstorm app.

What's a tweetstorm?

And this is the inner loop of it.

    my @tweetstorm_text ; # previously generated. URL shortening and numbering are assumed
    my $screen_name ; #your screen name as text
    my $status_id ; #created but not set, so first tweet won't be a reply-to

    for my $status ( @tweetstorm_text } ) {

        my $tweet ; # the object containing the twee
        $tweet->{status} = $status ;
        $tweet->{in_reply_to_status_id} = $status_id if $status_id ;

        if ( $twit->update($tweet) ) { #if Tweet is successful

            # we get your profile, which contains your most recent tweet
            # $status_id gets set to the ID of that tweet, which then
            # gets used for the next tweet.

            my $profiles
                = $twit->lookup_users( { screen_name => $config->{user} } ) ;
            my $profile = shift @$profiles ;
            my $prev    = $profile->{status} ;
            $status_id = $profile->{status}->{id} ;
            }
        }

So, that's it. I've thrown the whole thing on GitHub if you want to play with it yourself.

Thoughts on ML Techniques to better handle Twitter

Thinking things through afk and thus not as polished as my normal posts.

Been thinking about grouping my friends (those I follow) strictly by relationship mapping. In part, I haven't done this because I can't read the math in the paper describing it, and in part because there are points where I serve as connecting node between two clusters and they have started interacting independently. I know a joy of Twitter is that it allows people to connect by interest and personality, not geography, but when a programmer in Washington and an activist in Indiana talk food and cats with each other, it makes my "programmer" cluster and my "Indiana" cluster less distinct.

So, what to do?

Topic Modelling.

I know about this via the Talking Machines podcast, and, without mathematic notation, if you take a body of text as a collection of words, the words it contains will vary by subject. If the topic is "politics", the text might contain "vote" and "veto" and "election" and "impeach". If the topic is "football", we'd see "lateral", "quarterback", "tackle" and "touchdown".

Rather than separating Twitter followers into groups simply by interactions, I could start with both certain lists I have curated (and yes, there are both "local tweeters" and "programmer" lists) and hashtags (because if you hashtag your tweet #perl, you likely are talking about Perl) to start to identify what words are more likely to come up when discussing certain subjects, then start adding then to those lists automatically.

If I can work this out 140 characters at a time.

Perl on Ubuntu on Windows: Finding The Right Test Case

I'm still hung on getting CPAN working for Ubuntu on Windows. In the comments, Chas. Owens gave me great advice for proceeding with this problem:

Write a minimal test case, run the test case with strace on Ubuntu on Windows and Ubuntu on Linux. The outputs are supposed be identical, if they aren't (and we expect them not to be because of the error), then you have a good shot at identifying the misbehaving syscall (which is the only difference between Ubuntu on Windows and Ubuntu on Linux).

Once you see the difference, look into what the syscall does and try to determine which system is implementing it correctly (probably Linux). If it is Windows post to the github tracker with the test case and the identified syscall. If it is Linux, then report it to Ubuntu.

My first thought was to do this within cpanm, but I thought sudo su then strace -o platform -ff strace cpanm YAML::XS was a bit much. In part because when platform was Linux, it generated one file and hundreds on Windows.

Then it struck me that instead, I should just focus on the tests themselves. I cloned Ingy döt Net's YAML repo and tried to run prove test/ in both places. Went fine with Linux, failed with Windows. Butrealized after a second, it succeeded while using my personal perl, not system perl. /usr/bin/prove test/ failed on Ubuntu. apt-get install libtest-base-perl on both systems helped a lot, but now it wants Test::More, (I know because I searched for what modules the tests are asking for.)

For all I know, there's a package that provides Test::More, but it isn't libtest-more-perl, and digging further into that seems like a waste.

So I'm thinking it through again, looking at a failing test in YAML::XS:

use t::TestYAMLTests tests => 2;
use utf8;

is Dump("1234567890\n1234567890\n1234567890\n"), "--- |
  1234567890
  1234567890
  1234567890
", 'Literal Scalar';

is Dump("A\nB\nC\n"), q{--- "A\nB\nC\n"} . "\n", 'Double Quoted Scalar';

By "failing test" I am saying it works in natural Linux but not Ubuntu on Windows. And it's striking me: I need to find Dump. Where is Dump? In the C. It is an XS module, is it not? So, it's striking me that the solution is in C.

Which means I have to write C.

More later.

I think there's only been one time when I coded C on the clock, and only one time when my C knowledge was required on the clock.

The former was at a former workplace, where I wrote and compiled some C to compare UltraEdit with another editor, so I could help decide which we were going to buy a site license for. As I can only remember UltraEdit, I can say that's the one I liked better. The code itself was scarcely better than Hello World.

The latter was at another former workplace, where there was a tool that allowed mechanical engineers to drag together components like traces, and then first turned those traces into C code and compiled them. There was an issue where it wouldn't work, and I found the error logs and worked back.

I'm looking at perl_libyaml.c. I'm looking at perl_libyaml.h. I don't have nearly enough C skills to start here.

/*
 * This is the main Dump function.
 * Take zero or more Perl objects and return a YAML stream (as a string)
 */
void
Dump(SV *dummy, ...)
{
    dXSARGS;
    perl_yaml_dumper_t dumper;
    yaml_event_t event_stream_start;
    yaml_event_t event_stream_end;
    int i;
    SV *yaml = sv_2mortal(newSVpvn("", 0));
    sp = mark;

    set_dumper_options(&dumper);

    /* Set up the emitter object and begin emitting */
    yaml_emitter_initialize(&dumper.emitter);
    yaml_emitter_set_unicode(&dumper.emitter, 1);
    yaml_emitter_set_width(&dumper.emitter, 2);
    yaml_emitter_set_output(
        &dumper.emitter,
        &append_output,
        (void *) yaml
    );
    yaml_stream_start_event_initialize(
        &event_stream_start,
        YAML_UTF8_ENCODING
    );
    yaml_emitter_emit(&dumper.emitter, &event_stream_start);

    dumper.anchors = newHV();
    dumper.shadows = newHV();

    sv_2mortal((SV *)dumper.anchors);
    sv_2mortal((SV *)dumper.shadows);

    for (i = 0; i < items; i++) {
        dumper.anchor = 0;

        dump_prewalk(&dumper, ST(i));
        dump_document(&dumper, ST(i));

        hv_clear(dumper.anchors);
        hv_clear(dumper.shadows);
    }

    /* End emitting and destroy the emitter object */
    yaml_stream_end_event_initialize(&event_stream_end);
    yaml_emitter_emit(&dumper.emitter, &event_stream_end);
    yaml_emitter_delete(&dumper.emitter);

    /* Put the YAML stream scalar on the XS output stack */
    if (yaml) {
        SvUTF8_off(yaml);
        XPUSHs(yaml);
    }
    PUTBACK;
}

Hrm. Maybe back to the Perl.

Looking at YAML.pm, YAML::Dumper.pm and YAML::Dumper::Base.pm make it seem like you could make a master class on how modules go together. Previously, when I've dived in to figure out issues, yeah, there's lots of confusing parts but I could find my way to the broken thing. perl_libyaml.c is looking much more approachable now.