Testing Perl Modules on Windows: A Cry For Help

I've been working on modules again, after a recent push, and I found a big project whose .travis.yml file only went to Perl 5.20. I thought I'd get some dev brownie points by adding the next two stable versions, and found that my build was dying in the Pull Request submission process.

Specifically, it was dying on the checks. It passes Travis-CI, which runs the tests on Unix-like systems, but it was failing with Appveyor.

"What is Appveyor?", I asked myself.

Perhaps this isn't a direct quote.

Appveyor is a continuous integration service that tests against Windows systems. Scott Hanselman wrote a glowing review of it three years ago.

But there's no .appveyor.yml file in the project, so it runs and fails.

I've mentioned this on the project IRC, and no, I'm not going to name names, because there is movement toward testing on Windows, and even if it doesn't work, I admire the goal.

I wrote this three years ago, in response to a Python conference video:

2) Sure, real programmers use Unix/Linux to run their code, real programmers, but beginner programmers don't come in knowing how to set up an environment and come in with the (Windows) computer they have, and the documentation sucks and they feel lost and they don't like it and they don't feel the power, and they're gone. Even if you dislike Microsoft, good tools for Windows and good documentation are important for new programmers, important for building community, important for inclusiveness.

I run Windows, but I program on and for Linux, and only have one project based on Windows. But I have installed ActiveState and Strawberry Perls, and think that if you write general-purpose modules, you should be sure to test against Windows as well as Linux.

But, Travis-CI has documentation covering Perl projects. Appveyor says you can use Perl 5.20. eserte wrote a post on Appveyor for blogs.perl.org last year, but I'd love to see better documentation from either them, the Perl community or both. Following is the YAML from eserte, with a switch to "check only master branch", but as with Travis, which uses perlbrew and allows testing as far back as 5.8.8, I think having it test against older versions of Perl, both ActivePerl and Strawberry, would be the thing.

branches:
  only:
    - master

skip_tags: true

cache:
  - C:\strawberry



install:

  - if not exist "C:\strawberry" cinst strawberryperl

  - set PATH=C:\strawberry\perl\bin;C:\strawberry\perl\site\bin;C:\strawberry\c\bin;%PATH%

  - cd C:\projects\%APPVEYOR_PROJECT_NAME%

  - cpanm --installdeps .



build_script:

  - perl Makefile.PL

  - dmake test

If you have a more fully-featured .appveyor.yml file you'd like the Perl community to use, especially distinguishing MakeMaker modules from Dist::Zilla modules, I'd love to see it.

Contact Me! (If you REALLY need to)

A recent comment from Schlomi Fish said:

Hi! I cannot seem to find any contact information on this page. How should I contact you?

And then linked to a FAQ entry explaining his position on the state of email, comparing the futility of hiding addresses and the benefits of being open.

I have to say, I hadn't thought about this in ... years? In general, I'm active on Twitter (@jacobydave), which is good if you are, but not helpful if you aren't. I try to keep track of the comments, but that doesn't fit every message a person would want to send me.

So, a friendly "Hey, you should put your email on your blog" comment makes sense to me.

But, adding more traffic to the mailbox that friends and relatives have access to doesn't. I'm happy to put up an email address, but I'm less than happy to make it my main email address. It goes to context; my coworkers generally don't get that one either.

I had a long, barely touched by me but used enough by others project that was R syntax highlighting in Komodo Edit, which is dead because it's now native, but the ActiveState packaging used an email address to set the id, so, I created rlangsyntax@gmail.com.

So, to the right, in a section called "More Of Me", there is a requested mailto: pointing to rlangsyntax@gmail.com. I will check it. Use it in good health.

Coffee and Code and Calendars and R

I have what you might call a conflicted relationship with caffeine.



Long story short: I found it necessary to cut down on caffeine, limiting myself to two cups a day, preferrably before noon, and, as sort of a measure of accountability, and as a way to gain more skill with SQL and R, I wrote tools that store when I drink coffee and, at the end of the work day, tweeting the image.

I forget exactly where I found the calendar heatmap code, but it was several years ago, and was one of the first instances of ggplot2 that I found and put into service. I chose a brown-to-black color scheme because, well, obviously it needed to look like coffee.

This image, with "Dave has had n cups of coffee today" is autotweeted every day at 6pm Eastern. Recently, it has drawn interest.

@hoelzro It's very cargo-cult on my end. I should blog or gist it.

Thank @hadleywickham , I just put data into it.

— 'Dev' in Creole (@JacobyDave) May 6, 2017

So here it is, both blogged and gisted.


I started doing that thing with YAML in Perl, to keep database keys out of programs. I'm reasonably okay with my SQL skills, I think, but I am clear that my R code is largely cargo-cult. It'd be good to replace 2,4,6 with the days of the week,  and I am reasonably sure that it's reversed from the way you'd normally expect weekdays to go. Some day, I'll have the knowledge required to make those changes.

If you have questions and comments about this, I'd be glad to take them on, but I'm very much the learner when it comes to R.

count is not uniq...

I recall reading something online about 20 years ago (gasp!) where the authors were looking for a core set of knowledge that would constitute "knowing Unix", and found that there just wasn't. Knowing Unix was like the Humpty Dance, in that no two people do it the same.

And, presumably, you have Unix down when you appear to be in pain.

I have been a Unix/Linux user since the 1990s and I only found out about uniq -c because of the last post. I had been using sort | uniq forever, and have recently stopped in favor of sort -u, which I also learned about recently.

I find that uniq is kinda useless without a sort in front of it; if your input is "foo foo foo bar foo" (with requisite newlines, of course), uniq without sort will give you "foo bar foo" instead of "foo bar" or "bar foo", either of which is closer to what I want.

So, I could see adding alias count=" sort | uniq " to my bash setup, but adding a count program to my ~/bin seems much better to me, much closer to the Right Thing.

marc chantreux suggested an implementation of count that is perhaps better and certainly shorter than the one I posted. There was regex magic that I simply didn't need, because I wanted the count to stand by itself (but I might revisit to remove the awk step, because as a user, I'm still a bit awkward with it.)

B)

my %seen ;

map { $seen{$_}++ } do {
    @ARGV ? @ARGV : map { chomp ; $_ } <>;
    } ;

while ( my ( $k, $v ) = each %seen ) {
    say join "\t", $v, $k ;
    }

I like marc's use of the ternary operator to handle STDIN vs @ARGV, but I'm somewhat inconsistently against map over for. I know people who thing that map not going into an array is a problem, so I don't go back to it often.

I do, however, do for my $k ( keys %seen ) { ... } enough that I'm sort of mad at myself for not encountering each before.

ETA: It's been brought to my attention that using map {} as a replacement for for () {} is not good.

One! One New Utility! Bwa-ha-hahaha!

Classic unix utilities give you a number of great tools, and you can use sed and awk and bash when those aren't enough.

But sometimes ...

I use ~ as a scratch space all too often, which leaves me with a huge amount of files that I stopped playing with a while ago. I can get to the point of knowing what types, sure, as I show here.

$ ls *.* | awk -F. '{ print $NF }' 
jpg
jpg
jpg
jpg
txt
txt
txt
pl
txt
txt
pl
txt
pl
pl
txt
html
pl
pl
gz
mp3
pl
pl
pl
pl
txt
pl
pl
sh
sh
txt
pl
pl
diff
txt
txt
txt
pl
txt
pl
txt
txt
txt
txt
py
...

But this only gets you so far. I can sort and know that there's a LOT of Perl files, perhaps too many, but nothing was immediate about telling me how many.

But hey, I am a programmer, so I wrote a solution.

And here it is in a shell, combined with sort in order to give me the numbers, which includes a lot of throwaway Perl programs.

$ ls *.* | awk -F. '{ print $NF }' | count | sort -nr

95 pl
59 txt
10 sh
10 jpg
8 py
6 html
6 csv
5 js
2 gz
2 diff
1 zip
1 ttf
1 tt
1 svg
1 sql
1 Rd
1 R
1 pub
1 png
1 pdf
1 mp4
1 mp3
1 log
1 json
1 foo
1 conf
1 cnf

I suppose I need to do some cleanup in $HOME today...