I was procrastinating about a work project, thinking about magnets, thinking about crushers, thinking about thermite, when I thought about my status tracker.
It goes by the name status.pl because status is already a DBus utility. I made some other decisions relating to command-line interface, moving away from a cool idea I had a few years ago that hasn't aged well. I changed it to record.pl, which isn't similarly overloaded on my system, at least, and wrote a few flags for it, mostly for overkill. I mean, at core, it's because Len Budney convinced me my scripts needed manners, especially it not doing anything when just called.
So, I wrote this. All the details are tucked in Status.pm, which I am not sharing and which needs an overhaul too. All in all, I am reasonably happy with this; If there was a useless use of sub signatures award, I think this'd be a contender, but both that and postderef are used in get_records.pl, so it's okay. (I put no warnings because I'm running 5.26 on my desktop, but it is likely that I'll copy this to cluster machines running our 5.20 or the system 5.10 Perls, which would need the warnings.) I could and probably should go without IO::Interactive, but still.
So, my question is, beyond the user documentation that gets passed through Pod::Usage, what documentation does this program need? I'm always at a loss for examples of what is needed. In general, programmers discount documentation; most editor themes I see make comments gray and muted, which has always angered me, but looking at this (and yes, it's a slight case) I'm at a loss to decide what would be important to add.
So, what comments would you add? What do you find wanting?
(If desired, to add to this conversation, I can add the display section and the module.)
I can write a tool that goes through all old tweets and deletes ones that don't pass criteria, but I prefer to get out ahead of the issue and leave the record as it is.
And, as a later pass, one can pull a corpus, use a module like Algorithm::NaiveBayes and make your own classifier, rather than using Microsoft Research's Text Analytics API or another service. I was somewhere along that process when the hosting died, so I'm not bringing it here.
I was kinda under the thrall of Building Maintainable Software, or at least the first few chapters of it, so I did a few things differently in order to get their functions less than 15 lines, but the send_tweet function didn't get the passes it'd need, and I could probably give check_status some love, or perhaps even roll it into something like WebService::Microsoft::TextAnalytics. In the mean time, this should allow you to always tweet on the bright side of life.
#!/usr/bin/env perl
use feature qw{ postderef say signatures } ;
use strict ;
use warnings ;
use utf8 ;
no warnings qw{ experimental::postderef experimental::signatures } ;
use Carp ;
use Getopt::Long ;
use IO::Interactive qw{ interactive } ;
use JSON ;
use LWP::UserAgent ;
use Net::Twitter ;
use YAML qw{ LoadFile } ;
my $options = options() ;
my $config = config() ;
if ( check_status( $options->{ status }, $config ) >= 0.5 ) {
send_tweet( $options, $config ) ;
}
else { say qq{Blocked due to negative vibes, man.} }
exit ;
sub send_tweet ( $options, $config ) {
my $twit = Net::Twitter->new(
traits => [ qw/API::RESTv1_1/ ],
consumer_key => $config->{ twitter }{ consumer_key },
consumer_secret => $config->{ twitter }{ consumer_secret },
ssl => 1,
) ;
my $tokens = $config->{ twitter }{ tokens }{ $options->{ username } } ;
if ( $tokens->{ access_token }
&& $tokens->{ access_token_secret } ) {
$twit->access_token( $tokens->{ access_token } ) ;
$twit->access_token_secret( $tokens->{ access_token_secret } ) ;
}
if ( $twit->authorized ) {
if ( $twit->update( $options->{ status } ) ) {
say { interactive } $options->{ status } ;
}
else {
say { interactive } 'FAILED TO TWEET' ;
}
}
else {
croak( "Not Authorized" ) ;
}
}
sub check_status ( $status, $config ) {
my $j = JSON->new->canonical->pretty ;
my $key = $config->{ microsoft }{ text_analytics }{ key } ;
my $id = 'tweet_' . time ;
my $object ;
push @{ $object->{ documents } },
{
language => 'EN',
text => $status,
id => $id,
} ;
my $json = $j->encode( $object ) ;
my $api = 'https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment' ;
my $agent = LWP::UserAgent->new ;
my $request = HTTP::Request->new( POST => $api ) ;
$request->header( 'Ocp-Apim-Subscription-Key' => $key ) ;
$request->content( $json ) ;
my $response = $agent->request( $request ) ;
if ( $response->is_success ) {
my $out = decode_json $response->content ;
my $doc = $out->{ documents }[ 0 ] ;
return $doc->{ score } ;
}
else {
croak( $response->status_line ) ;
}
return 1 ;
}
sub config () {
my $config ;
$config->{ twitter } = LoadFile( join '/', $ENV{ HOME }, '.twitter.cnf' ) ;
$config->{ microsoft } = LoadFile( join '/', $ENV{ HOME }, '.microsoft.yml' ) ;
return $config ;
}
sub options () {
my $options ;
GetOptions(
'help' => \$options->{ help },
'username=s' => \$options->{ username },
'status=s' => \$options->{ status },
) ;
show_usage( $options ) ;
return $options ;
}
sub show_usage ($options) {
if ( $options->{ help }
|| !$options->{ username }
|| !$options->{ status } ) {
say { interactive } <<'HELP';
Only Positive Tweets -- Does text analysis of content before tweeting
-u user Twitter screen name (required)
-s status Status to be tweeted (required)
-h help This screen
HELP
exit ;
}
}
__DATA__
.microsoft.yml looks like this
---
text_analytics:
key: GENERATED_BY_MICROSOFT
.twitter.cnf looks like this
---
consumer_key: GO_TO_DEV.TWITTER.COM
consumer_secret: GO_TO_DEV.TWITTER.COM
tokens:
your_username:
access_token: TIED_TO_YOU_AS_USER_NOT_DEV
access_token_secret:TIED_TO_YOU_AS_USER_NOT_DEV
I cover access_tokens in https://varlogrant.blogspot.com/2016/07/nettwitter-cookbook-how-to-tweet.html
Early in my playing with Bootstrap, I made this as a way to begin to play with it. It is about as simple a GitHub API to LWP to Template Toolkit to Bootstrap tool as I could have written. I'm now thinking about how to make this prettier, but for now, it's what I use to generate, when I remember to redo my GitHub page. Learn and enjoy.
I'm always curious about how people customize their prompt. I put name and machine in, with color-coding based on which machine, because while spend most of my time on one or two hosts, I have reason to go to several others.
I work in a sub-basement, and for most of my work days, I couldn't tell you if it was summer and warm, winter and frozen, or spring and waterlogged outside, so one of the things I learned to check out is current weather information. I used to put the temperature on the front panel of our HP printer, but we've moved to Xerox.
Currently, I use DarkSky, formerly forecast.io. I know my meteorologist friends would recommend more primary sources, but I've always found it easy to work with.
I had this code talking to Redis, but I decided that it was an excuse to use Redis and this data was better suited for storing in YAML, so I rewrote it.
store_temp
#!/usr/bin/env perl
# stores current temperature with YAML so that get_temp.pl can be used
# in the bash prompt to display current temperature
use feature qw{ say state } ;
use strict ;
use warnings ;
use Carp ;
use Data::Dumper ;
use DateTime ;
use IO::Interactive qw{ interactive } ;
use JSON ;
use LWP::UserAgent ;
use YAML::XS qw{ DumpFile LoadFile } ;
my $config = config() ;
my $url
= 'https://api.darksky.net/forecast/'
. $config->{apikey} . '/'
. ( join ',', map { $config->{$_} } qw{ latitude longitude } ) ;
my $agent = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 } ) ;
my $response = $agent->get($url) ;
if ( $response->is_success ) {
my $now = DateTime->now()->set_time_zone('America/New_York')->datetime() ;
my $content = $response->content ;
my $forecast = decode_json $content ;
my $current = $forecast->{currently} ;
my $temp_f = int $current->{temperature} ;
store( $now, $temp_f ) ;
}
else {
say $response->status_line ;
}
exit ;
sub store {
my ( $time, $temp ) = @_ ;
say {interactive} qq{Current Time: $time} ;
say {interactive} qq{Current Temperature: $temp} ;
my $data_file = $ENV{HOME} . '/.temp.yaml' ;
my $obj = {
curr_time => $time,
curr_temp => $temp,
} ;
DumpFile( $data_file, $obj ) ;
}
# ======================================================================
# Reads configuration data from YAML file. Dies if no valid config file
# if no other value is given, it will choose current
#
# Shows I need to put this into a module
sub config {
my $config_file = $ENV{HOME} . '/.forecast.yaml' ;
my $output = {} ;
if ( defined $config_file && -f $config_file ) {
my $output = LoadFile($config_file) ;
$output->{current} = 1 ;
return $output ;
}
croak('No Config File') ;
}
And this is the code that reads the YAML and prints it, nice and short and ready to be called in PS1.
get_temp
#!/usr/bin/env perl
# retrieves the current temperature from YAML to be used in the bash prompt
use feature qw{ say state unicode_eval unicode_strings } ;
use strict ;
use warnings ;
use utf8 ;
binmode STDOUT, ':utf8' ;
use Carp ;
use Data::Dumper ;
use YAML::XS qw{ LoadFile } ;
my $data_file = $ENV{HOME} . '/.temp.yaml' ;
my $output = {} ;
if ( defined $data_file && -f $data_file ) {
my $output = LoadFile($data_file) ;
print $output->{curr_temp} . '°F' || '' ;
exit ;
}
croak('No Temperature File') ;
I thought I put a date-diff in there. I wanted to be able say 'Old Data' if the update time was too long ago. I should change that.
I should really put the config files in __DATA__ for show, because it will show that the location is hard-coded. For a desktop or server, that makes sense; it can only go as far as the power plug stretches. But, for other reasons, I adapted my bash prompt on my Linux laptop, and I recently took it to another state, so I'm thinking more and more that I need to add a step, to look up where I am before I check the temperature.
store_geo_temp
#!/usr/bin/env perl
# Determines current location based on IP address using Google
# Geolocation, finds current temperature via the DarkSky API
# and stores it into a YAML file, so that get_temp.pl can be
# in the bash prompt to display current local temperature.
use feature qw{ say state } ;
use strict ;
use warnings ;
use utf8 ;
use Carp ;
use Data::Dumper ;
use DateTime ;
use IO::Interactive qw{ interactive } ;
use JSON::XS ;
use YAML::XS qw{ DumpFile LoadFile } ;
use lib $ENV{HOME} . '/lib' ;
use GoogleGeo ;
my $json = JSON::XS->new->pretty->canonical ;
my $config = config() ;
my $location = geolocate( $config->{geolocate} ) ;
croak 'No Location Data' unless $location->{lat} ;
my $forecast = get_forecast( $config, $location ) ;
croak 'No Location Data' unless $forecast->{currently} ;
say {interactive} $json->encode($location) ;
say {interactive} $json->encode($forecast) ;
my $now = DateTime->now()->set_time_zone('America/New_York')->datetime() ;
my $current = $forecast->{currently} ;
my $temp_f = int $current->{temperature} ;
store( $now, $temp_f ) ;
exit ;
# ======================================================================
# Reads configuration data from YAML files. Dies if no valid config files
sub config {
my $geofile = $ENV{HOME} . '/.googlegeo.yaml' ;
croak 'no Geolocation config' unless -f $geofile ;
my $keys = LoadFile($geofile) ;
my $forecastfile = $ENV{HOME} . '/.forecast.yaml' ;
croak 'no forecast config' unless -f $forecastfile ;
my $fkeys = LoadFile($forecastfile) ;
$keys->{forecast} = $fkeys->{apikey} ;
croak 'No forecast key' unless $keys->{forecast} ;
croak 'No forecast key' unless $keys->{geolocate} ;
return $keys ;
}
# ======================================================================
# Takes the config for the API keys and the location, giving us lat and lng
# returns the forecast object or an empty hash if failing
sub get_forecast {
my ( $config, $location ) = @_ ;
my $url
= 'https://api.darksky.net/forecast/'
. $config->{forecast} . '/'
. ( join ',', map { $location->{$_} } qw{ lat lng } ) ;
my $agent = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 } ) ;
my $response = $agent->get($url) ;
if ( $response->is_success ) {
my $content = $response->content ;
my $forecast = decode_json $content ;
return $forecast ;
}
return {} ;
}
sub store {
my ( $time, $temp ) = @_ ;
say {interactive} qq{Current Time: $time} ;
say {interactive} qq{Current Temperature: $temp} ;
my $data_file = $ENV{HOME} . '/.temp.yaml' ;
my $obj = {
curr_time => $time,
curr_temp => $temp,
} ;
DumpFile( $data_file, $obj ) ;
}
A few things I want to point out here. First off, you could write this with Getopt::Long and explicit quiet and verbose flags, but Perl and IO::Interactive allow me to make this context-specific and implicit. If I run it myself, interactively, I am trying to diagnose issues, and that's when say {interactive} works. If I run it in crontab, then it runs silently, and I don't get an inbox filled with false negatives from crontab. This corresponds to my personal preferences; If I was to release this to CPAN, I would likely make these things controlled by flags, and perhaps allow latitude, longitude and perhaps API keys to be put in that way.
But, of course, you should not get in the habit, because then your keys show up in the process table. It's okay if you're the only user, but not best practice.
This is the part that's interesting. I need to make it better/strong/faster/cooler before I put it on CPAN, maybe something like Google::Geolocation or the like. Will have to read some existing Google-pointing modules on MetaCPAN before committing. Geo::Google looks promising, but it doesn't do much with "Where am I now?" work, which is exactly what I need here.
Google's Geolocation API works better when you can point to access points and cell towers, but that's diving deeper than I need; the weather will be more-or-less the same across the widest accuracy variation I could expect.
GoogleGeo
package GoogleGeo ;
# interfaces with Google Geolcation API
# https://developers.google.com/maps/documentation/geolocation/intro
use feature qw{say} ;
use strict ;
use warnings ;
use Carp ;
use Data::Dumper ;
use Exporter qw(import) ;
use Getopt::Long ;
use JSON::XS ;
use LWP::Protocol::https ;
use LWP::UserAgent ;
our @EXPORT = qw{
geocode
geolocate
} ;
my $json = JSON::XS->new->pretty ;
my $agent = LWP::UserAgent->new ;
sub geocode {
my ($Google_API_key,$obj) = @_ ;
croak unless defined $Google_API_key ;
my $url = 'https://maps.googleapis.com/maps/api/geocode/json?key='
. $Google_API_key ;
my $latlng = join ',', $obj->{lat}, $obj->{lng} ;
$url .= '&latlng=' . $latlng ;
my $object = { latlng => $latlng } ;
my $r = $agent->post($url) ;
if ( $r->is_success ) {
my $j = $r->content ;
my $o = $json->decode($j) ;
return $o ;
}
return {} ;
}
sub geolocate {
my ($Google_API_key) = @_ ;
my $url = 'https://www.googleapis.com/geolocation/v1/geolocate?key='
. $Google_API_key ;
my $object = {} ;
my $r = $agent->post( $url, $object ) ;
if ( $r->is_success ) {
my $j = $r->content ;
my $o = $json->decode($j) ;
return {
lat => $o->{location}{lat},
lng => $o->{location}{lng},
acc => $o->{accuracy},
} ;
}
return {} ;
}
'here' ;
If this has been helpful or interesting to you, please tell me so in the comments.
A Delorean with a Perl-powered center column and the cutest little Flux Capacitor on the dashboard. Oh, the wonders you can see at a developer conference.
"A man's got to know his limitations."
That's a line from Dirty Harry Callahan in Magnum Force, but it really described my planning for the Perl Conference. Once the calendar was up, I went in, first and foremost thinking "What are skills I need to learn?"
One crucial skill is version control. It's difficult to add to my main workflow, as I develop in production. (I live in fear.) But I'm increasingly adding it to my side projects. It is especially part of the process for maintaining the site for Purdue Perl Mongers, as well as aspects of HackLafayette, but beyond certain basics, I just didn't know much about how to use Git and version control to improve my projects. I learned how CPAN Testers tests your code on many platforms after you upload to CPAN, and how Travis-CL and Appveyor test against Linux, macOS and Windows after pushing to GitHub, but how track changes, align issues with branches, etc., are all new to me. So, I started Wednesday with Genehack and Logs Are Magic: Why Git Workflows and Commit Structure Should Matter To You. (Slides) I fully expect to crib ideas and aliases from this talk for some time to come.
There was a talk at a local software group featuring a project leader from Amazon on the technology involved with Alexa, which involved a lot of how this works, going from speech-to-text to tokenization and identifying of crucial keywords -- "Alexa, have Domino's order me a Pizza" ultimately boiling down to "Domino's" and "Pizza" -- and proceeding from there. It gave a sense of how Amazon is taking several hard problems and turning them into consumer tools.
What came very late in the talk is how to interface my systems with Amazon's "Talking Donkey", and I had a few conversations where we talked about starting the day with "Alexa, what's fresh hell is this?" and getting back a list of systems that broke overnight, but I lacked a strong idea of what is needed to make my systems interact with the Alexa system.
But, thankfully, Jason Terry's Amazon, Alexa and Perl talk covered this, albeit more in the "Turn my smart lights on" sense than in the "Tell me what my work systems are doing" sense. Still, very much something I had been interested in.
But, as I implied, Amazon does a lot of heavy lifting with Alexa, getting it down to GETs and and POSTs against your API. If you're running this from home, where you have a consistent wired connection, this works. But, if you're running this, for example, in your car, you need it to be small, easy, and self-contained. Chris Prather decided to MAKE New Friends with a Raspberry Pi 3. This was a case of Conference-Driven Development, and he didn't have it ready to demonstrate at this time.
I've been trying to move my lab to the New Hotness over time, and because I will have to tie things back to existing systems, I have avoided learning too much in order to make it work. Joel Berger presented Vue.js, Mojolicious, and PostgreSQL chat in less than 50 lines. I've heard good things about Vue.js, which I heard from the project head was the parts he needed from Angular without the stuff he didn't use. (RFC Podcast) I use jQuery and Template and not much more, so the "only what you need" aspect sounds good to me. I fully expect to bug Joel on Twitter and IRC about this and other things I need to do with Mojolicious over the coming months. (Slides)
But, of course, not every talk needs to speak directly to my short-term needs. Stevan Little has been trying to add an object system to the Perl 5 for quite some time, and in Hold My Beer and Watch This, he talks about Moxie, his latest attempt.
OK, the Moxie talk was in the same room as the Mojolicious talk and this next one. I could have gone to one of the others, but I decided against. Oh well. I would put "do more object-oriented development" as a thing to learn, so I was glad to hear it.
The last full talk was The Variable Crimes We Commit Against JavaScript by Julka Grodel. I would say that I code Javascript half as much as I code Perl, but twice as much as I code in other languages. I knew certain parts, like how let gives lexical scoping like Perl's my. I had heard about let from my JS Twitter list, as well as from Matt Trout's talk, but there were certainly things I didn't know.
And, actually, things I still don't. I had technical difficulties with my laptop, and if I could've worked it out that day, I would've tried to set up a lightning talk. Alas, not only did it not work -- in the end, I swapped out the WiFi, and I think if I switched back, it'd be sane again -- I missed a lot of her talk about arrow functions, which piqued the interest of Perl's functional programming guru, Mark Jason Dominus. (In a not-surprising addition to my limitations, I still haven't gotten to the end of Higher Order Perl.) Anyway, I believe that, after I finish this post, I will go back and watch this again.
After this, there were Lightning Talks, interspersed with ads for various Perl Mongers groups, and urges for you to start a Perl Mongers group. I am likely going to do another post, with these talks from all three days, but to end this one, I'll show off one where M. Allan Noah talks about the SANE Project and how he reverse engineers scanner output to allow him to support scanners without vendor input.
Lack of documentation is a limitation, but knowing the limitation does not mean you have to stop, and lack of documentation will not stop him. We should all draw inspiration from that.
Now that I'm two weeks past the end, most of the talks are up, and I would like to commend the organizers. Conference organization is hard, and this venue made it harder. The US Patent and Trademark Office is a great place, but there are aspects that they as a venue wanted secured and I would've preferred to be more open, but it was a beautiful venue and I be glad to return.
But, the thing I'd like to commend them most on is the high quality and fast turnaround on talk videos. The audio is good, the video is clear and well-lit, and the slides take precedence over the speaker in framing. It's everything I want in a conference video.
The beginning of the third day, my laptop hit a weird issue where it couldn't enable the wireless NIC. I fixed it at home, once I had access to precision screwdrivers again, but this meant that I couldn't blog the last day, either.
But waiting affords me the opportunity to add videos of the talks I attended. So, yay.
I started Day 2 with Sam Batschelet's Dancing In The Cloud, which was not what I was expecting, being more a deep dive in another domain. I was hoping for an introduction to using the Dancer framework on cloud providers. It was interesting, and I now know what kubernetes does and that etcd exists, but this is far enough away from where I live that I don't know if it'll give me much more than buzzword recognition when I next attend a technical conference.
Again, I would like to reiterate that it was deep for me. Good talk, good speaker, but I was the wrong audience. Read the talk descriptions, not just the titles!
This was followed by Chad Granum on Test2, which is not yet on YouTube. I came to TPC wanting to up my testing game, and I've been looking through Andy Lester's modules and brian d foy's Stack Overflow posts trying to get into the mindset. This talk is a long-form walk through the diffs between the old and new test frameworks, and by showing how to test using Test2, it showed a lot about how to test in general. Once the talk is up, I am sure I will go through it again and again, trying to get my head around it.
But just because we don't have this talk on YouTube, that doesn't mean we don't have Chad talking about testing.
My next talk was Adventures in Failure: Error handling culture across languages by Andrew Grangaard, which is another point of growth for me. All too often, I just let errors fall, so knowing how they're commonly done is a good thing, well presented.
Damian Conway was the keynote speaker, and his talk was called Three Little Words. Those are, of course, I ❤ Perl.
Also, class has method. Conway proceeded to implement Perl6-style OOP with Dios, or Declarative Inside-Out Syntax, which needed Keyword::Declare to enable the creation of new keywords in Perl 5. To get this going, he needed to more quickly parse Perl, so along comes PPR, the Pattern-based Perl Recognizer. It was in this talk that the previous Lightning Talk from Chad, including his module Test2::Plugin::Source, comes from.
The Lightning Talks were good, but the one that comes up as a good one to put here is Len Budney's Manners for Perl Scripts. This makes me want to look into CLI::Startup.
I hope to get to Day 3 and maybe another post just on Lightning Talks soon. An hour's dive into topics won't get you too deep, but an hours' lightning talks will will open up a number of possibilities for you to follow.
A few lessons learned before I dive into the talks I attended and what I gained from them:
I had figured that aspects of my Every Day Carry would not be appreciated in a government building, so I left my multitool off my belt, but, even after I had put everything I could think of into the tray, I had still made the machines go beep, causing the guards to pull out the wand. Tomorrow, pack lighter.
I take notes on a paper notebook, but I used my phone and tablet to connect to others via IRC and Twitter, as well as keeping track of my calendar, and I ran through my batteries before the end of the talks. In part, it was switching between the local WiFi and the cellular network, doing well with neither, but I'm not convinced it'd wouldn't be drained regardless. At lunch, I need to find a place to charge. I often come with a power strip for just this purpose, too.
I didn't break out the laptop once. If I don't use it more, I should just leave it and have less to carry.
Hopefully, I will remember this and come prepared for the next one.
Now, to the talks:
I started the day with MetaCPAN, the Grand Tour. I had been convinced earlier that MetaCPAN is better than CPAN for looking into modules, but there's more, including modules I am going to have start using to search CPAN.
Graham Knop's Continuous Integration for CPAN followed. I had been aware of Travis-CI, and had become aware of AppVeyor recently, but the tooling available in Perl to work with these was less familiar to me. I was unaware that you can specify Linux or OSX in Travis, This was something I was thinking and asking questions of other developers about. I have issues on FreeBSD, which I'm told is something that Gitlab-CI can help me with, but somehow, I doubt I can connect Github to Gitlab, but I could be wrong.
Steven Lembark had much more with Dockerizing CPAN Testers: Running an Isolated Test Site than I could fit into my head, and I think I'll have to go back to the tape once it's available, but I think it's a useful addition to the world.
After lunch, I went to Joel Berger's Variables, Scoping and Namespaces, which he set as a talk for beginners. He went so far as to suggest more established developers go elsewhere. Since I never thought I learned all of Perl when I was learning, it was very much a lot of things I did already know, but a little "So that's why we do that", some more "Oooh. I forgot about that", and one weird trick that explains how to mock functions for tests.
(That, fundamentally, is my big item to work on as a developer. Testing.)
After this, I attended Matt S. Trout's ES6: Almost an acceptable Perl 5? and it gave me a sense of treating Javascript like Perl, but since I don't code Perl like Matt does, I probably won't code ES6 like Matt does. My notes peter out about halfway through, but they do include several gems, such as lodash, that might improve the code I do write.
Following this is Lightning Talks, which has a bunch of interesting points, going from "Write for OpenSource.com" to "Try learning Dart" to "Creating Test JSON from JSON Schemas" to "Using Instapaper, IFTTT and Buffer to tweet what you read" to the Alien modules, which I almost understand now. Or maybe not. Certainly, though, I'd be installing and trying Dave Rolsky's RTx-toGitHub right now if I wasn't so tired.
Finally, Sawyer X talked about Perl 5.26 and the changes that came and the changes that are coming. The thing that comes to mind first is that things that have been deprecated since 5.0 are finally being pulled. I understand the critics who think that removing . from @INC is insignificant, but I am still for it. I also like that Perl recognizes unhandled merges and dies now.
Tomorrow, I will be learning about Dancer, Test2 and more with @INC, and visiting with family after
I've been working on modules again, after a recent push, and I found a big project whose .travis.yml file only went to Perl 5.20. I thought I'd get some dev brownie points by adding the next two stable versions, and found that my build was dying in the Pull Request submission process.
Specifically, it was dying on the checks. It passes Travis-CI, which runs the tests on Unix-like systems, but it was failing with Appveyor.
But there's no .appveyor.yml file in the project, so it runs and fails.
I've mentioned this on the project IRC, and no, I'm not going to name names, because there is movement toward testing on Windows, and even if it doesn't work, I admire the goal.
I wrote this three years ago, in response to a Python conference video:
2) Sure, real programmers use Unix/Linux to run their code, real programmers, but beginner programmers don't come in knowing how to set up an environment and come in with the (Windows) computer they have, and the documentation sucks and they feel lost and they don't like it and they don't feel the power, and they're gone. Even if you dislike Microsoft, good tools for Windows and good documentation are important for new programmers, important for building community, important for inclusiveness.
I run Windows, but I program on and for Linux, and only have one project based on Windows. But I have installed ActiveState and Strawberry Perls, and think that if you write general-purpose modules, you should be sure to test against Windows as well as Linux.
branches:
only:
- master
skip_tags: true
cache:
- C:\strawberry
install:
- if not exist "C:\strawberry" cinst strawberryperl
- set PATH=C:\strawberry\perl\bin;C:\strawberry\perl\site\bin;C:\strawberry\c\bin;%PATH%
- cd C:\projects\%APPVEYOR_PROJECT_NAME%
- cpanm --installdeps .
build_script:
- perl Makefile.PL
- dmake test
If you have a more fully-featured .appveyor.yml file you'd like the Perl community to use, especially distinguishing MakeMaker modules from Dist::Zilla modules, I'd love to see it.
I recall reading something online about 20 years ago (gasp!) where the authors were looking for a core set of knowledge that would constitute "knowing Unix", and found that there just wasn't. Knowing Unix was like the Humpty Dance, in that no two people do it the same.
And, presumably, you have Unix down when you appear to be in pain.
I have been a Unix/Linux user since the 1990s and I only found out about uniq -c because of the last post. I had been using sort | uniq forever, and have recently stopped in favor of sort -u, which I also learned about recently.
I find that uniq is kinda useless without a sort in front of it; if your input is "foo foo foo bar foo" (with requisite newlines, of course), uniq without sort will give you "foo bar foo" instead of "foo bar" or "bar foo", either of which is closer to what I want.
So, I could see adding alias count=" sort | uniq " to my bash setup, but adding a count program to my ~/bin seems much better to me, much closer to the Right Thing.
marc chantreux suggested an implementation of count that is perhaps better and certainly shorter than the one I posted. There was regex magic that I simply didn't need, because I wanted the count to stand by itself (but I might revisit to remove the awk step, because as a user, I'm still a bit awkward with it.)
B)
my %seen ;
map { $seen{$_}++ } do {
@ARGV ? @ARGV : map { chomp ; $_ } <>;
} ;
while ( my ( $k, $v ) = each %seen ) {
say join "\t", $v, $k ;
}
I like marc's use of the ternary operator to handle STDIN vs @ARGV, but I'm somewhat inconsistently against map over for. I know people who thing that map not going into an array is a problem, so I don't go back to it often.
I do, however, do for my $k ( keys %seen ) { ... } enough that I'm sort of mad at myself for not encountering each before.
Classic unix utilities give you a number of great tools, and you can use sed and awk and bash when those aren't enough.
But sometimes ...
I use ~ as a scratch space all too often, which leaves me with a huge amount of files that I stopped playing with a while ago. I can get to the point of knowing what types, sure, as I show here.
But this only gets you so far. I can sort and know that there's a LOT of Perl files, perhaps too many, but nothing was immediate about telling me how many.
But hey, I am a programmer, so I wrote a solution.
And here it is in a shell, combined with sort in order to give me the numbers, which includes a lot of throwaway Perl programs.
I've been working with beefy virtual machines, docker containers, and build processes lately. Believe it or not, working on projects aimed at making Mozilla developers more productive can mean executing code that can take anywhere from a minute to an hour, which in itself can hit how productive I can be. For the longer tasks, I often get away from my desk, make a cup of coffee, and check in to see how the rest of the Walsh clan is doing.
When I walk away, however, it would be nice to know when the task is done, so I can jet back to my desk and get back to work. My awesome Mozilla colleague Byron "glob" Jones recently showed me his script for task completion notification and I forced him to put it up on GitHub so you all can get it too; it's called ding!
OK, that sounds cool. So I go to Github and I see one line that gives me pause.
Requires ding.mp3 and error.mp3 in same directory as script. OSX only.
I can handle the image thing, but I don't own or run an OSX computer. (I have one somewhere, but it's ancient and has no functioning battery. I don't use it.)
"So," I think, "how could I do this on my Linux box? What's the shortest path toward functionality on this concept?"
Well, recently, I have been playing with Text-to-Speech. Actually, I have been a long-time user of TTS, using festival then espeak to tell me the current time and temperature on the hour and half-hour. I switched to Amazon's Polly in December, deciding that the service sounded much better than the on-my-computer choices. (Hear for yourself.) So, I knew how to handle the audio aspects.
The other part required me to get much more familiar with Perl's system function than I had been previously.
I'm not yet 100% happy with this code, but I'm reasonably okay with it so far. Certainly the concept has been proven. (I use the audio files from globau's ding.) With enough interest, I will switch it from being a GitHub gist to being a repo.
As with most things I blog about, this starts with Twitter. I follow a lot of people on Twitter, and I use Lists. I want to be able to group people more-or-less on community, because there's the community where they talk about programming, for example, and the community where they talk about music, or the town I live in.
I can begin to break things up myself, but curation is a hard thing, so I wanted to do it automatically. And I spent a long time not knowing what to do. I imagined myself traversing trees in what looks like linked lists reimagined by Cthulhu, and that doesn't sound like much fun at all.
Eventually, I decided to search on "graphs and Perl". Of course, I probably should've done it earlier, but oh well. I found Graph. I had used GD::Graph before, which is a plotting library. (There has to be some index of how overloaded words are.) And once I installed it, I figured it out: As a programmer, all you're dealing with are arrays and hashes. Nothing scary.
Word Ladder
We'll take a problem invented by Lewis Carroll called a "word ladder", where you find your way from one word (for example, "cold") to another ("warm") by changing one letter at a time:
cold coRd cArd Ward warM
Clearly, this can and is often done by hand, but if you're looking to automate it, there are three basic problems: what are the available words, how do you determine when words are one change away, and how do you do this to get the provable shortest path?
First, I went to CERIAS years ago and downloaded word lists. Computer security researchers use them because real words are bad passwords, so, lists of real words can be used to create rainbow tables and the like. My lists are years old, so there may be new words I don't account for, but unlike Lewis Carroll, I can get from APE to MAN in five words, not six.
ape apS aAs Mas maN
Not sure that Lewis Carroll would've accepted AAS, but there you go
There is a term for the number of changes it takes to go from one word to another, and it's called the Levenshtein Distance. I first learned about this from perlbrew, which is how, if you type "perlbrew isntall", it guesses that you meant to type "perlbrew install". It's hardcoded there because perlbrew can't assume you have anything but perl and core modules. I use the function from perlbrew instead of Text::Levenshtein but it is a module worth looking into.
#!/usr/bin/env perl
use feature qw{say} ;
use strict ;
use warnings ;
use Data::Dumper ;
use Graph ;
use List::Util qw{min} ;
use Storable ;
for my $l ( 3 .. 16 ) {
create_word_graph($l) ;
}
exit ;
# -------------------------------------------------------------------
# we're creating a word graph of all words that are of length $length
# where the nodes are all words and the edges are unweighted, because
# they're all weighted 1. No connection between "foo" and "bar" because
# the distance is "3".
sub create_word_graph {
my $length = shift ;
my %dict = get_words($length) ;
my @dict = sort keys %dict ; # sorting probably is unnecessary
my $g = Graph->new() ;
# compare each word to each word. If the distance is 1, put it
# into the graph. This implementation is O(N**2) but probably
# could be redone as O(NlogN), but I didn't care to.
for my $i ( @dict ) {
for my $j ( @dict ) {
my $dist = editdist( $i, $j ) ;
if ( $dist == 1 ) {
$g->add_edge( $i, $j ) ;
}
}
}
# Because I'm using Storable to store the Graph object for use
# later, I only use this once. But, I found there's an endian
# issue if you try to open Linux-generated Storable files in
# Strawberry Perl.
store $g , "/home/jacoby/.word_$length.store" ;
}
# -------------------------------------------------------------------
# this is where we get the words and only get words of the correct
# length. I have a number of dictionary files, and I put them in
# a hash to de-duplicate them.
sub get_words {
my $length = shift ;
my %output ;
for my $d ( glob( '/home/jacoby/bin/Toys/Dict/*' ) ) {
if ( open my $fh, '<', $d ) {
for my $l ( <$fh> ) {
chomp $l ;
$l =~ s/\s//g ;
next if length $l != $length ;
next if $l =~ /\W/ ;
next if $l =~ /\d/ ;
$output{ uc $l }++ ;
}
}
}
return %output ;
}
# -------------------------------------------------------------------
# straight copy of Wikipedia's "Levenshtein Distance", straight taken
# from perlbrew. If I didn't have this, I'd probably use
# Text::Levenshtein.
sub editdist {
my ( $f, $g ) = @_ ;
my @a = split //, $f ;
my @b = split //, $g ;
# There is an extra row and column in the matrix. This is the
# distance from the empty string to a substring of the target.
my @d ;
$d[ $_ ][ 0 ] = $_ for ( 0 .. @a ) ;
$d[ 0 ][ $_ ] = $_ for ( 0 .. @b ) ;
for my $i ( 1 .. @a ) {
for my $j ( 1 .. @b ) {
$d[ $i ][ $j ] = (
$a[ $i - 1 ] eq $b[ $j - 1 ]
? $d[ $i - 1 ][ $j - 1 ]
: 1 + min( $d[ $i - 1 ][ $j ], $d[ $i ][ $j - 1 ], $d[ $i - 1 ][ $j - 1 ] )
) ;
}
}
return $d[ @a ][ @b ] ;
}
Following are what my wordlists can do. Something tells me that, when we get to 16-letter words, it's more a bunch of disconnected nodes than a graph.
1718 3-letter words 6404 4-letter words 13409 5-letter words 20490 6-letter words 24483 7-letter words 24295 8-letter words 19594 9-letter words 13781 10-letter words 8792 11-letter words 5622 12-letter words 3349 13-letter words 1851 14-letter words 999 15-letter words 514 16-letter words
My solver isn't perfect, and the first thing I'd want to add is ensuring that both the starting and ending words are actually in the word list. Without that, your code goes on forever.
So, I won't show off the whole program below, but it does use Storable, Graph and feature qw{say}.
dijkstra( $graph , 'foo' , 'bar' ) ;
# -------------------------------------------------------------------
# context-specific perl implementation of Dijkstra's Algorithm for
# shortest-path
sub dijkstra {
my ( $graph, $source, $target, ) = @_ ;
# the graph pre-exists and is passed in
# $source is 'foo', the word we're starting from
# $target is 'bar', the word we're trying to get to
my @q ; # will be the list of all words
my %dist ; # distance from source. $dist{$source} will be zero
my %prev ; # this holds our work being every edge of the tree
# we're pulling from the graph.
# we set the the distance for every node to basically infinite, then
# for the starting point to zero
for my $v ( $graph->unique_vertices ) {
$dist{$v} = 1_000_000_000 ; # per Wikipeia, infinity
push @q, $v ;
}
$dist{$source} = 0 ;
LOOP: while (@q) {
# resort, putting words with short distances first
# first pass being $source , LONG WAY AWAY
@q = sort { $dist{$a} <=> $dist{$b} } @q ;
my $u = shift @q ;
# say STDERR join "\t", $u, $dist{$u} ;
# here, we end the first time we see the target.
# we COULD get a list of every path that's the shortest length,
# but that's not what we're doing here
last LOOP if $u eq $target ;
# this is a complex and unreadable way of ensuring that
# we're only getting edges that contain $u, which is the
# word we're working on right now
for my $e (
grep {
my @a = @$_ ;
grep {/^${u}$/} @a
} $graph->unique_edges
) {
# $v is the word on the other end of the edge
# $w is the distance, which is 1 because of the problem
# $alt is the new distance between $source and $v,
# replacing the absurdly high number set before
my ($v) = grep { $_ ne $u } @$e ;
my $w = 1 ;
my $alt = $dist{$u} + $w ;
if ( $alt < $dist{$v} ) {
$dist{$v} = $alt ;
$prev{$v} = $u ;
}
}
}
my @nodes = $graph->unique_vertices ;
my @edges = $graph->unique_edges ;
return {
distances => \%dist,
previous => \%prev,
nodes => \@nodes,
edges => \@edges,
} ;
}
I return lots of stuff, but the part really necessary is %prev, because that, $source and $target are everything you need. Assuming we're trying to go from FOR to FAR, a number of words will satisfy $prev{FOR}, but it's the one we're wanting. In the expanded case of FOO to BAR, $prev->{BAR} = 'FAR', $prev->{FAR} is 'FOR', and $prev->{FOR} is 'FOO'.
And nothing in there is complex. It's all really hashes or arrays or values. Nothing a programmer should have any problem with.
CPAN has a number of other modules of use: Graph::Dijkstra has that algorithm already written, and Graph::D3 allows you to create a graph in such a way that you can use it in D3.js. Plus, there are a number of modules in Algorithm::* that do good and useful things. So go in, start playing with it. It's deep, there are weeds, but it isn't scary.
This started while driving to work. If I get mail from coworkers, I get Pushover notifications, and halfway from home, I got a bunch of notifications.
We don't know the cause of the issue, but I do know the result
We have env set on our web server set so that Perl is /our/own/private/bin/perl and not /usr/bin/perl, because this runs in a highly-networked and highly-clustered environment, mostly RedHat 6.8 and with 5.10.1 as system perl, if we want to have a consistent version and consistent modules, we need our own. This allows us to have #!/usr/bin/env perl as our shbang.
And this morning, for reasons I don't know, it stopped working. Whatever perl was being called, it wasn't /our/own/private/bin/perl. And this broke things.
One of the things that broke is this: Whatever perl is /usr/bin/env perl, it doesn't have Modern::Perl.
I'm for Modern Perl. My personal take is that chromatic and Modern Perl kept Perl alive in with Perl 5 while Larry Wall and the other language developers worked on Perl 6. Thus, I am grateful that it exists. But, while I was playing with it, I found a problem: Modern::Perl is not in Core, so you cannot rely on it being there, so a script might be running with a version greater than 5.8.8 and be able to give you everything you need, which to me is normally use strict, use warnings and use feature qw{say}, but if you're asking for Modern::Perl for it, it fails, and because you don't know which Modern thing you want, you don't know how to fix it.
This is part of my persistent hatred of magic. If it works and you don't understand how, you can't fix it if it stops working. I got to the heavy magic parts of Ruby and Rails and that, as well as "Life Happens", are why I stopped playing with it. And, I think, this is a contributing factor with this morning's brokenness.
I've been looking at command-line code for both fun and work. I know I can have one module handle just the interface, and have the module where the functionality happens pass the functionality along.
#!/usr/bin/env perl
use feature qw'say state' ;
use strict ;
use warnings ;
use utf8 ;
my $w = Wit->new( @ARGV ) ;
$w->run() ;
package Wit ;
use lib '/home/jacoby/lib' ;
use base 'Witter' ;
use Witter::Twitter ;
1;
package Witter ;
# highly adapted from perlbrew.
use feature qw{ say } ;
use strict ;
use warnings ;
sub new {
my ( $class, @argv ) = @_ ;
my $self ;
$self->{foo} = 'bar' ;
$self->{args} = [] ;
if (@argv) {
$self->{args} = \@argv ;
}
return bless $self, $class ;
}
sub run {
my ($self) = @_ ;
$self->run_command( $self->{args} ) ;
}
sub run_command {
my ( $self, $args ) = @_ ;
if ( scalar @$args == 0
|| lc $args->[0] eq 'help'
|| $self->{help} ) {
$self->help(@$args) ;
exit ;
}
if ( lc $args->[0] eq 'commands' ) {
say join "\n\t", '', $self->commands() ;
exit ;
}
my $command = $args->[0] ;
my $s = $self->can("twitter_$command") ;
unless ($s) {
$command =~ y/-/_/ ;
$s = $self->can("twitter_$command") ;
}
unless ($s) {
my @commands = $self->find_similar_commands($command) ;
if ( @commands > 1 ) {
@commands = map { ' ' . $_ } @commands ;
die
"Unknown command: `$command`. Did you mean one of the following?\n"
. join( "\n", @commands )
. "\n" ;
}
elsif ( @commands == 1 ) {
die "Unknown command: `$command`. Did you mean `$commands[0]`?\n"
;
}
else {
die "Unknown command: `$command`. Typo?\n" ;
}
}
unless ( 'CODE' eq ref $s ) { say 'Not a valid command' ; exit ; }
$self->$s(@$args) ;
}
sub help {
my ($self,$me,@args) = @_ ;
say 'HELP!' ;
say join "\t", @args;
}
sub commands {
my ($self) = @_ ;
my @commands ;
my $package = ref $self ? ref $self : $self ;
my $symtable = do {
no strict 'refs' ;
\%{ $package . '::' } ;
} ;
foreach my $sym ( sort keys %$symtable ) {
if ( $sym =~ /^twitter_/ ) {
my $glob = $symtable->{$sym} ;
if ( defined *$glob{CODE} ) {
$sym =~ s/^twitter_// ;
$sym =~ s/_/-/g ;
push @commands, $sym ;
}
}
}
return @commands ;
}
# Some functions removed for sake of brevity
package Witter::Twitter ;
use strict ;
use feature qw{ say state } ;
use warnings FATAL => 'all' ;
use Exporter qw{import} ;
use Net::Twitter ;
use JSON::XS ;
our $VERSION = 0.1 ;
our @EXPORT ;
for my $entry ( keys %Witter::Twitter:: ) {
next if $entry !~ /^twitter_/mxs ;
push @EXPORT, $entry ;
}
sub twitter_foo {
my ( $self, @args ) = @_ ;
say "foo" ;
say join '|', @args ;
}
1 ;
In the above example, I'm just doing the one add-on module, Witter::Twitter and one function, Witter::Twitter::foo, but clearly, I would want it open-ended, so that if someone wanted to add Witter::Facebook, all the information about the Facebook functions would be in that module.
Then, of course, I would have to use another prefix than twitter_, but we'll leave that, and ensuring that modules don't step on each others' names, to another day.
The part that concerns me is help. Especially help foo. It should be part of the the module it's in; If Witter::Twitter is the module with foo(), only it should be expected to know about foo().
But how to communicate it? I'm flashing on our %docs and $docs{foo}= 'This is foo, silly' but the point of the whole thing is to allow the addition of modules that the initial module doesn't know about, and it would require knowing to look for %Witter::Twitter::docs.
I suppose adding a docs_ function that looks like this.
sub docs_foo {
return q{
This explains the use of the 'foo' command
...
};
END
}
I'm diving into this in part because I have code that uses basically this code, and I need to add functionality to it, and while I'm in there, I might as well make user documentation better. Or even possible.
I'm also parallel-inspired by looking at a Perl project built on and using old Perl ("require 5.005") and recent blog posts about Linus Torvalds and "Good Taste". There's something tasteful about being able to add use Your::Module and nothing else to code, but if the best I can do is knowledge that there's a foo command, with no sense of what it does, that seems like the kind of thing that Linus would rightfully curse me for.
Is there a better way of doing things like this? I recall there being interesting things in Net::Tumblr that would require me to step up and learn Moose or the like. This is yet another important step toward me becoming a better and more tasteful programmer, but not germane to today's ranting.
I had planned to get into following, lists and blocking, but a wild hair came over me and I decided I wanted to make a tweetstorm app.
What's a tweetstorm?
And this is the inner loop of it.
my @tweetstorm_text ; # previously generated. URL shortening and numbering are assumed
my $screen_name ; #your screen name as text
my $status_id ; #created but not set, so first tweet won't be a reply-to
for my $status ( @tweetstorm_text } ) {
my $tweet ; # the object containing the twee
$tweet->{status} = $status ;
$tweet->{in_reply_to_status_id} = $status_id if $status_id ;
if ( $twit->update($tweet) ) { #if Tweet is successful
# we get your profile, which contains your most recent tweet
# $status_id gets set to the ID of that tweet, which then
# gets used for the next tweet.
my $profiles
= $twit->lookup_users( { screen_name => $config->{user} } ) ;
my $profile = shift @$profiles ;
my $prev = $profile->{status} ;
$status_id = $profile->{status}->{id} ;
}
}
Write a minimal test case, run the test case with strace on Ubuntu on Windows and Ubuntu on Linux. The outputs are supposed be identical, if they aren't (and we expect them not to be because of the error), then you have a good shot at identifying the misbehaving syscall (which is the only difference between Ubuntu on Windows and Ubuntu on Linux).
Once you see the difference, look into what the syscall does and try to determine which system is implementing it correctly (probably Linux). If it is Windows post to the github tracker with the test case and the identified syscall. If it is Linux, then report it to Ubuntu.
My first thought was to do this within cpanm, but I thought sudo su then strace -o platform -ff strace cpanm YAML::XS was a bit much. In part because when platform was Linux, it generated one file and hundreds on Windows.
Then it struck me that instead, I should just focus on the tests themselves. I cloned Ingy döt Net's YAML repo and tried to run prove test/ in both places. Went fine with Linux, failed with Windows. Butrealized after a second, it succeeded while using my personal perl, not system perl. /usr/bin/prove test/ failed on Ubuntu. apt-get install libtest-base-perl on both systems helped a lot, but now it wants Test::More, (I know because I searched for what modules the tests are asking for.)
For all I know, there's a package that provides Test::More, but it isn't libtest-more-perl, and digging further into that seems like a waste.
use t::TestYAMLTests tests => 2;
use utf8;
is Dump("1234567890\n1234567890\n1234567890\n"), "--- |
1234567890
1234567890
1234567890
", 'Literal Scalar';
is Dump("A\nB\nC\n"), q{--- "A\nB\nC\n"} . "\n", 'Double Quoted Scalar';
By "failing test" I am saying it works in natural Linux but not Ubuntu on Windows. And it's striking me: I need to find Dump. Where is Dump? In the C. It is an XS module, is it not? So, it's striking me that the solution is in C.
Which means I have to write C.
More later.
I think there's only been one time when I coded C on the clock, and only one time when my C knowledge was required on the clock.
The former was at a former workplace, where I wrote and compiled some C to compare UltraEdit with another editor, so I could help decide which we were going to buy a site license for. As I can only remember UltraEdit, I can say that's the one I liked better. The code itself was scarcely better than Hello World.
The latter was at another former workplace, where there was a tool that allowed mechanical engineers to drag together components like traces, and then first turned those traces into C code and compiled them. There was an issue where it wouldn't work, and I found the error logs and worked back.
I'm looking at perl_libyaml.c. I'm looking at perl_libyaml.h. I don't have nearly enough C skills to start here.
/*
* This is the main Dump function.
* Take zero or more Perl objects and return a YAML stream (as a string)
*/
void
Dump(SV *dummy, ...)
{
dXSARGS;
perl_yaml_dumper_t dumper;
yaml_event_t event_stream_start;
yaml_event_t event_stream_end;
int i;
SV *yaml = sv_2mortal(newSVpvn("", 0));
sp = mark;
set_dumper_options(&dumper);
/* Set up the emitter object and begin emitting */
yaml_emitter_initialize(&dumper.emitter);
yaml_emitter_set_unicode(&dumper.emitter, 1);
yaml_emitter_set_width(&dumper.emitter, 2);
yaml_emitter_set_output(
&dumper.emitter,
&append_output,
(void *) yaml
);
yaml_stream_start_event_initialize(
&event_stream_start,
YAML_UTF8_ENCODING
);
yaml_emitter_emit(&dumper.emitter, &event_stream_start);
dumper.anchors = newHV();
dumper.shadows = newHV();
sv_2mortal((SV *)dumper.anchors);
sv_2mortal((SV *)dumper.shadows);
for (i = 0; i < items; i++) {
dumper.anchor = 0;
dump_prewalk(&dumper, ST(i));
dump_document(&dumper, ST(i));
hv_clear(dumper.anchors);
hv_clear(dumper.shadows);
}
/* End emitting and destroy the emitter object */
yaml_stream_end_event_initialize(&event_stream_end);
yaml_emitter_emit(&dumper.emitter, &event_stream_end);
yaml_emitter_delete(&dumper.emitter);
/* Put the YAML stream scalar on the XS output stack */
if (yaml) {
SvUTF8_off(yaml);
XPUSHs(yaml);
}
PUTBACK;
}
Hrm. Maybe back to the Perl.
Looking at YAML.pm, YAML::Dumper.pm and YAML::Dumper::Base.pm make it seem like you could make a master class on how modules go together. Previously, when I've dived in to figure out issues, yeah, there's lots of confusing parts but I could find my way to the broken thing. perl_libyaml.c is looking much more approachable now.
My first pass, on a very old computer, had the C version taking a second and a half, the Perl5 version taking a minute and a half, and the Perl6 version taking an hour and a half. But I no longer use that computer, and am now using a much newer one, so I took the opportunity to try again.
I have Nvidia cards in my box, so I could eventually write Overkill IV about that, but I don't have the knowledge or time right now. I did install rakudobrew and the newest version of Perl6, but otherwise, everything was unchanged.
C dropped to 0.6 seconds. Perl5 now takes 16 seconds. Perl6 now takes 10 minutes. So, the newer computer brought substantial speed gains for each, but the most dramatic is with Perl6, which tells us that the Perl6 developers have not only worked on correctness and completeness, but speed. Good for them! Good for us!
But wait! There's more! @loltimo saw the post and suggested that I use permutations.
"What are permutations?", you ask? That's what basically happens in recursive_magic_box(), where all the possible variations are created. Given A, B and C, there are six possible orderings: ABC ACB BAC BCA CAB CBA. If I was given all possible orderings of an array, rather than (mis)handling it on my own and having to check that I got it right, would that make things faster?
As it turns out, yes. 47 seconds, rather than 10 minutes. More than 10x faster. But still, not as fast as Perl5. I now want to implement permutations for Perl5 and see if it makes that one faster.
I promised a longer post on Bash on Ubuntu on Windows 10, explaining why I use it, when I don't and why I even use Windows 10, but that isn't today.
Today is me trying to figure out who to report a bug to, and how to effectively report it.
The system (called "Ubuntu on Windows" for the rest of this blog post) is Ubuntu 14.04 LTS running kinda with Windows as the kernel instead of Linux. A good thing is that, if you can apt-get install a package, you can put it on Ubuntu on Windows. I don't think you can use it if 1) it connects to the kernel (because the kernel is Windows) or 2) it pops up a window. I've heard about people using XMing to get around that, but I haven't tried it yet.
The problem is that you get what you have available with dpkg, and the perl that comes with 14.04 is 5.18, where current is 5.24. Some modules haven't been packaged, and there's usually a gap between the package version and the cpan version, so I tend to use perlbrew and use my own perl instead of system perl.
And cpan and cpanm can't build on Ubuntu on Windows. I demonstrated this by trying to install YAML::XS with cpanm. I won't include the build log, but I will link to the build log. The problem comes at lines 146, 162 and 207:
# Failed test 'Literal Scalar'
# at t/dump-heuristics.t line 4.
# got: '--- '1234567890
#
# 1234567890
#
# 1234567890
#
# '
# '
# expected: '--- |
# 1234567890
# 1234567890
# 1234567890
# '
# Failed test 'Double Quoted Scalar'
# at t/dump-heuristics.t line 10.
# got: '--- 'A
#
# B
#
# C
#
# '
# '
# expected: '--- "A\nB\nC\n"
# '
# Looks like you failed 2 tests of 2.
t/dump-heuristics.t ......
...
# Failed test 'Dumping Chinese hash works'
# at t/utf8.t line 31.
# got: '---
# Email: boss@opcafe.net
# 地址: 新竹市 300 石坊街 37-8 號
# 店名: OpCafé
# 時間: 11:01~23:59
# 電話: '03-5277806
#
# 0991-100087
#
# '
# '
# expected: '---
# Email: boss@opcafe.net
# 地址: 新竹市 300 石坊街 37-8 號
# 店名: OpCafé
# 時間: 11:01~23:59
# 電話: "03-5277806\n0991-100087\n"
# '
# Looks like you failed 1 test of 8.
t/utf8.t .................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/8 subtests
The problem involves incorrectly handling newlines, but all this works and YAML::XS installs perfectly on my personal Perl, which I demonstrate below.
$ which perl ; perl -v | head -n 5 ; perl -MYAML::XS -e "print 'Yet Another Perl Hacker'"
/home/jacoby/perl5/perlbrew/perls/perl-5.24.0/bin/perl
This is perl 5, version 24, subversion 0 (v5.24.0) built for i686-linux
(with 1 registered patch, see perl -V for more detail)
Copyright 1987-2016, Larry Wall
Yet Another Perl Hacker
So, if it's Perl on Ubuntu on Windows, and the Perl works perfectly other places (including Strawberry Perl on the same Windows 10 machine (but only in PowerShell) ) and the Ubuntu works perfectly on other places, it must be the Windows, right?
That's what I thought, too. There's a GitHub repo for Ubuntu on Windows. Well, really, it's an issue tracker for Ubuntu on Windows, because there's nothing else in there. You can't clone and build your own. But, it does put the issue tracker in a place where the natural audience for Ubuntu on Windows would go. I put in an issue, but I think I might have asked wrong. I have cleaned up my issue some, but I see no sign that anyone else has seen it.
So, what is my next step? It isn't just YAML::XS; that's just a module I wanted at the time. It isn't just cpanm; I tried with cpan, too. It is just with Windows 10. Thoughts and suggestions from the greater Perl community?
I've covered how you handle keys and authenticating with Twitter
previously, so look into those for more information as we go ahead
with sending tweets!
There was a time when Twitter was just text, so you could just
send a link to an image. There were sites like TwitPix that hosted
images for you, but eventually Twitter developed the ability to
host media.
my $media ;
push @$media, $config->{ file } ;
push @$media, 'icon.' . $suffix ;
if ( $twit->update_with_media( $status , $media ) ) {
say 'OK' ;
}
I use the second in order to specify the file name that isn't whatever
long and random filename it has. The module has ways to guess the
correct mime type, but you can specify to avoid that. The last one,
starting with undef, allows you to create or modify an image with
Image::Magick or whatever you choose and tweet it without having to
involve the filesystem at all.
A few words about what we mean by media. We mean images or video; no
audio file option. By images, we mean PNG, JPG, GIF or WEBP. No SVG,
and less that 5MB as of this writing. (Check https://dev.twitter.com/
for more info later.)
Video has other constraints, but I don't tweet just video often, if
ever. I generally put it to YouTube and post the link.
There's a couple ways you can have more fun with images on Twitter.
Those would be with your profile image, the square image that sits
next to your tweet, and the profile banner, the landscape image that
shows up at the top of your page.
They work like the above, handling media the same way.
my $image ;
push @$image, $config->{ file } ;
push @$image, 'icon.' . $suffix ;
if ( $twit->update_profile_image( $image ) ) {
say 'OK' ;
}
my $banner ;
push @$banner, $config->{ file } ;
push @$banner, 'icon.' . $suffix ;
if ( $twit->update_profile_banner( $banner ) ) {
say 'OK' ;
}
Profile images are square, and are converted to square if your initial
image is not square. For both, JPG, GIF or PNG are allowed and are
converted upon upload. If you try to upload an animated GIF, they will
use the first frame. This can make your profile image less fun, but if
you have a feed full of pictures that throb and spin, that can make
you feel seasick.
And, since we're hitting profile-related issues, perhaps I should
handle your profiles and we can get this behind us.
Changing your username, going from @jacobydave to something else, is
doable but not via the API. It isn't a thing you can or should be
doing with an API. You can change other details, however. You can
change your name ("Dave Jacoby"), description ("Guy blogging about
Perl's Net::Twitter library. Thought Leader. Foodie."), website
("http://varlogrant.blogspot.com/") and location ("Dark Side of the
Moon").
use Getopt::Long ;
my $config ;
GetOptions(
'description=s' => \$config->{ description },
'location=s' => \$config->{ location },
'name=s' => \$config->{ name },
'web=s' => \$config->{ url },
) ;
my $params ; for my $p ( qw{ name url location description } ) {
$params->{ $p } = $config->{ $p } if $config->{ $p } ; }
$params->{ include_entities } = 0 ; $params->{ skip_status } = 1
;
if ( $twit->update_profile( $params ) ) {
say 'OK' ;
}
}
I *DO* have fun fun with location, setting it with a number of
strings; "My Happy Place" on Saturday, "Never got the hang..." for
Thursdays. You can set this to any string, and Twitter does nearly
nothing with it. I'd suggest you ignore it or set it to something
general or clever, and forget it.
Now that we've covered how you tweet and handle your identity a little, next time we'll start to get into relationships, which are the things that make a social network social.
Below is the code that I use most often. It is a command-line tool, where it's used something along the lines of named_twitter.pl jacobydave text of my tweet. Except, that's not how I type it, thanks to my .alias file. I have twitter aliased to '~/bin/named_twitter jacobydave ' and normally tweet like twitter this is how I tweet.
This isn't to say I never automate tweets; I certainly do. But it is a rare part of what I do with the Twitter API and Net::Twitter. I will dive deeper into issues with tweeting and direct messages, both in a technical and social matter, in a later post.
But I said that you have the consumer key and secret, which identify you as a service, and the access token and secret, which identify you as a user of the service. In the above code sample, I use YAML to store this data, but you could use JSON, Sqlite or anything else to store it. Among other benefits, you can put your code into GitHub, as I did above, without exposing anything.
As I proceed, I will assume that tokens are handled somehow and proceed directly to the cool part.
Again, you get the consumer key by going to apps.twitter.com and creating a new app.
You log into apps.twitter.com with your Twitter account, click "Create New App" and fill in the details. When I created my three, you had to specifically choose "can send DMs" specifically, so if you're creating apps to follow along, I do suggest you allow yourself that option.
