A Little Fun with OpenCV

I have a webcam that I have set to take pictures at work. Is this a long-term art project? Is it a classic Twitter over-share? An example of Quantified Self gone horribly, horribly wrong? I honestly don't know.

I do know that it's brought me into interesting places before. Getting my camera to work with V4L. Tweeting images. Determining if the screen is locked, both on Unity and Gnome. I've come to the point where I can set @hourly ~/bin/tweet_a_picture.pl. I don't, rather 0 9-18 * * 1-5 ~/bin/tweet_a_picture.pl, because if it's before 9am or on a weekend, I won't be here, so don't even try.

But if I'm pulled away, there are photos like the above. Who needs to see my space without me?

Yeah, who needs to see my space with me? We'll pass on that.

One of the members of Purdue Perl Mongers formerly worked with OpenCV, and I asked him to present on that topic this month. He even created a GitHub repo with his example code. It's Python, not Perl, which will be a digression for this blog, but not an unprecedented one. This gave me enough confidence in OpenCV and how it worked to create a program that uses it to tell if anyone's in the frame.

(The demo code suggested installing python-opencv for Ubuntu, to which I must add that the crucial opencv-data does not come with it. Just a warning.)

# webcam_here.py

# usage: webcam_here.py
# 1

# when runs, returns the number of upper bodies, and thus people, it currently sees

import cv2
# pass either device id (0 usually for webcam) or path to a video file
cam = cv2.VideoCapture(0)

# face cascade bundled with OpenCV
# classifier = cv2.CascadeClassifier('/usr/share/opencv/haarcascades/haarcascade_frontalface_default.xml')
classifier = cv2.CascadeClassifier('/usr/share/opencv/haarcascades/haarcascade_upperbody.xml')
# I found that haarcascade_upperbody did the best for identifying me.

# read in frame
retval, frame = cam.read()

# convert to grayscale
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# run classifier
results = classifier.detectMultiScale(
 frame_gray,
 # let's assume face will be close/big if in front of webcam
 minSize=(300,300)
)

# number of found upper bodies in webcam
print len(results)

And here it is in context.

# tweet_a_picture.pl
use 5.010 ;
use strict ;
use warnings ;
use IO::Interactive qw{ interactive } ;
use Net::Twitter ;
use YAML::XS qw{ DumpFile LoadFile } ;
use Getopt::Long ;

use lib '/home/jacoby/lib' ;
use Locked qw{ is_locked there };

my $override = 0 ;

GetOptions(
    'override' => \$override ,
    ) ;

my $config_file = $ENV{ HOME } . '/.twitter.cnf' ;
my $config      = LoadFile( $config_file ) ;

my $user = 'screen_name' ;
my $status = 'Just took a picture' ;

if ( is_locked() == 0 && there() == 1 || $override ) {
    say { interactive } 'unlocked' ;
    # highest supported resolution of my old-school webcam 
    # Twitter DOES have a 5MB limit on file size
    # https://dev.twitter.com/rest/media/uploading-media#imagerecs
    my $cam = q{/usr/bin/fswebcam -q -r 640x480 --jpeg 85 -D 1 --no-banner} ;
    my $time = time ;
    my $image = "/home/jacoby/Pictures/Self/shot-${time}.jpeg" ;
    qx{$cam $image} ;
    tweet_pic( $status , $image ) ;
    say { interactive } $time ;
    }

sub tweet_pic {
    my ( $status , $pic ) = @_ ;
    my ( $access_token, $access_token_secret ) ;
    ( $access_token, $access_token_secret ) = restore_tokens( $user ) ;
    my $twit = Net::Twitter->new(
        traits              => [qw/API::RESTv1_1/],
        consumer_key    => $config->{ consumer_key },
        consumer_secret => $config->{ consumer_secret },
        ssl => 1 ,
        ) ;

    if ( $access_token && $access_token_secret ) {
        $twit->access_token( $access_token ) ;
        $twit->access_token_secret( $access_token_secret ) ;
        }
    unless ( $twit->authorized ) {
        say qq{UNAUTHTORIZED} ; exit ;
        # I really handle this better
        # http://varlogrant.blogspot.com/2016/07/nettwitter-cookbook-how-to-tweet.html
        }

    my $img_ref ;
    my $file = $pic ;
    my ( $filename ) = reverse split m{/} , $pic ;
    push @$img_ref , $file ;
    push @$img_ref , $filename ;

    if ( $twit->update_with_media( $status , $img_ref  ) ) {
        no warnings ;
        say { interactive } $status ;
        }
    else {
        say { interactive } 'FAIL' ;
        }
    }

# is in Locked.pm, but placed here for simplicity
# there are issues with recognizing my webcam, which 
# we quash by sending STDERR to /dev/null
sub there {
    my $checker = '/home/jacoby/bin/webcam_here.py' ;
    return -1 if ! -f $checker ;
    my $o = qx{$checker 2> /dev/null} ;
    chomp $o ;
    return $o ;
    }

sub restore_tokens {
    my ( $user ) = @_ ;
    my ( $access_token, $access_token_secret ) ;
    if ( $config->{ tokens }{ $user } ) {
        $access_token = $config->{ tokens }{ $user }{ access_token } ;
        $access_token_secret =
            $config->{ tokens }{ $user }{ access_token_secret } ;
        }
    return $access_token, $access_token_secret ;
    }

sub save_tokens {
    my ( $user, $access_token, $access_token_secret ) = @_ ;
    $config->{ tokens }{ $user }{ access_token }        = $access_token ;
    $config->{ tokens }{ $user }{ access_token_secret } = $access_token_secret ;
    DumpFile( $config_file, $config ) ;
    return 1 ;
    }

Clearly, this is the start, it's currently very cargo-cult code, and it has several uses. A coming use is to go through my archive of pictures and identifying the ones where I'm gone. That'll either require me getting up to speed with Image::ObjectDetect or Python, but it's a thing I'm willing to do.

An interesting side-issue: I've found that the existing Haar cascades (the XML files that define a thing OpenCV can identify) do not like my glasses, and thus cannot identify my eyes or face with them on, thus, me using the upperbody cascade. I think I should train my own Haar classifier; I know I have enough pics of me for it.