Taking the Great Leap Forward with Dancer2

I am working on understanding a raft of technologies, including Dancer and Bootstrap, in order to make our web presence look more current and, more importantly, be more maintainable. 

I'm learning a lot, which is not the positive statement that it sounds like. Rip Van Winkle certainly learned a lot after he slept for twenty years and woke up in post-Revolution America. 

For most of my time as a web developer, when I needed to do authentication, I did it with Apache's built-in server authentication. The number of users I needed to handle was always small enough, and except for a few things where it was entirely for me, I was not the person in charge of creating and maintaining the password system.

I know and believe in a few points. I know that I as admin of a system should not have access to the plain-text passwords of the users. I know that it is common to have two password fields when creating/changing passwords, to ensure you have the right spelling. I'm not 100% bought into that one, but I understand it. I know you keep an email address for "Forgot Password" systems can use your email system as a factor to ensure you're authorized to change that password. And I know you should use encryption systems created by experts, rather than roll your own and create a system that's full of holes. 

I've been using Dancer2::Plugin::Auth::Extensible, trying to get the parts I'd want for a generic system before working on things that I'd want for the lab, and there's much I'm comfortable with. I can get people logged in. I can set roles and limit access to users with specific roles. I can store the date of the last login, which might be useful. And it's all backed by MySQL, which means that, even without an admin dashboard, I have the skills to change anything about a user profile that needs changing.

But we don't want that. We want the techs and the users to have the ability to set values for the user, if for no other reason than I want to be able to move on to other things. So I need to figure out, as a standard, how these things go together, so I can try to implement it. I have things that I'm getting together. I do have questions, though.

• Clearly, lots consider the repeat-your-password thing as an important part of the password workflow, and clearly, this is a check that I need to at least be able to do. I'm seeing a huge task-duplication thing, because you want to be able to say "passwords don't match" on the client side before the user presses go, but you always want to check things on the server side before you click "submit", because the user might block Javascript. Is this something that Bootstrap can help with? Or will I have to write something like that? I'm willing and able, but with the layout stuff and the way of the future encouraging us to have CSS and JS that's combined and minified and gzipped and included on every page, I'd like to have that taken care of automatically by the framework than go custom.
• It's not immediately clear in the docs how to enable password encryption. I do need to read that more. (Solved. It was in the docs. I need to read the docs.)
• I'm hitting the concept of roles and finding that they'd make certain things very useful. I'd like to be able to handle things like unix groups instead, but as is, they allow certain things that will make the end result a lot easier.
  
  I found, however, that, while the tools to check and control access due to roles are solid, setting and removing roles is less so. I asked the Dancer2 IRC about it, and was told to make a Github issue. I did, and then I wrote something that, within context of my tooling, adds add_user_role and remove_user_role functions. So I have that covered and can move forward.

There's more than this. I could see us wanting a website that has static, CGI and Dancer2 paths, although I think that, when I wake up with a start at 3am, bathed in sweat with a racing heartbeat, this thought is what I was dreaming about. But I'll wait for a while before I have to worry about that.

And, with a parting shot from @perigrin.

Wondering how to proceed with my web application

I don't want to go too deep into the detail because what it actually does is very specific to the lab. There is a lot of data I'm wanting to collect, select, connect and commit to a database. Which is very bog-standard for my web work.

Thing is, it is a lot of state that I want to put together, and I hate putting together lots of state iteratively via repeated web inputs, because each time you go back to the server, you increase time and annoyance and the potential for catastrophe. So, I really want this to be an AJAX-lead mostly-javascript thing.

You want to know the absolute coolest thing about coding Javascript? You can put it all together via Javascript.  You can have a page that looks like <body></body> and have absolute control over everything on that page. It is true that, when you do that, this means that people who run without Javascript get absolutely nothing, and this is a problem, but there are times when you can create something in HTML with some small Javascript additions that are not necessary, and there are times when putting it together in Javascript is the whole of the point and re-implementing in HTML and CGI are just in the way.

Except....

We are experimenting with MVC frameworks, very specifically Dancer. We've seen and done enough with Dancer to decide that this is the way we want to go with new web development. In a way, I'm pushed, but in another way, I see this as accepting the change of technology. I don't want my skills to be rooted in the early 90s more than they already are. So, I for one welcome the new MVC overlords. But I'm seeing a Template Toolkit page that says <body> <!-- Insert Cool Stuff Here --> </body> which kinda obviates the purpose of making it dynamic like that in the first place.

Right now, I'm thinking through the data, deciding on the right way to make and modify the database tables to get at it, and mocking up what I want it to look like in straight HTML, but I am getting close to the point where I have to make that decision, and the way isn't clear to me.

More Gripes and Misunderstandings about Dancer

It has been suggested that I'm not using MVC separation, which is fair, as I don't really see MVC separation. I found the POD for Dancer::Plugin::Database, and have modified my package to use it. weather is the old one, weather2 is the new one using the plugin.

package TestApp ;
use Dancer ':syntax' ;
use Dancer::Plugin::Database ;
use lib '/home/jacoby/lib' ;
use MyDB 'db_connect' ;
use DateTime ;
use Data::Dumper ;

our $VERSION = '0.1' ;

get '/' => sub {
    template 'index' ;
    } ;
 
get '/weather' => sub {
    my $dbh = db_connect() ;
    my $sql = <<SQL;
    # Last Eight Hours
    SELECT  time , AVG( temp_f )
    FROM weather WHERE zip = "47909"
    AND HOUR( TIMEDIFF( SYSDATE() , time ) ) < 9
    GROUP BY HOUR(time)
    ORDER BY time
SQL
    my $hr = $dbh->selectall_arrayref( $sql ) or croak $dbh->errstr ;
    my @hr ;
    for my $point ( @$hr ) {
        my ( $time, $temp ) = @$point ;
        my @time = split /\D+/ ,  $time ;
        my $dt = DateTime->new(
            year  => $time[0] ,
            month => $time[1] ,
            day   => $time[2] ,
            hour => $time[3] ,
            minute => $time[4] ,
            second => $time[5] ,
            ) ;
        my $data ;
        $data->{ time } = $dt->format_cldr( 'h:mm a' ) ;
        $data->{ temp } = sprintf '%.1f', $temp ;
        next if $data->{ time } !~ /00/ ;
        push @hr, $data ;
        }
    my $string = 'Temperature in West Lafayette for the last 8 hours' ;
    my $hr_ptr = \@hr ;
    template 'weather' , {
        string => $string ,
        weather => $hr_ptr
        } ;
    } ;

get '/weather2' => sub {
    my $string = 'Temperature in West Lafayette for the last 8 hours' ;
    my $sth = database->prepare(
        'SELECT  time , AVG( temp_f ) temp
        FROM weather WHERE zip = "47909"
        AND HOUR( TIMEDIFF( SYSDATE() , time ) ) < 9
        GROUP BY HOUR(time)
        ORDER BY time '
        ) ;
    $sth->execute( ) ;
    template 'weather' , {
        string => $string ,
        xxx => $sth->fetchall_arrayref( ) ,
        } ;
    } ;

true ;

I have been pointed to http://search.cpan.org/~xsawyerx/Dancer-1.3091/lib/Dancer/Tutorial.pod, and, beyond my wanting to be more specific in the munging of the data, weather is very similar to "Our First Route Handler". (How I store some data is not necessarily how I want to display some data.)

weather2 uses http://search.cpan.org/~bigpresh/Dancer-Plugin-Database-1.60/lib/Dancer/Plugin/Database.pm and you can see that there's much less code in there. The stuff I put into  MyDB exists in Dancer::Plugin::Database, and I probably could mess with the data before sending it on, instead of hooking up $sth->fetchall_arrayref() to the output.

The big thing I see here, which I could do something about with weather but probably not with the plugin-using weather2, is the SQL code being here. I would much rather shove that SQL off into a library somewhere, which I've done elsewhere, than have it at this point. But maybe, that's an aspect of me not really getting the MVC thing yet.

I Can't Help Feeling Stupid Standing 'Round

We're jumping from straight Perl and CGI.pm for web applications to an MVC, a jump we've been wanting to do for years. I've tried Catalyst a couple times, and both times, I spend most of my learning curve thinking "If I wrote this up without an MVC, I'd be done and on to the next thing by now", and in general, I get told much the same thing by my bosses, in the form of "move on  to this other, non-MVC-related task".

But that was Catalyst, which seemingly brings along half of CPAN with it. This time, I'm trying Dancer with Template Toolkit, and I'm getting some progress. Here's a chunk of code I have working.

package TestApp ;
use Dancer ':syntax' ;
use lib '/home/jacoby/lib' ;
use MyDB 'db_connect' ;
use DateTime ;

our $VERSION = '0.1' ;

get '/' => sub {
    template 'index' ;
    } ;

get '/weather' => sub {
    my $dbh = db_connect() ;
    my $sql = <<SQL;
    # Last Eight Hours
    SELECT  time , AVG( temp_f )
    FROM weather WHERE zip = "47909"
    AND HOUR( TIMEDIFF( SYSDATE() , time ) ) < 9
    GROUP BY HOUR(time)
    ORDER BY time
SQL
    my $hr = $dbh->selectall_arrayref( $sql ) or croak $dbh->errstr ;
    my @hr ;
    for my $point ( @$hr ) {
        my ( $time, $temp ) = @$point ;
        my @time = split /\D+/ ,  $time ;
        my $dt = DateTime->new(
            year  => $time[0] ,
            month => $time[1] ,
            day   => $time[2] ,
            hour => $time[3] ,
            minute => $time[4] ,
            second => $time[5] ,
            ) ;
        my $data ;
        $data->{ time } = $dt->format_cldr( 'h:mm a' ) ;
        $data->{ temp } = sprintf '%.1f', $temp ;
        next if $data->{ time } !~ /00/ ;
        push @hr, $data ;
        }
    my $vars = {
        string => 'Temperature in West Lafayette for the last 8 hours' ,
        weather => \@hr ,
        } ;

    template 'weather' , { var => $vars } ;
    } ;

true ;

A few notes here: MyDB.pm is a module I use to get a DBI object without putting the DB connection info into the code, making it safer for me to dump it in public like this. I've been collecting weather data for the West Lafayette area so I have a significant set to use when I want to play around with SQL and R.

Normally, when I pass things around, I pass, for example, a hash of hashes, then sort on the keys of the hash. Here, I'm making an array of hashes, so it's pre-sorted. I'm liking that. Still wrapping my head around how to put data where Template Toolkit can get to it. I like having my HTML as HTML, with occasional <code>[% FOREACH object IN array %] [% object.value %] [% END %] </code> sorts of thing added. I used to have lots of <code> print qq{<div> $div_contents </div>} ;</code> in my code, before I got here, where the coding standard is <code> print $cgi->div( $div_contents ) ; </code>, which is fine and all, but I always was very proud of my HTML and it feels good to be getting back into writing it in the midst of the coming of HTML5 and all.

But I'm curious if I did this the correct way of putting together the data, or if there's an easier better way that isn't apparent from the Dancer::Cookbook POD. It seems like I should be expecting a greater leap than this seems to give. What am I missing, if anything?