Perl is a language that has a strong testing culture, the CPAN Testing Service (CPANTS) for example is amazing. When you release any module it gets tested on a huge variety of systems and operating systems and Perl versions automatically and for free.

Perl also has great testing tools, Test:: family of modules are hard to beat.

Writing a new module in a Test Driven Development (TDD) style is very much possible and is my preferred way of tackling the Perl Weekly Challenge when I put time aside to take them on. What you can also do is TDD the script itself via a "modulino".

This week's challenge is a script to tell you if the number you provide is a binary palindrome, you'll probably want that context for the code examples that follow. :-)

To make this possible you can wrap all the code in your script in a sub (for example run()). then call that sub only when the code is being called from Perl, rather than from the test suite. But first we want to start with a test.

use Test::More;

require_ok('./ch-1.pl');

done_testing;

This uses the standard Test::More module for "require_ok" so we can load the script itself. This test will fail till you create the script itself.

As I am creating a script that outputs to the screen (stdout) I want to test the output of the command so I use the Test::Output module.

use Test::More;
use Test::Output;

require_ok('./ch-1.pl');

stdout_is { &run() }
    'foo',
    'Dumb test to see if the test works';

done_testing;

So then we can expand the script to look like this:

__PACKAGE__->run() unless caller;

sub run {
    print "foo";
}

1;

From here we can take small steps towards out goal whilst having the safety of tests. So the test might become more sensible like this:

use Test::More;
use Test::Output;

require_ok('./ch-1.pl');

stdout_is { &run(5) }
    'xxxxx',
    '5 is a palindrome';

done_testing;

This is a small test that confirms that the script gives the desired answer in the format we want. In this case I have already coded up the module in a TDD style; so this is really about testing the wiring up of the code. Version one might look like this:

__PACKAGE__->run() unless caller;

use lib './lib';
use Binary::Palindrome;

sub run {
    my $n = $ARGV[0] || shift;

    my $bp = Binary::Palindrome->new;

    print $bp->is_palindrome($n);
}

1;

So in this still more than I actually did in step one. I actually just added the use lines and ran the tests to see if everything stayed the same. I use tests that way a lot; confirming I have not added a typo or simple syntax error. I take small steps so spotting the simple mistakes is... well simple.

The my $n = $ARGV[0] || shift; line is a way of making the sub handle input from the command line (the first arg, aka $ARGV[0]) or a scalar from the tests via shift.

In the above situation the test should fail as I am looking for "xxxxx", but I know that the code "should" return 1. So the test fails and tells me I am ok. SO then I can change the test to:

stdout_is { &run(5) }
    '1',
    '5 is a palindrome';

And this should pass, next we want to confirm the real behaviour, so test becomes:

stdout_is { &run(5) }
'1 as binary representation of 5 is 101 which is Palindrome.',
    '5 is a palindrome';

This will fail of course, so lets change the script to pass this test (and forgive me I am skipping steps):

__PACKAGE__->run() unless caller;

use lib './lib';
use Binary::Palindrome;

sub run {
    my $n = $ARGV[0] || shift;

    my $bp = Binary::Palindrome->new;
    if ( my $res = $bp->is_palindrome($n) ) {
        print "$res as binary representation of $n is ",
            $bp->represent_as_binary($n), " which is Palindrome.";
    }
}

1;

This should pass, so lets add the next test:

stdout_is { &run(4) }
'0 as binary representation of 4 is 100 which is NOT Palindrome.',
    '4 is NOT a palindrome';

Which fails, so then we add the code:

    if ( my $res = $bp->is_palindrome($n) ) {
        print "$res as binary representation of $n is ",
            $bp->represent_as_binary($n), " which is Palindrome.";
    }
    else {
        print "$res as binary representation of $n is ",
            $bp->represent_as_binary($n), " which is NOT Palindrome.";
    }

Adding this else makes our test passes. So we are in a good place.

This is a slightly contrived example, and there are many improvements to make. However, I now have a solid base to work from. This is a style of working I quite like. Think of it perhaps as Behaviour Driven Development (BDD) as well as TDD. o Or perhaps this could be described as an "Integration test" or even an "End to End test" given what we are building. I am not pedantic about the terms of purity of my TDD/BDD.

Personally I am aiming for tools that make my development easier and more reliable and this sort of testing does that for me.

When I coded this up I did TDD the module itself, so the two methods (is_palindrome and represent_as_binary) have tests.

Not doing the script and it's tests first, has affected the design of the module. The script is more complicated and the code less efficient as I a using both methods in the script AND I use represent_as_binary in the is_palindrome method. If it had side effects (i.e. wrote to a file or DB) then this would be bad; or if it was slow.

The next step might be to create a third method, that returns a data structure with the result and the binary representation. This would be more efficient; I would probably leave the construction of the text in the script as it's the "presentation layer" and leaves the module a little cleaner; though your mileage may vary. A method that returned the full answer would be very tidy in the script (thin controller).

Please do checkout the full code in the Perl Weekly Challenge git repo (https://github.com/manwar/perlweeklychallenge-club/tree/master/challenge-118/lance-wicks/perl).

And if you have any questions please drop me a message.

This week I have worked on some Go-lang for Fantasy-Judo.com adding some internationalisation (Italian support and French translation). I also did a Perl live coding session where I worked on a command line tool I wrote last year for my Zettelkasten note taking system.

I forgot to zoom the terminal so it's a little hard to read, apologies for that. And some of the audio seems muted as I think the music in the background has triggered some copyright bot in Twitch.

The script originally added back links to files that related to one another automatically; I've decided not to do that as many people suggest this is an anti-pattern in the Zettelkasten world. But I did/do still want to know when I have notes that:

• Don't link to other notes.
• Don't have links from other notes.

So I picked up the repo and started working on it, the stream is just me pottering around in Perl, so although I include it above and am leaving it up, it's pretty dry/dull... you have been warned.

What sticks in my mind from the session were a couple of things:

• Deleted code is debugged code (Jeff Sickel)
• Test2::V0 and Test::MockFile don't play nicely together
• TDD is a skill that comes with practice

"Deleted code is debugged code." – Jeff Sickel

— Programming Wisdom (@CodeWisdom) April 30, 2021

This tweet resonated with me after the live coding session. The reason being that I knew I had a small bug in my code, where an additional blank line was added each time I ran the script and it added back links. I had done some debugging back in August 2020 when I worked on this script last, but not solved the problem.

Today the code is fully "debugged" as I have now deleted the code in question. As a developer the quote an experience is important to remember. When we find bugs, should we dig into them and write complicated fixes/tests? Or as per this example maybe the best answer is to delete the code. "Less is More" and all that good stuff.

Test2::V0 vs Test::MockFile

As I'd not worked on the code for a while I started by running the tests (Always run the tests first is a mantra we could all do with chanting... especially developers in practical coding interviews, run the darn tests).

In my situation, the there were odd errors being thrown, I am presuming this has something to do with the version of Perl and modules on my current machine and the machine I was writing the code on originally as I don't recall seeing the errors back then.

On the video you'll see me scratching my head, then starting a new test file, with no tests, just the including the initial use statements. This showed that including the two libraries together caused the error. I realise now as I write this perhaps I could have done some more diagnosis (and probably will later) but at the time I solved my problem the easiest way I could... deleting the code.

Well, sort of. What I actually did was stop using Test2::V0 and switched to Test::More and seeing as the errors went away I made minor changes to my tests to use that instead.

Test Driven Development (TDD) is a skill that comes with practice.

If you watch my live coding sessions, you'll notice I try and work in a TDD style. Which when you are doing an abstract problem is a form of "deliberate practice". This week when working on an actual tool I was also operating in a TDD style. And it felt I think pretty smooth; and smooth as a result of the using it in the Perl Weekly Challenge solutions.

This is not a surprise, all the TDD people I trust do suggest that you should learn TDD outside of your job, as it's hard and you get it wrong at first. That you should not slow down your salary earning work learning the skill. This weeks coding felt like the TDD was flowing nicely as a result of the practice done in other weeks.

I found myself describing is comments what I wanted in pseudo-code comments in the script, then writing tests and code that delivered that, then replacing the pseudo-code with calls to the tested methods I'd written. The code feels pretty comfortable. I also find that I am writing a module that is larger than the script. So the script is pretty expressive and understandable:

use strict;
use warnings;

use lib './lib';
use Zettlr::Backlinker;

my $ZB = Zettlr::Backlinker->new;

my $files = $ZB->get_file_list('/home/lancew/zettel');

for my $file (@$files) {
    if ( $ZB->number_of_links_out($file) == 0 ) {
        print "NO LINKS OUT: $file\n";
    }

    if ( $ZB->number_of_links_in( $file, $files ) == 0 ) {
        print "NO LINKS IN: $file\n";
    }

}

As you can see, I am not doing much, just getting a list of files, checking the number of outbound links and checking the number of inbound links. The code has three corresponding methods. Tidy.

The tests I wrote in parts, but the overall tests I wrote that evening look like this:

use strict;
use warnings;

use Test::MockFile;
use Test::More;

use Zettlr::Backlinker;

my $CLASS = Zettlr::Backlinker->new;

my $file_1_content = <<HERE;
# This is the first file
#tag1 ~tag2

Paragraph one has no links.

Paragraph 2 links to [[20200716164925]] Coding standards.
Which is only the ID and not the full file name


Paragraph 3 links to [[20200802022902]] Made up at the time I wrote the test.

Paragraph 4 links to [[20200802022902]] and [[20200716164911]] to test a bug.

Paragraph 4 links to the second file [[22222222222222]] So should be a backlink for that file

HERE

my $file_2_content = <<HERE;
# This is the second file
#tag1 ~tag2

Paragraph one has no links.


HERE

my $file_3_content = <<HERE;
# This is the second file
#tag1 ~tag2

Links to [[11111111111111]] and [[22222222222222]]

HERE
my $mock_file_1
    = Test::MockFile->file( '11111111111111 some file.md', $file_1_content );
my $mock_file_2 = Test::MockFile->file( '22222222222222 another file.md',
    $file_2_content );
my $mock_file_3
    = Test::MockFile->file( '33333333333333 third file.md', $file_3_content );
my $mock_file_4 = Test::MockFile->file( 'README.md', $file_1_content );

my @file_list = (
    '11111111111111 some file.md',
    '22222222222222 another file.md',
    '33333333333333 third file.md',
    'README.md',
);

my $mock_dir = Test::MockFile->dir( '/foo', \@file_list, { mode => 0700 } );

subtest 'number_of_links_out' => sub {
    is $CLASS->number_of_links_out('11111111111111 some file.md'),
        4, 'Has 4 unique links in it, 5 in total as one is repeate2d';
    is $CLASS->number_of_links_out('22222222222222 another file.md'),
        0, 'Has no links in it';
    is $CLASS->number_of_links_out('33333333333333 third file.md'),
        2, 'Has 2 links in it';
};

subtest 'number_of_links_in' => sub {
    is $CLASS->number_of_links_in(
        '11111111111111 some file.md', @file_list
        ),
        1, 'Only the third file links to file 1';
    is $CLASS->number_of_links_in(
        '22222222222222 another file.md', @file_list
        ),
        2, 'Both the other files link to this file';
    is $CLASS->number_of_links_in(
        '33333333333333 third file.md', @file_list
        ),
        0, 'Neither of the other files link to this file';
};
done_testing;

As I hope you can see, I have two things I am testing links in links out, broken into two subtest blocks. I like using subtest to give clarity in TAP output and visibly in the test file itself. I could/should move the here-docs lower in the file as that's noise. Though arguably, it give the reader the content of the three test files before they see the tests.

I used Test::MockFile for the first time on this project. I don't think it's particularly popular and ideally I'd not use it and structure things in different ways. But the tool is all about reading (and originally writing) to files on disk so this test library was great for allowing me to create files and directories without actually creating temp files.

The methods I wrote look like this:

sub number_of_links_out {
    my ( $self, $filename ) = @_;

    my $links = $self->get_links_from_files($filename);

    return scalar( @{ $links->{$filename} } );
}

sub number_of_links_in {
    my ( $self, $filename, @file_list ) = @_;

    my $links = $self->get_links_from_files(@file_list);
    delete $links->{$filename};

    $filename =~ m/^(\d+)/;
    my $occurances = 0;
    for my $key ( keys %$links ) {
        for my $link ( @{ $links->{$key} } ) {
            $occurances++ if $link eq $1;
        }

    }
    return $occurances;
}

The first one is pretty concise now, it was longer, but I did refactor a little. The second came later and I was tired, and it shows. It is still the verbose version I tend to start with and whittle down once I have a working solution. I actually finished this code off stream as I was not getting things right. Just after stopping the stream I stopped making small mental mistakes and got it working properly.

This makes sense too, coding is a skill. And your skills just as your skills improve with practice, they deteriorate with tiredness.

This is one of those things that a professional coder knows. You are creating something that requires skill, working tired or sick is something you learn not to do. I have told many people about "that time" when I wrote some code at work when I had a cold/flu. The (awesome) tester who worked with me took one look and straight out told me it was rubbish and looked like someone else had written it. They saw the result of using a skill sick/fatigued, it's just not as good as when you are fresh and healthy.

Given writing software is done in our heads, making sure that our heads are clear should be a priority you'd think. But how often do we ask that question of ourselves, let alone peers or reports (if you are a manager)? How much effort are we as an industry wasting because we are allowing people creating software to do it tired or sick?

So along with practice, I am adding "Rest and wellness matter!" to my list of things I learnt this week.

This week I have more l10n work to do on/in Go-lang and want to look at how to run the above script every time I push to my Zettelkasten and have the results emailed to me. I will also try and do the Perl Weekly Challenge (I seem to be settling into a fortnightly rhythm of doing them". I really should also put some time into the Judo simulation project I have been "renovating" from the early noughties.

Lance.

This week I tried a new OBS setup to livecode the solution on Twitch.tv/lancew. The setup is thanks to Martin Wimpress who shared his setup ( Creating the Slim.AI OBS Studio set ) and this broadcast is extra special as I was able to bring Mohammad Anwar into the live stream; which was awesome.

This week I solved the "Chowla Numbers" task of the Perl Weekly Challenge, first in Perl; then in Elm.

Both I did as TDD-ish as I could, some cheating where I just Data::Dumper'd to STDOUT in the Perl version. The Elm one, I used the tests a bit more as I am less familiar in the language, and to be frank I didn't know an easy way to dump to the screen. ;-)

The Elm version I ended up with this:

module Chowla exposing (chowla, n, nums)

import List


nums : Int -> List Int
nums max =
    List.range 1 max
        |> List.map n


n : Int -> Int
n num =
    let
        denominators =
            List.range 2 (num - 1)

        numbers =
            List.map (chowla num) denominators
    in
    List.foldl (+) 0 numbers


chowla : Int -> Int -> Int
chowla num denom =
    case remainderBy denom num of
        0 ->
            denom

        _ ->
            0

The Perl looks like this:

package Chowla;

use Moo;

sub list {
    my ($self,$max) = @_;
    my @chowla_numbers;
    for my $n (1..$max) {
        push @chowla_numbers, $self->n($n);
    }
    return \@chowla_numbers;
}

sub n {
    my ($self,$num) = @_;

    my $total = 0;
    for my $n (2..$num-1) {
        if ($num % $n == 0) {
            $total += $n;
        }
    }
    return $total;
}

1;

Neither are particularly long, the Elm without type signatures is 15 lines. The Perl 21 lines (though if you formatted it differently it would be about the same).

The Elm code, not having a for loop relies on map and fold (specifically foldl). I could/should try the Perl with a map. There is no equivalent of foldl in Perl. foldl is in interesting command, basically like a map, but collapsing in on itself via a function, in my case + so just summing all the numbers.

Testing the code in Perl was as ever easy, Perl's Test2:V0 is a great tool. On the Elm side I am getting more comfortable with the testing, I did not use any of the built in fuzzing on this one, I will have to explore it more.

Over the next week I have some more Go code I want to work on for https:://fantasy-judo.com and I'd like to work on modernising my https://www.vwjl.net/ a little more, the combat simulation code is where my attention should go. Given that, I may not live stream the weekly challenge this week, but might stream some general hacking about.

Having an additional day off this week (extending the UK Easter break) I decided to do another short live coding session.

Once again the subject being the (Perl Weekly Challenge)[https://perlweeklychallenge.org/blog/perl-weekly-challenge-107/#TASK1] by Mohammad Anwar.

This weeks task #1 is to write a script that produces a list of the first three "self describing numbers". I tackled it without any prep (pre-reading or planning), which is always risky, but it went OK.

As ever I try and work in a Test Driven Development (TDD) style. This means I try and work out small steps with tests that confirm my hypothesis as I go along. I am not super strict about what that means. I use TDD as a tool to help me solve problems, rather than a strict approach. Especially on this sort of recreational coding; I don't worry at all about coverage or too much about being strict TDD... whatever your definition of strict TDD is.

I use warn and Data::Dumper too. The mindset is the same as TDD, I am testing the idea of what is next.

There has been a discussion on Twitter about TDD (as always) and I think I fall into the camp of thinking that TDD is about driving my design, not providing coverage of edge cases. That's a different issue. Unit tests may still be the tool, but the objective is different. If you watch the video you'll see I only really code the "happy path"; intentionally so. I am trying to determine the solution to the problem. So it's about interfaces and making it easy to make changes without regressing.

You'll see in the video that I end up in a muddle a couple of times. I notice that that's when I have not been using tests to guide me. I also notice that at the end I was easily able to demonstrate changing the if statement from eq to == and know immediately if it was OK as the tests proved it still worked. This is what TDD gives you. That ability to refactor (even if it is only going from eq to ==) with some confidence. The confidence encourages me/you to refactor more than as it's not scary.

Another observation I should point out (and I have done so before) is that I tend to start with minimal test and package... as it saves me from typo bugs that can be ever so frustrating. I try not to write a full set of tests of examples.

The last thing I will add, is that if you watch the video... you'll see I end up with I think 3 working solutions to the task requirement. Starting with simply returning an array of the three numbers expected with no real logic. My test proves that the call works. It also provides the foundation that the next piece of coding provided. The second solution was looping through all the numbers from 1 to 30,000 and checking (via a new sub) if that number was self-describing. Again, I did this the "cheats way" of looking for the specific three numbers and returning positively with three simple if statements.

Only then did I write a solution that actually worked across any number.

If this was a professional coding problem. I would ideally do it the same way. Why? Because I had a provably working solution I "could" ship very quickly. The later solutions were better and more in line with what would actually be wanted; but getting those first two solutions coded up and tested is important at it allowed me to learn quickly and in small increments and if I'd committed the code at the right points a colleague could have carried on the project if need be. Or having gotten it out the door fast; maybe it turns out we don't need that feature and we revert with minimal time invested in the feature.

In this example, it's not such a big deal. But I've been in the situation where the code changes are hundreds-thousands of lines of code for a new feature that we didn't know would work. And watching a huge piece of work fail and being cancelled is far more painful emotionally as a developer (and financially as a business) than a small change.

Thanks again to Mohammad for this awesome project of his!

This week I quickly did the Perl Weekly Challenge as a live coding session on Twitch (now also on Youtube: https://www.youtube.com/watch?v=E4Bwh-FVTns ).

If you watch the video hopefully you'll see the approach I take to these challenges, I don't write defensively, I do write in a test first and test as you go basis. Below is the test file that i ended up with, it could be made better I nkow, but it got me to a working, tested solution nice and quickly.

# t/01-maxgap.t
use Test2::V0 -target => Maxgap;

my @example1 = (2, 9, 3, 5);
my $expected1 = 4;

is $CLASS->maxgap(@example1), $expected1, 'Example 1';

my @example2 = (1, 3, 8, 2, 0);
my $expected2 = 5;

is $CLASS->maxgap(@example2), $expected2, 'Example 2';


my @example3 = (5);
my $expected3 = 0;

is $CLASS->maxgap(@example3), $expected3, 'Example 3';

done_testing;

Using test2::V0 hides the horrible "stutter", by which I mean the module name and the method repeat: Maxgap::maxgap. So ideally I would change the names.

The code looks like this:

# lib/Maxgap.pm
package Maxgap;

sub maxgap {
    my ($self, @integers) = @_;

    @integers = sort @integers;

    my $maxgap = 0;
    for my $i (1 .. @integers-1){
        my $gap = $integers[$i] - $integers[$i-1];
        if ( $gap > $maxgap) {
            $maxgap = $gap;
        }
    }

    return $maxgap;
}

1;

It's not too complex, and I am sure with some refactoring it could be made more idiomatic. This version however works and was up and running in less than 30 minutes (of coding time).

The final step is to use it in a simple command line script:

# ch-1.pl
use strict;
use warnings;

use lib './lib';
use Maxgap;

my $maxgap = Maxgap::maxgap(@ARGV);

print "Output: $maxgap\n";

You can see the "stutter" is pretty pronounced here. There is zero error checking so that's not great.

Anyway... I am working on the Elm version and will share another day if I come up with a version I like.