performance - Is the Perl Goatse 'Secret Operator' efficient?

Question

The "goatse operator" or the =()= idiom in Perl causes an expression to be evaluated in list context.

An example is:

my $str = "5 and 4 and a 3 and 2 1 BLAST OFF!!!";
my $count =()= $str =~ /d/g; # 5 matches...
print "There are $count numbers in your countdown...

";

As I interprete the use, this is what happens:

1. $str =~ /d/g matches all the digits. The g switch and list context produces a list of those matches. Let this be the "List Producer" example, and in Perl this could be many things.
2. the =()= causes an assignment to an empty list, so all the actual matches are copied to an empty list.
3. The assignment in scalar context to $count of the list produced in 2. gives the count of the list or the result of 5.
4. The reference count of the empty list =()= goes to zero after the scalar assignment. The copy of the list elements is then deleted by Perl.

The questions on efficiency are these:

1. Am I wrong in how I am parsing this?
2. If you have some List Producer and all you are interested in is the count, is there a more efficient way to do this?

It works great with this trivial list, but what if the list was hundreds of thousands of matches? With this method you are producing a full copy of every match then deleting it just to count them.

Answer 1

Perl 5 is smart about copying lists. It only copies as many items as are on the left hand side. It works because list assignment in scalar context yields the number of items on the right hand side. So, n items will be created by the regex, but they won't be copied and discarded, just discarded. You can see the difference the extra copy makes in the naive case in the benchmark below.

As for efficiency, an iterative solution is often easier on memory and CPU usage, but this must be weighed against the succinctness of the goatse secret operator. Here are the results of benchmarking the various solutions:

naive: 10
iterative: 10
goatse: 10

for 0 items:
               Rate iterative    goatse     naive
iterative 4365983/s        --       -7%      -12%
goatse    4711803/s        8%        --       -5%
naive     4962920/s       14%        5%        --

for 1 items:
               Rate     naive    goatse iterative
naive      749594/s        --      -32%      -69%
goatse    1103081/s       47%        --      -55%
iterative 2457599/s      228%      123%        --

for 10 items:
              Rate     naive    goatse iterative
naive      85418/s        --      -33%      -82%
goatse    127999/s       50%        --      -74%
iterative 486652/s      470%      280%        --

for 100 items:
             Rate     naive    goatse iterative
naive      9309/s        --      -31%      -83%
goatse    13524/s       45%        --      -76%
iterative 55854/s      500%      313%        --

for 1000 items:
            Rate     naive    goatse iterative
naive     1018/s        --      -31%      -82%
goatse    1478/s       45%        --      -75%
iterative 5802/s      470%      293%        --

for 10000 items:
           Rate     naive    goatse iterative
naive     101/s        --      -31%      -82%
goatse    146/s       45%        --      -75%
iterative 575/s      470%      293%        --

Here is the code that generated it:

#!/usr/bin/perl

use strict;
use warnings;

use Benchmark;

my $s = "a" x 10;

my %subs = (
    naive => sub {
        my @matches = $s =~ /a/g;
        return scalar @matches;
    },
    goatse => sub {
        my $count =()= $s =~ /a/g;
        return $count;
    },
    iterative => sub {
        my $count = 0;
        $count++ while $s =~ /a/g;
        return $count;
    },
);

for my $sub (keys %subs) {
    print "$sub: @{[$subs{$sub}()]}
";
}

for my $n (0, 1, 10, 100, 1_000, 10_000) {
    $s = "a" x $n;
    print "
for $n items:
";
    Benchmark::cmpthese -1, \%subs;
}