Refactoring with Statsd, Toggle and Scientist

lancew@cpan.org

About Me

Lance Wicks,

Development team leader at www.cv-library.co.uk

 

100,000+ jobs from 10,000+ companies, 10,000,000+ users, 2,000,000+ job applications per month.

 

 

Perl 5 code base started in 2000.

Yes, we are hiring!

http://www.cv-library.co.uk/yapceu

Net::StatsdToggle
Scientist

Our Situation

16 year old code base, Perl5, fast growth, lots of new features, 400,000 lines of code.

 

Refactoring is essential, but difficult.

 

Zero tolerance to breaking the site, or negatively impacting performance.

 

Our Refactoring Rules

  • If you can't test it, don't refactor.


  • If you can't measure it, don't refactor.
    ​

  • If you can't turn it on (and off!), don't refactor.

Our Refactoring Rules

  • If you can't test it, don't refactor.
                Test::More
     
  • If you can't measure it, don't refactor.
    ​            Net::Statsd
     
  • If you can't turn it on (and off!), don't refactor.
                Toggle

Test::More

 

If you can't test it, don't refactor it.

 

 

EXODIST

 

use Test::More; 

my $transactions_via_old = $candidate->transactions;
my $transactions_via_new = $candidate->transactions_new;

is_deeply(
    $transactions_via_old, 
    $transactions_via_new,
    'Old and new methods return same data structures'
);

done_testing;

Net::Statsd

 

If you can't measure it, don't refactor it.

 

 

COSIMO

 

use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];

for my $candidate (@active_candidates) {
    # Expensive transaction in loop of large data set
    update_with_new_total($candidate);
    
    Net::Statsd::inc('candidates.updated_with_new_totals');
}


Net::Statsd::timing(
    'candidates.new_totals',
    Time::HiRes::tv_interval($start_time) * 1000
);

Toggle

 

If you can't turn it on AND off, don't refactor it.

 

 

CVLIBRARY

use Toggle;
use Redis;

my $redis = Redis->new;
my $toggle = Toggle->new( storage => $redis );

$toggle->define_group( staff => sub {
    my $user = shift;
    return $user->has_role('staff');
});

if ( $toggle->is_active(test_new_query => $user) ) {
    $db_repo->get_transactions(user => $user)
}

Scientist

 

 

Bringing it all together.

 

LANCEW





# TODO: Lets talk about that later. :-)

How we got here.

Step 1: Automated testing

Unit, integration/service and end to end tests all run by Jenkins on every commit.

Step 1: Automated testing

use CVLibrary::SalaryConverter 'convert_salary';
use Test::Fatal;
use Test::More;

sub is_salary_converted_from_day_month : Test {
    is convert_salary( from => 'day', to => 'month', value => 75 ), 1629;
}

sub is_salary_converted_from_month_to_year : Test {
    is convert_salary( from => 'month', to => 'annum', value => 1629 ), 19548;
}

sub is_salary_converted_from_year_to_week : Test {
    is convert_salary( from => 'annum', to => 'month', value => 19500 ), 1625;
}

sub will_class_throw_if_from_param_invalid : Tests {
    ok exception {
        convert_salary( from => 'year', to => 'day', value => 22000 );
    };
}

sub will_class_throw_if_to_param_invalid : Tests {
    ok exception {
        convert_salary( from => 'annum', to => 'monthly', value => 22000 );
    };
}

package S2z8N3;{
    $zyp=S2z8N3;use Socket;
        (S2z8N3+w1HC$zyp)&
    open SZzBN3,"<$0"
  ;while(<SZzBN3>){/\s\((.*p\))&/
    &&(@S2zBN3=unpack$age,$1)}foreach
   $zyp(@S2zBN3)
  while($S2z8M3++!=$zyp-
  30){$_=<SZz8N3>}/^(.)/|print $1
      ;$S2z8M3=0}s/.*//|print}sub w1HC{$age=c17
;socket(SZz8N3,PF_INET,SOCK_STREAM,getprotobyname('tcp'))&&
connect(SZz8N3,sockaddr_in(023,"\022\x17\x\cv"))
       ;S2zBN3|pack$age}
use Test::More;
use S2z8N3;

... ????

First place, 1st Annual Obfuscated Perl Contest: Joe Futrelle.

Legacy Code is hard to test.

“No Battle Plan Survives Contact With the Enemy”

- Helmuth von Moltke

“No software Survives Contact With the users”

Step 2: Measuring

Net::Statsd & Graphite starting in 2013.

 

Step2: Measuring


Net::Statsd::increment('site.logins');


use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];
 
my $candidate_applications = $self->db->get_candidate_applications();

# note: time value sent to timing should
# be in milliseconds.
Net::Statsd::timing(
    'candidate.applications',
    Time::HiRes::tv_interval($start_time) * 1000
);
 
Net::Statsd::gauge('core.temperature' => 55);

Net::Statsd -> Graphite.

 

Step 2: Measuring

Net::Statsd -> Graphite.

 

Step2: Measuring

Net::Statsd -> Graphite.

 

Step 3: Turn it On/Off

Toggle.

 

Step 3: Feature Toggles

Toggle


if ( $toggle->is_enabled('chat') ) {
    # Code for cool new chat feature
}

if ( $toggle->is_enabled('chat' => $user) ) {
    # Code for cool new chat feature some users
}

Step 3: Feature Toggles

Toggle


$toggle->activate_percentage( chat => 100 );

if ( $toggle->is_enabled('chat') ) {
    # Code for cool new chat feature
}

$toggle->define_group( admins => sub { shift->is_an_admin() } );

if ( $toggle->is_enabled('chat' => $user) ) {
    # Code for cool new chat feature 
}

Step 3: Feature Toggles

Toggle

Bringing it together for refactoring.

Bring it all together



if ( $toggle->is_enabled('new_candidate_applications') ) {
    my $candidate_applications = $self->db->get_applications_new();
}
else
{
    my $candidate_applications = $self->db->get_applications();
}

Bring it all together

use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];

if ( $toggle->is_enabled('new_candidate_applications') ) {
    my $candidate_applications = $self->db->get_applications_new();

    Net::Statsd::timing(
        'candidate.applications.new',
        Time::HiRes::tv_interval($start_time) * 1000
    ); 
}
else
{
    my $candidate_applications = $self->db->get_applications();

    Net::Statsd::timing(
        'candidate.applications.old',
        Time::HiRes::tv_interval($start_time) * 1000
    );
}

Bring it all together II

use Test::More;
use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];

my $candidate_applications_new = $self->db->get_applications_new();

Net::Statsd::timing(
    'candidate.applications.new',
    Time::HiRes::tv_interval($start_time) * 1000
); 

$start_time = [ Time::HiRes::gettimeofday ];
my $candidate_applications_old = $self->db->get_applications();
Net::Statsd::timing(
    'candidate.applications.old',
    Time::HiRes::tv_interval($start_time) * 1000
);

warn is_deeply($candidate_applications_old, $candidate_applications_new);

my $candidate_applications = $canidate_applications_old;

Bring it all together III

use Test::Deep::NoTest;
use Time::HiRes;

my $start_time = [ Time::HiRes::gettimeofday ];

my $candidate_applications_new = $self->db->get_applications_new();

Net::Statsd::timing(
    'candidate.applications.new',
    Time::HiRes::tv_interval($start_time) * 1000
); 

$start_time = [ Time::HiRes::gettimeofday ];
my $candidate_applications_old = $self->db->get_applications();
Net::Statsd::timing(
    'candidate.applications.old',
    Time::HiRes::tv_interval($start_time) * 1000
);
 
if (eq_deeply($candidate_applications_new,
              $candidate_applictaions_old)) {
  Net::Statsd::increment('candidate.applications.ok');
}

my $candidate_applications = $candidate_applications_old;

Github Scientist

Scientist

 

  • Compare results
  • Timing
  • Toggle on/off
  • Publish Results

 

Perl5 Scientist

 

  • Compare results
  • Timing
  • Toggle on/off
  • Publish Results

 

Scientist

use Scientist;
 
 
my $experiment = Scientist->new(
  experiment => 'MyTest',
  use => \&old_code,
  try => \&new_code,
);
 
my $answer = $experiment->run;
 
warn 'There was a mismatch between control and candidate'
  if $experiment->result->{'mismatched'};

Scientist

use Scientist;
 
 
my $experiment = Scientist->new(
  experiment => 'candidate.applications',
  use => \&old_code,
  try => \&new_code,
);
 
my $candidate_applications = $experiment->run;
 
warn 'There was a mismatch between control and candidate'
  if $experiment->result->{'mismatched'};
 
say 'Timings:';
say '"Use" code:   ', $experiment->result->{control}{duration}, ' microseconds';
say '"Try" code: ', $experiment->result->{candidate}{duration}, ' microseconds';

Extending Scientist

package My::Scientist;
use parent 'Scientist';
 
use Net::Statsd;
 
sub publish {
  my $self = shift;
 
  my $experiment = $self->result->{experiment};
  # Increment counter for every match or mismatch
  Net::Statsd::increment("$experiment.mismatch") if $self->result->{mismatched};
  Net::Statsd::increment("$experiment.match") unless $self->result->{mismatched};
 
  # Log timings (converting Scientist microseconds to StatsD miliseconds)
  # Note: This implementation rounds down the duration.
  Net::Statsd::timing("$experiment.control",   
                      int $self->result->{control}{duration} * 1_000);
  Net::Statsd::timing("$experiment.candidate", 
                      int $self->result->{candidate}{duration} * 1_000);
}
 
1;

Bring it all together III

use Test::Deep::NoTest;
use Time::HiRes;

my $start_time = [ Time::HiRes::gettimeofday ];

my $candidate_applications_new = $self->db->get_applications_new();

Net::Statsd::timing(
    'candidate.applications.new',
    Time::HiRes::tv_interval($start_time) * 1000
); 

$start_time = [ Time::HiRes::gettimeofday ];
my $candidate_applications_old = $self->db->get_applications();
Net::Statsd::timing(
    'candidate.applications.old',
    Time::HiRes::tv_interval($start_time) * 1000
);
 
if (eq_deeply($candidate_applications_new,
              $candidate_applictaions_old)) {
  Net::Statsd::increment('candidate.applications.ok');
}

my $candidate_applications = $candidate_applications_old;

Bring it all together IV

use My::Scientist;

my $experiment = My::Scientist->new(
  experiment => 'candidate.applications',
  use        => \&get_applications(),
  try        => \&get_applications_new(),
);
 
my $candidate_applications = $experiment->run;

Bring it all together IV

use My::Scientist;

my $experiment = My::Scientist->new(
  enabled    => $toggle->is_active('candidate.applications'),
  experiment => 'candidate.applications',
  use        => \&get_applications(),
  try        => \&get_applications_new(),
);
 
my $candidate_applications = $experiment->run;

Scientist

use My::Scientist;

my $experiment = My::Scientist->new(
  experiment => 'candidate.applications',
  use        => \&get_applications(),
  try        => \&get_applications_new(),
);
 
my $candidate_applictaions = $experiment->run;
  • Always returns "use" result
  • Runs "try" code inside eval{}
  • Runs try/use in random order
  • Publish() called by run
    • Timings
    • Match/Mismatch
       
  • Designed for Production
  • Used a lot in Development
     
  • Good for Getter, not Setters

Our Refactoring Rules

  • If you can't test it, don't refactor.
                Test::More
     
  • If you can't measure it, don't refactor.
    ​            Net::Statsd
     
  • If you can't turn it on (and off!), don't refactor.
                Toggle

Scientist

Scientist also available in Perl6

Refactoring with Statsd, Toggle and Scientist

lancew@cpan.org

Thank you!
Any Questions?

Refactoring with Statsd, Toggle and Scientist

lancew@cpan.org