Refactoring with Statsd, Toggle and Scientist

lancew@cpan.org

About Me

Lance Wicks,

Development team leader at www.cv-library.co.uk

100,000+ jobs from 10,000+ companies, 10,000,000+ users, 2,000,000+ job applications per month.

Perl 5 code base started in 2000.

Yes, we are hiring!

http://www.cv-library.co.uk/yapceu

Net::StatsdToggle
Scientist

Our Situation

16 year old code base, Perl5, fast growth, lots of new features, 400,000 lines of code.

Refactoring is essential, but difficult.

Zero tolerance to breaking the site, or negatively impacting performance.

Our Refactoring Rules

If you can't test it, don't refactor.
If you can't measure it, don't refactor.
If you can't turn it on (and off!), don't refactor.

Our Refactoring Rules

If you can't test it, don't refactor.
Test::More
If you can't measure it, don't refactor.
Net::Statsd
If you can't turn it on (and off!), don't refactor.
Toggle

Test::More

If you can't test it, don't refactor it.

EXODIST

use Test::More; 

my $transactions_via_old = $candidate->transactions;
my $transactions_via_new = $candidate->transactions_new;

is_deeply(
    $transactions_via_old, 
    $transactions_via_new,
    'Old and new methods return same data structures'
);

done_testing;

Net::Statsd

If you can't measure it, don't refactor it.

COSIMO

use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];

for my $candidate (@active_candidates) {
    # Expensive transaction in loop of large data set
    update_with_new_total($candidate);
    
    Net::Statsd::inc('candidates.updated_with_new_totals');
}


Net::Statsd::timing(
    'candidates.new_totals',
    Time::HiRes::tv_interval($start_time) * 1000
);

Toggle

If you can't turn it on AND off, don't refactor it.

CVLIBRARY

use Toggle;
use Redis;

my $redis = Redis->new;
my $toggle = Toggle->new( storage => $redis );

$toggle->define_group( staff => sub {
    my $user = shift;
    return $user->has_role('staff');
});

if ( $toggle->is_active(test_new_query => $user) ) {
    $db_repo->get_transactions(user => $user)
}

Scientist

Bringing it all together.

LANCEW





# TODO: Lets talk about that later. :-)

How we got here.

Step 1: Automated testing

Unit, integration/service and end to end tests all run by Jenkins on every commit.

Step 1: Automated testing

use CVLibrary::SalaryConverter 'convert_salary';
use Test::Fatal;
use Test::More;

sub is_salary_converted_from_day_month : Test {
    is convert_salary( from => 'day', to => 'month', value => 75 ), 1629;
}

sub is_salary_converted_from_month_to_year : Test {
    is convert_salary( from => 'month', to => 'annum', value => 1629 ), 19548;
}

sub is_salary_converted_from_year_to_week : Test {
    is convert_salary( from => 'annum', to => 'month', value => 19500 ), 1625;
}

sub will_class_throw_if_from_param_invalid : Tests {
    ok exception {
        convert_salary( from => 'year', to => 'day', value => 22000 );
    };
}

sub will_class_throw_if_to_param_invalid : Tests {
    ok exception {
        convert_salary( from => 'annum', to => 'monthly', value => 22000 );
    };
}


package S2z8N3;{
    $zyp=S2z8N3;use Socket;
        (S2z8N3+w1HC$zyp)&
    open SZzBN3,"<$0"
  ;while(<SZzBN3>){/\s\((.*p\))&/
    &&(@S2zBN3=unpack$age,$1)}foreach
   $zyp(@S2zBN3)
  while($S2z8M3++!=$zyp-
  30){$_=<SZz8N3>}/^(.)/|print $1
      ;$S2z8M3=0}s/.*//|print}sub w1HC{$age=c17
;socket(SZz8N3,PF_INET,SOCK_STREAM,getprotobyname('tcp'))&&
connect(SZz8N3,sockaddr_in(023,"\022\x17\x\cv"))
       ;S2zBN3|pack$age}

use Test::More;
use S2z8N3;

... ????

First place, 1st Annual Obfuscated Perl Contest: Joe Futrelle.

Legacy Code is hard to test.

“No Battle Plan Survives Contact With the Enemy”

- Helmuth von Moltke

“No software Survives Contact With the users”

Step 2: Measuring

Net::Statsd & Graphite starting in 2013.

Step2: Measuring


Net::Statsd::increment('site.logins');


use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];
 
my $candidate_applications = $self->db->get_candidate_applications();

# note: time value sent to timing should
# be in milliseconds.
Net::Statsd::timing(
    'candidate.applications',
    Time::HiRes::tv_interval($start_time) * 1000
);
 
Net::Statsd::gauge('core.temperature' => 55);

Net::Statsd -> Graphite.

Step 2: Measuring

Net::Statsd -> Graphite.

Step2: Measuring

Net::Statsd -> Graphite.

Step 3: Turn it On/Off

Toggle.

Step 3: Feature Toggles

Toggle


if ( $toggle->is_enabled('chat') ) {
    # Code for cool new chat feature
}

if ( $toggle->is_enabled('chat' => $user) ) {
    # Code for cool new chat feature some users
}

Step 3: Feature Toggles

Toggle


$toggle->activate_percentage( chat => 100 );

if ( $toggle->is_enabled('chat') ) {
    # Code for cool new chat feature
}

$toggle->define_group( admins => sub { shift->is_an_admin() } );

if ( $toggle->is_enabled('chat' => $user) ) {
    # Code for cool new chat feature 
}

Step 3: Feature Toggles

Toggle

Bringing it together for refactoring.

Bring it all together



if ( $toggle->is_enabled('new_candidate_applications') ) {
    my $candidate_applications = $self->db->get_applications_new();
}
else
{
    my $candidate_applications = $self->db->get_applications();
}

Bring it all together

use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];

if ( $toggle->is_enabled('new_candidate_applications') ) {
    my $candidate_applications = $self->db->get_applications_new();

    Net::Statsd::timing(
        'candidate.applications.new',
        Time::HiRes::tv_interval($start_time) * 1000
    ); 
}
else
{
    my $candidate_applications = $self->db->get_applications();

    Net::Statsd::timing(
        'candidate.applications.old',
        Time::HiRes::tv_interval($start_time) * 1000
    );
}

Bring it all together II

use Test::More;
use Time::HiRes;
my $start_time = [ Time::HiRes::gettimeofday ];

my $candidate_applications_new = $self->db->get_applications_new();

Net::Statsd::timing(
    'candidate.applications.new',
    Time::HiRes::tv_interval($start_time) * 1000
); 

$start_time = [ Time::HiRes::gettimeofday ];
my $candidate_applications_old = $self->db->get_applications();
Net::Statsd::timing(
    'candidate.applications.old',
    Time::HiRes::tv_interval($start_time) * 1000
);

warn is_deeply($candidate_applications_old, $candidate_applications_new);

my $candidate_applications = $canidate_applications_old;

Bring it all together III

use Test::Deep::NoTest;
use Time::HiRes;

my $start_time = [ Time::HiRes::gettimeofday ];

my $candidate_applications_new = $self->db->get_applications_new();

Net::Statsd::timing(
    'candidate.applications.new',
    Time::HiRes::tv_interval($start_time) * 1000
); 

$start_time = [ Time::HiRes::gettimeofday ];
my $candidate_applications_old = $self->db->get_applications();
Net::Statsd::timing(
    'candidate.applications.old',
    Time::HiRes::tv_interval($start_time) * 1000
);
 
if (eq_deeply($candidate_applications_new,
              $candidate_applictaions_old)) {
  Net::Statsd::increment('candidate.applications.ok');
}

my $candidate_applications = $candidate_applications_old;

Github Scientist

Scientist

Compare results
Timing
Toggle on/off
Publish Results

Perl5 Scientist

Compare results
Timing
Toggle on/off
Publish Results

Scientist

use Scientist;
 
 
my $experiment = Scientist->new(
  experiment => 'MyTest',
  use => \&old_code,
  try => \&new_code,
);
 
my $answer = $experiment->run;
 
warn 'There was a mismatch between control and candidate'
  if $experiment->result->{'mismatched'};

Scientist

use Scientist;
 
 
my $experiment = Scientist->new(
  experiment => 'candidate.applications',
  use => \&old_code,
  try => \&new_code,
);
 
my $candidate_applications = $experiment->run;
 
warn 'There was a mismatch between control and candidate'
  if $experiment->result->{'mismatched'};
 
say 'Timings:';
say '"Use" code:   ', $experiment->result->{control}{duration}, ' microseconds';
say '"Try" code: ', $experiment->result->{candidate}{duration}, ' microseconds';

Extending Scientist

package My::Scientist;
use parent 'Scientist';
 
use Net::Statsd;
 
sub publish {
  my $self = shift;
 
  my $experiment = $self->result->{experiment};
  # Increment counter for every match or mismatch
  Net::Statsd::increment("$experiment.mismatch") if $self->result->{mismatched};
  Net::Statsd::increment("$experiment.match") unless $self->result->{mismatched};
 
  # Log timings (converting Scientist microseconds to StatsD miliseconds)
  # Note: This implementation rounds down the duration.
  Net::Statsd::timing("$experiment.control",   
                      int $self->result->{control}{duration} * 1_000);
  Net::Statsd::timing("$experiment.candidate", 
                      int $self->result->{candidate}{duration} * 1_000);
}
 
1;

Bring it all together III

use Test::Deep::NoTest;
use Time::HiRes;

my $start_time = [ Time::HiRes::gettimeofday ];

my $candidate_applications_new = $self->db->get_applications_new();

Net::Statsd::timing(
    'candidate.applications.new',
    Time::HiRes::tv_interval($start_time) * 1000
); 

$start_time = [ Time::HiRes::gettimeofday ];
my $candidate_applications_old = $self->db->get_applications();
Net::Statsd::timing(
    'candidate.applications.old',
    Time::HiRes::tv_interval($start_time) * 1000
);
 
if (eq_deeply($candidate_applications_new,
              $candidate_applictaions_old)) {
  Net::Statsd::increment('candidate.applications.ok');
}

my $candidate_applications = $candidate_applications_old;

Bring it all together IV

use My::Scientist;

my $experiment = My::Scientist->new(
  experiment => 'candidate.applications',
  use        => \&get_applications(),
  try        => \&get_applications_new(),
);
 
my $candidate_applications = $experiment->run;

Bring it all together IV

use My::Scientist;

my $experiment = My::Scientist->new(
  enabled    => $toggle->is_active('candidate.applications'),
  experiment => 'candidate.applications',
  use        => \&get_applications(),
  try        => \&get_applications_new(),
);
 
my $candidate_applications = $experiment->run;

Scientist

use My::Scientist;

my $experiment = My::Scientist->new(
  experiment => 'candidate.applications',
  use        => \&get_applications(),
  try        => \&get_applications_new(),
);
 
my $candidate_applictaions = $experiment->run;

Always returns "use" result
Runs "try" code inside eval{}
Runs try/use in random order
Publish() called by run
- Timings
- Match/Mismatch
Designed for Production
Used a lot in Development
Good for Getter, not Setters

Our Refactoring Rules

If you can't test it, don't refactor.
Test::More
If you can't measure it, don't refactor.
Net::Statsd
If you can't turn it on (and off!), don't refactor.
Toggle

Scientist

Scientist also available in Perl6

Refactoring with Statsd, Toggle and Scientist

lancew@cpan.org

Thank you!
Any Questions?

Refactoring with Statsd, Toggle and Scientist

lancew@cpan.org

Refactoring with Statsd, Toggle and Scientist

About Me

Yes, we are hiring!

Net::StatsdToggle Scientist

Our Situation

Our Refactoring Rules

Our Refactoring Rules

Test::More

Net::Statsd

Toggle

Scientist

How we got here.

Step 1: Automated testing

Step 1: Automated testing

Legacy Code is hard to test.

“No Battle Plan Survives Contact With the Enemy”

- Helmuth von Moltke

“No software Survives Contact With the users”

Step 2: Measuring

Step2: Measuring

Step 2: Measuring

Step2: Measuring

Step 3: Turn it On/Off

Step 3: Feature Toggles

Step 3: Feature Toggles

Step 3: Feature Toggles

Bringing it together for refactoring.

Bring it all together

Bring it all together

Bring it all together II

Bring it all together III

Github Scientist

Scientist

Perl5 Scientist

Scientist

Scientist

Extending Scientist

Bring it all together III

Bring it all together IV

Bring it all together IV

Scientist

Our Refactoring Rules

Scientist

Scientist also available in Perl6

Refactoring with Statsd, Toggle and Scientist

Refactoring with Statsd, Toggle and Scientist

YAPC:EU 2016 Refactoring

More from Lance Wicks

Net::StatsdToggle
Scientist