Agile Test Management

within Continuous Integration

Angie Jones

http://angiejones.tech

@techgirl1908

Senior Automation Engineer

Twitter, San Francisco, CA, USA

Once upon a time...

@techgirl1908

We've

automated...

NOW WHAT?

@techgirl1908

What is

CONTINUOUS

INTEGRATION

@techgirl1908

Do I add

ALL

of these???

@techgirl1908

...but what about

FAST FEEDBACK

@techgirl1908

...and

TRIAGING FAILURES

@techgirl1908

...and

TEAM OWNING QUALITY

@techgirl1908

@techgirl1908

The challenge for 

AGILE TEAMS

@techgirl1908

Build Tests

Smoke Tests

All Other Tests

Too slow!

Not ours!

DIVIDE

&

CONQUER

@techgirl1908

@techgirl1908

@techgirl1908

Build Tests

Smoke Tests

Work Zone Tests

Build Tests

Smoke Tests

Work Zone Tests

Build Tests

Smoke Tests

Work Zone Tests

Build Tests

Smoke Tests

Work Zone Tests

Registration

Shopping

Check Out

Search

@techgirl1908

Build Tests

Smoke Tests

All Tests

How do we

 

 

 

to a build?

ASSIGN THE TESTS

@techgirl1908

you're IT!

TAG

@techgirl1908

@Tag("shopping")

@Test

public void testViewProductListing() {

...

}

JUnit

@techgirl1908

@Tag("shopping")

@Tag("cart")

@Test

public void testAddProductToCart()

{

...

}

JUnit

@techgirl1908

@Test(groups="shopping")

public void testViewProductListing()

{

...

}

TestNG

@techgirl1908

@Test(groups={"shopping", "cart"})

public void testAddProductToCart()

{

...

}

TestNG

@techgirl1908

Configuration

@techgirl1908

Filtering in JUnit

@techgirl1908

<properties>
  <includeTags>shopping | cart</includeTags>
</properties>

Filtering in TestNG

@techgirl1908

<groups>
     <run>
          <include name="shopping"/>
          <include name="cart"/>
     </run>
</groups>

Filtering in TestNG

@techgirl1908

<groups>
     <run>
          <include name="shopping"/>
          <include name="cart"/>
     </run>
</groups>

Limitations

@techgirl1908

@techgirl1908

Separate files

@techgirl1908

Combinations

@techgirl1908

Lack of flexibility

Runtime Configuration

@techgirl1908

TestNG: mvn test -Dgroups=cart

Maven

@techgirl1908

JUnit: mvn -DincludeTags=cart

JUnit: gradlew test -DincludeTags=cart

Gradle

TestNG: gradlew test -P testGroups=cart

@techgirl1908

Build Tests

Smoke Tests

Registration Tests

Build Tests

Smoke Tests

Search Tests

Build Tests

Smoke Tests

Shopping Tests

Build Tests

Smoke Tests

Shopping + Cart Tests

Team 1

Team 3

Team 4

Team 2

Lessons Learned

@techgirl1908

Bugs can be missed

@techgirl1908

Tests have to be reliable

@techgirl1908

Related != Blocker

@techgirl1908

Divide & Conquer!

Angie Jones

http://angiejones.tech

@techgirl1908

Senior Automation Engineer

Twitter, San Francisco, CA, USA

Agile Test Management within Continuous Integration

By Angie Jones

Agile Test Management within Continuous Integration

Learn to manage a suite of thousands of tests within a CI environment and yet still get fast feedback

  • 3,698