Liebling, ich habe die Testpyramide auf den Kopf gestellt

JUG Karlsruhe

Tobse Fritz

  •       Reich
  •       Gut aussehend
     
  •       ITscope GmbH
  •       Kotlin Fanboy
  •        Java Entwickler
  •       Clean Coder  
  •       Hobby Fotograf
  •       Gaming Nerd  
  •       3D Printer
  •       Daddy

https://tobse.eu

image/svg+xml

JUG Karlsruhe

FRONTEND

BACKEND

bluecoders.com

FRONTEND

BACKEND

TESTS

bluecoders.com

JUG Karlsruhe

Lesson
Learned

HOW
TO

Technik
Check

DOS

DONT'S

&

Warum Testen?

Die Test-Pyramide

https://stackshare.io/itscope-gmbh/

SpringBoot

MariaDB

ElasticSearch

Redis

Such-Service

Image-Service

Tracking-Service

Price-Service

Vue.js

RedGiant

Technik
Check

SOFTWARE QUALITÄT

https://de.wikipedia.org/wiki/ISO/IEC_9126

Was nicht getestet ist,

geht kaputt

velocity-tempaltes
messages
model
Contract
getActualEndDateLegacy: Date
getActualEndDateLegacy: Date

Was nicht getestet ist,

geht kaputt

velocity-tempaltes
messages
model
Contract
getActualEndDateLegacy: Date
getActualEndDateLegacy: Date

Was nicht getestet ist,

geht kaputt

Was nicht getestet ist,

geht kaputt

velocity-tempaltes
messages
model
Contract
getActualEndDate: Date
<ul>
  #foreach ( $contract in $subs )
    <li>
    #set($aDate = $dateTool.getDate())
    #if ($contract.getActualEndDate())
      #set($aDate = $contract.getActualEndDateLegacy())
    #end
    $contractMessages.message("notify_change_plan_ends", $dateUtils.format($aDate))
    </li>
  #end
</ul>
notify_change.vm

User

Tested

User

Tested

User

Tested

User

Tested

User

Tested

Was nicht getestet ist,

wird missachtet

Was nicht getestet ist,

wird missachtet

@Test
public void allKeysAreTranslatedInEnglish() {
  List<MessageKey> missingEn = newArrayList();
  for (MessageKey key : translationLoader.getTranslation()) {
    if (key.getValues().get(Locale.GERMAN).hasTranslation() &&
        !key.getValues().get(Locale.ENGLISH).hasTranslation()) {
    	   missingEn.add(key);
       }  
  }
  String message = "These keys have no English translation in " + listKeys(missingEn);
  message += "\n Please provide an English translation";
  assertThat(message, missingEn, hasSize(0));
}

java.lang.AssertionError: These keys have no English translation in messages_en.properties:
Account_Api_CollapseAll
Please provide an English translation
Expected: a collection with size <0>
     but: collection size was <1>

Was nicht getestet ist,

wird missachtet

Frühjahrsputz - für jeden Commit

> Task :price-stock-service:ktlintMainSourceSetCheck FAILED
C:\Users\..\ServerJob.kt:15 Unused import
C:\Users\..\ServerJob.kt:73 Unexpected indentation (18) (should be 16)
C:\Users\..\ServerJob.kt:74 Unexpected indentation (17) (should be 16)
C:\Users\..\ServerJob.kt:74 Exceeded max line length (160) (cannot be auto-corrected)
> gradle ktlint

AUTOMATED

HACK THE

PEOPLE

Tests für schnelles
Feedback

Automatisiert

Tests?

Tests!

Tests!

Tests!

1 Bug

1 Neuer Test

1 Feature

1 Neuer Test

Lesson

Learned

Tests zum Lernen
/ Ausprobieren

Tests als Dokumentation

private final MoneyParser parserDE = new DefaultMoneyParser();

@Test
public void parseCurrencyDE() throws Exception {
  assertThat(parserDE.parseCurrency("20,11"), is(new Money("20.11")));
  assertThat(parserDE.parseCurrency("42100,00"), is(new Money("42100.00")));
  assertThat(parserDE.parseCurrency("42100"), is(new Money("42100")));
  assertThat(parserDE.parseCurrency("42.100"), is(new Money("42100")));
  assertThat(parserDE.parseCurrency("42.100,00"), is(new Money("42100.00")));
  assertThat(parserDE.parseCurrency("42.100,11"), is(new Money("42100.11")));
  assertThat(parserDE.parseCurrency("-42.100,11"), is(new Money("-42100.11")));
}

TO

HOW

Tests als Dokumentation



@Test
public void parseCurrencyDE() throws Exception {
  // Preapre
  MoneyParser parserDE = new DefaultMoneyParser();
  
  // Execute
  Money parsed = parserDE.parseCurrency("20,11");
  
  // Verify
  assertThat(parsed, is(new Money("20.11")));
}

Prepare

Execute

Verify

Given

When

Then

TO

HOW

Tests als Dokumentation


@Test
@DisplayName("Parse German currency")
public void parseCurrencyDE() throws Exception {
  // Prepare
  MoneyParser parserDE = new DefaultMoneyParser();
  
  // Execute
  Money parsed = parserDE.parseCurrency("20,11");
  
  // Verify
  assertThat(parsed, is(new Money("20.11")));
}

TO

HOW

Tests als Dokumentation


@Test
@DisplayName("Parse German currency")
public void parseCurrencyDE() throws Exception {
  // Prepare
  MoneyParser parserDE = new DefaultMoneyParser();
  
  // Execute
  Money parsed = parserDE.parseCurrency("20,11");
  
  // Verify
  assertThat(parsed, is(new Money("20.11")));
}
  • Passes the tests
  • Reveals intention
  • No duplication
  • Fewest elements

Kent Beck
Extreme Programming
Test Driven Development

TO

HOW

Tests als Dokumentation

  • Passes the tests
  • Reveals intention
  • No duplication
  • Fewest elements

TO

HOW

Tests als Dokumentation

  • Passes the tests
  • Reveals intention
  • No duplication
  • Fewest elements
@Test
public void someTest() {
  StringBuilder sb = new StringBuilder();
  sb.append("test");
  sb.append((char) 65);
  sb.append((char) 10);
  sb.append((char) 13);
  sb.append((char) 127);
  for (int i = 0; i < 31; i++) {
    sb.append((char) i);
  }
  assertTrue(MessageUtil.foo(sb.toString()).indexOf(10) > -1);
  assertTrue(MessageUtil.foo(sb.toString()).indexOf(13) > -1);
  assertEquals(MessageUtil.foo(sb.toString()).indexOf(3), -1);
  assertEquals(MessageUtil.foo(sb.toString()).indexOf(15), -1);
  assertEquals(MessageUtil.foo(sb.toString()).indexOf(28), -1);
}

TO

HOW

Tests als Dokumentation

  • Passes the tests
  • Reveals intention
  • No duplication
  • Fewest elements
@Test
public void removeCtrlChars_parameterTest() {
  StringBuilder sb = new StringBuilder();
  sb.append("test");
  sb.append((char) 65);
  sb.append((char) 10);
  sb.append((char) 13);
  sb.append((char) 127);
  for (int i = 0; i < 31; i++) {
    sb.append((char) i);
  }
  assertTrue(MessageUtil.removeCtrlChars(sb.toString()).indexOf(10) > -1);
  assertTrue(MessageUtil.removeCtrlChars(sb.toString()).indexOf(13) > -1);
  assertEquals(MessageUtil.removeCtrlChars(sb.toString()).indexOf(3), -1);
  assertEquals(MessageUtil.removeCtrlChars(sb.toString()).indexOf(15), -1);
  assertEquals(MessageUtil.removeCtrlChars(sb.toString()).indexOf(28), -1);
}

TO

HOW

Tests als Dokumentation

  • Passes the tests
  • Reveals intention
  • No duplication
  • Fewest elements
@Test
public void removeCtrlCharsKeepsNewLine() {
  String newLine = String.valueOf((char) 10); // new line (/n)
  String cleaned = MessageUtil.removeCtrlChars(newLine);
  assertTrue(MessageUtil.removeCtrlChars(newLine).contains(newLine));
}

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertFalse(cleaned.contains(shiftIn));
}

TO

HOW

Aussagekräftige Fehler

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertFalse(cleaned.contains(shiftIn));
}
java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertFalse(Assert.java:65)
	at de.itscope.mv.service.plcrawler.MessageUtilTest.
        removeCtrlCharsRemovesShiftIn(MessageUtilTest.java:39)

Lesson

Learned

Aussagekräftige Fehler

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertThat(cleaned, is(emptyString()));
}
java.lang.AssertionError: 
Expected: is an empty string
     but: was ""

Hamcrest

Lesson

Learned

Aussagekräftige Fehler

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertThat("Shift control symbol should be removed",
		  cleaned, is(emptyString()));
}
java.lang.AssertionError: Shift control symbol should be removed
Expected: is an empty string
     but: was ""

Hamcrest

Lesson

Learned

Randfälle Testen

Ein Software-Tester
läuft in eine Bar und

Jasmine Harpley - ministryoftesting.com

schlendert in die Bar...

gallopiert in die Bar...

rennt in die Bar...

spaziert in die Bar...

TO

HOW

Randfälle Testen

Jokin Aspiazu - ministryoftesting.com

Ein Tester
läuft in eine
Bar und
bestellt
-1 Bier.

TO

HOW

Randfälle Testen

@Test
public void parseEmpty() {
  MoneyParser parserDE = new DefaultMoneyParser();
  assertThrows(ParseException.class, () -> parserDE.parseCurrency(""));
}
@Test
public void parseNull() {
  MoneyParser parserDE = new DefaultMoneyParser();
  assertThrows(ParseException.class, () -> parserDE.parseCurrency(null));
}
@Test
public void parseNull() {
  MoneyParser parserDE = new DefaultMoneyParser();
  assertThat(parserDE.parseCurrency("4.20"), is(new Money("420")));
}

TO

HOW

Unit

Service

UI

Automatisiert

langsam

schnell

mehr Integration

mehr Isolation

Die Testpyramide

Unit

Service

UI

Automatisiert

API Tests

Integrations Tests

Component Tests

Die Testpyramide

Unit

Service

UI

Automatisiert

Die Testpyramide

Service wird hochgefahren

Test von Service-Funktionen, bei dem andere Services gemockt sind

User Ähnlichen Case, der durch
mehrere Codestellen
(Klassen, Funktionen) durchführt

Funktionalität einer Service-Methode im Zusammenspiel mit allen anderen Komponenten sicherstellen

Unit

Service

UI

Automatisiert

Die Testpyramide

Unit

Service

UI

Manuelle Tests

Die Testpyramide

Unit-Tests - klein und schnell

public class MoneyParserTest

Die 100 schnellsten aus 128 in Modul

max: 0,8 s

∅ 60 ms

Das Testprotokoll

Lesson
Learned

Unit

Service

UI

Manuelle Tests

AUTOMATED

Die Ampel im Büro

4 Fehlgeschlagene Tests

Lesson

Learned

Test Step blockiert       merge

Lesson

Learned

Mocks

Lesson
Learned

or no Mocks

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
    //...
  }
 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
    //...
  }
 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
    val status: OrderStatus = placeOrder(product)
    if (status == OrderStatus.OK) {
      emailService.sendMail("You ordered $product")
    }
    return status
  }

  private fun placeOrder(product: Product): OrderStatus {
    return OrderStatus.OK
  }
 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
@Test
fun testOrderService() {
  val orderService = OrderService(emailService = ???)
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Mocks

or no Mocks

@Test
fun testOrderService() {
  val orderService = OrderService(emailService = object : IEmailService {
    override fun sendMail(message: String) {
        // ignore
    }
  })
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}
class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

Lesson

Learned

Mocks

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
@Test
fun testOrderService() {
  val orderService = OrderService(emailService = Mockito.mock(IEmailService::class.java))
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Lesson

Learned

Mocks

or no Mocks

@Test
fun testOrderService() {
  val orderService = OrderService(...)
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Lesson

Learned

Mocks

or no Mocks

@Test
fun testOrderService() {
  val emailService = Mockito.mock(IEmailService::class.java)
  val orderService = OrderService(emailService)
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Lesson

Learned

Mocks

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}
@Test
fun testOrderService() {
  val orderService = OrderService(emailService = mockk<IEmailService>()))
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Lesson

Learned

Mocks

mockStatic(SessionUtils.class);
mockStatic(JMX.class);
mockStatic(UI.class);
mockStatic(RequestContexts.class);
mockStatic(Activator.class);
mockStatic(RedgiantPlugin.class);
mockStatic(MetricFactory.class);
mockStatic(RedgiantModelActivator.class);

RedgiantModelActivator mockActivator = mock(RedgiantModelActivator.class);
ApplicationContext mockApplicationContext = mock(ApplicationContext.class);
Environment mockEnvironment = mock(Environment.class);
FrameworkRegistry mockFrameworkRegistry = mock(FrameworkRegistry.class);
DOrderProposalDContainer mockProposalItems = mock(DOrderProposalDContainer.class);
IPartnershipAccountCache mockAccountCache = mock(IPartnershipAccountCache.class);
DCompanySettingsItem mockCompanySettingsItem = mock(DCompanySettingsItem.class);
AbstractProperty mockProperty = mock(AbstractProperty.class);
PushConfiguration mockPushConfiguration = mock(PushConfiguration.class);
AbstractProperty mockTriggerProperty = mock(AbstractProperty.class);
IRequestContext mockRequestContext = mock(IRequestContext.class);
Page mockPage = mock(Page.class);
WebBrowser mockWebBrowser = mock(WebBrowser.class);
Customer mockCustomer = mock(Customer.class);
when(mockCustomer.isShopCustomer()).thenReturn(false);
IAccountCacheService mockAccountCacheService = Mockito.mock(IAccountCacheService.class);
RedgiantPlugin mockRedgiantPlugin = mock(RedgiantPlugin.class);
UserProfile mockProfile = new UserProfile(mock(User.class), mockCustomer);

MOCKS

EVERYWHERE MOCKS

CartOptimizationManagerTest

Lesson

Learned

Mocks von Pojos

@Test
fun mockPojoTest() {
  val product = mock(JProduct::class.java)
  `when`(product.name).thenReturn("mocked")
  
  assertEquals(product.name,  "mocked")
}

Lesson

Learned

Mocks von Pojos

@Test
fun mockPojoTest() {
  val product = mock(JProduct::class.java)
  `when`(product.name).thenReturn("mocked")
  
  assertEquals(product.name,  "mocked")
  
  product.name = "new"
  
  assertEquals(product.name,  "new")
}

Lesson

Learned

Mocks static mit PowerMock 

fun isWeekday(): Boolean {
  return when(LocalDateTime.now().dayOfWeek){
    DayOfWeek.SUNDAY,DayOfWeek.SATURDAY -> false
    else -> true
  }
}

Lesson

Learned

Mocks static mit PowerMock 

@RunWith(PowerMockRunner.class)
@PrepareForTest(LocalDateTime.class)
public class MockTimeTest {

@Test
public void testLocalDateTime() {
  mockStatic(LocalDateTime.class);
  LocalDateTime now = LocalDateTime.of(2023, 2, 8, 19, 0);
  when(LocalDateTime.now()).thenReturn(now);
  PowerMockito.stub(PowerMockito.method(LocalDateTime.class, "now")).toReturn(now);
  
  LocalDateTime received = LocalDateTime.now();
  
  assertEquals(now, received);
}

Lesson

Learned

@Test
public void testLocalDateTime(){
  try (MockedStatic<?> mocked = mockStatic(LocalDateTime.class)) {
    LocalDateTime now = LocalDateTime.of(2023, 2, 8, 19, 0);
    mocked.when(LocalDateTime::now).thenReturn(now);
    
    LocalDateTime received = LocalDateTime.now();
    
    assertEquals(now, received);
  }
}

Mocks static mit mockito 

 A fatal error has been detected by the Java Runtime Environment:

 

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffaf734ac26, pid=32216, tid=22264

Lesson

Learned

fun isWeekdayDefault(now: LocalDateTime = LocalDateTime.now()): Boolean {
  return when(now.dayOfWeek){
    DayOfWeek.SUNDAY,DayOfWeek.SATURDAY -> true
    else -> false
  }
}

Testen ohne Mocks

Lesson

Learned

Testen mit Reflections

Tip

private long getNumberOfNonStaticFieldsInClass(Class<?> clazz) {
  return Arrays.stream(clazz.getDeclaredFields())
               .filter(e -> !e.getName().equals("serialVersionUID"))
               .filter(e -> !Modifier.isStatic(e.getModifiers()))
               .count();
}
@Override
public MarginRule copyEntity() {
  MarginRule marginRule = new MarginRule();
  marginRule.setSuppliers(suppliers);
  marginRule.setProductTypes(productTypes);
  marginRule.setManufacturers(manufacturers);
  marginRule.setSurcharge(surcharge);
  marginRule.setComment(comment);
  return marginRule;
}
public void fieldsCheckFor_MarginRule() {
  Class<MarginRule> clazz = MarginRule.class;

  long numberOfFieldsInClass = getNumberOfNonStaticFieldsInClass(clazz);
  
  assertThat("You added/removed fields in this tracked entity!" +
            "Did you take care of #copyEntity()?"
            "If so, adjust 'is' value here ->", numberOfFieldsInClass, is(5));
}

Logging in Tests

JA

NEIN

Lesson

Learned

Logging in Tests

H2

Lesson

Learned

Logging in Tests

public void testCompanyService(){
  Log log = getLog(ThreadUtils.class);
  CompanyService companyService = new CompanyService();
  Company testCompany = new Company("test");
  companyService.saveCompany(testCompany);
  
  Company loaded = companyService.loadCompanyByName("test");
  
  log.debug("Loaded Company: " + loaded);
  
  assertEquals(testCompany, loaded);
}

Lesson

Learned

Logging in Tests

public void testCompanyService(){
  Log log = getLog(ThreadUtils.class);
  CompanyService companyService = new CompanyService();
  Company testCompany = new Company("test");
  companyService.saveCompany(testCompany);
  
  Company loaded = companyService.loadCompanyName("test");
  
  if(log.isDebugEnabled()){
    log.debug("Loaded Company: " + loaded);
  }
  
  assertEquals(testCompany, loaded);
}

Lesson

Learned

Logging in Tests

@Before
public void setUp() {
  TestLogUtil.disableLogForClass(AbstractJacksonWriter.class);
}

public static Log disableLogForClass(Class<?> classWithLogger) {
  Log logMock = PowerMockito.mock(Log.class);
  Whitebox.setInternalState(classWithLogger, logMock);
  return logMock;
}
  Log logMock = PowerMockito.mock(Log.class);
  Whitebox.setInternalState(classWithLogger, logMock);
  return logMock;
}

Lesson

Learned

Approval-Tests

@Test
public void testResetPasswordDE() {
  EmailDef emailDef = new EmailDefMock(EmailTemplate.Password.reset);

  ResetPasswordDTO dto = new ResetPasswordDTO("Sehr geehrter Herr Müller", 
                                               "mueller@mycomapany.de", 
                                                resetPasswordLink);

  EmailConfiguration emailConfig = new EmailConfigurationMock(dto, Locale.GERMANY);
  EmailConfiguration email = emailDef.createEmail(emailConfig);
  String mail = email.getBody();

  Approvals.verifyHtml(mail);
}

Lesson

Learned

Approval-Tests

Lesson

Learned

Architektur-Tests
mit ArchUnit

LinkFactory
ServiceProvider
CompanyService
OrderService

Tool-Tip

Architektur-Tests
mit ArchUnit

Tool-Tip

@Test
public void tesRedgiantLinkFactoryNotUsesRedgiantPlugin() {
  JavaClasses redgiantLinkFactory = new ClassFileImporter()
                                       .importClasses(LinkFactory.class);
  
  ArchRule rule = ArchRuleDefinition
     .theClass(LinkFactory.class)
     .should()
     .onlyDependOnClassesThat()
     .areNotAssignableTo(ServerProvider.class)
     .because("It's not allowed to use Services." +
              " Instead provide them as parameter.");
              
  rule.check(redgiantLinkFactory);
}
LinkFactory
ServiceProvider

Architektur-Tests
mit ArchUnit

new Label(getContactInformationHtml(contactDetails), ContentMode.HTML);

Tool-Tip

Architektur-Tests
mit ArchUnit

new Label(getContactInformationHtml(contactDetails), ContentMode.HTML);

new HtmlLabel(getContactInformationHtml(contactDetails), SanitizeMode.NORMAL);

Tool-Tip

Service Tests

Unit

Service

UI

You are here

Unit

Service

UI

You are here

Service Tests

Service Tests

SpringApp

Unit-Tests

Integrations-Tests

Selenium-Tests

Technik
Check

Service Tests

SpringApp

Unit-Tests

Integrations-Tests

Selenium-Tests

MariaDB

ElasticSearch

Redis

Such-Service

Image-Service

Tracking-Service

integration-test

Technik

Check

Service Tests

SpringApp

Unit-Tests

Integrations-Tests

Selenium-Tests

MariaDB

ElasticSearch

Redis

Such-Service

Image-Service

Tracking-Service

integration-test
AbstractIntegrationTest
createTestEnvironment()
initCompany()
initSession()

Technik

Check

Service Tests

Unit-Tests

Integrations-Tests

Selenium-Tests

integration-test
AbstractIntegrationTest
createTestEnvironment()
initCompany()
initSession()

AllTestsSuite
SofortPaymentTest
EmailPollerTest

erbt

erbt

führt aus

JUnit

Technik

Check

Service Tests

Integration-Test-UI

x

Technik

Check

Service Tests

PRO

CONS

  • Schnelle Tests
  • Verhalten (fast) wie live
  • Testsetup für
    für neue Test klein
  • Serverstart nötig
  • Lokale Ausführung
    nur über Web-UI
  • Kein Test-Scope, da alle Services mitstarten
  • Test Isolation aufwendig

Lesson

Learned

CompanyRepository
CompanyService
CompanyServiceAPI
Integration-Test
Company
CompanyService
UserService
CartService
UserService
UserService

TO

HOW

Maria

DB

Spring-Boot Test

Service Tests mit

CompanyRepository
CompanyService
CompanyServiceAPI
Integration-Test
Company
CompanyService
UserService
CartService
UserService
UserService

TO

HOW

Maria

DB

Spring-Boot Test

Service Tests mit

CompanyRepository
CompanyService
CompanyServiceAPI
Integration-Test
Company
CompanyService
UserService
CartService
UserService
UserService

TO

HOW

Maria

DB

Spring-Boot Test

Service Tests mit

CompanyRepository
CompanyService
CompanyServiceAPI
Integartion-Test
Company
CompanyService

TO

HOW

Maria

DB

H2

Spring-Boot Test

Service Tests mit

CompanyRepository
CompanyService
CompanyServiceAPI
Integartion-Test
Company
CompanyService

TO

HOW

H2

Spring-Boot Test

Service Tests mit

TO

HOW

Spring-Boot Test

Service Tests mit

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { StudentJpaConfig.class }, 
                      loader = AnnotationConfigContextLoader.class),
@PropertySource("h2-datasource.properties")
@Transactional
public class InMemorySampleDBTest {

  @Resource
  private StudentRepository studentRepository;

  @Test
  public void givenStudent_whenSave_thenGetOk() {
    Student student = new Student(1, "john");
    studentRepository.save(student);

    Optional<Student> result = studentRepository.findById(1L);
    Student found = result.get();
    assertEquals("john", found.getName());
  }
}
jdbc.driverClassName = org.h2.Driver
jdbc.url = jdbc:h2:mem:myDb

hibernate.dialect = org.hibernate.dialect.H2Dialect
hibernate.hbm2ddl.auto = create

h2-datasource.properties

JUnit

Testcontainer

Service Tests mit

TO

HOW

JUnit 

@BeforeTestClass
fun initMariaDB() {
  val dbContainer: JdbcDatabaseContainer<*> = MariaDBContainer(
      DockerImageName.parse("mariadb").withTag("10.4.12"))
}
testImplementation("org.testcontainers:testcontainers-bom:1.17.6")
testImplementation('org.testcontainers:mariadb')

Testcontainer

Service Tests mit

TO

HOW

JUnit 

@BeforeTestClass
fun initMariaDB() {
  val dbContainer: JdbcDatabaseContainer<*> = MariaDBContainer(
      DockerImageName.parse("mariadb").withTag("10.4.12"))
       .withExposedPorts(3306)
       .withUsername("user")
       .withPassword("jug")
}
testImplementation("org.testcontainers:testcontainers-bom:1.17.6")
testImplementation('org.testcontainers:mariadb')

Testcontainer

Service Tests mit

TO

HOW

JUnit

Testcontainer

Service Tests mit

TO

HOW

INFO o.t.d.DockerClientProviderStrategy       : Found Docker environment with local Npipe socket (npipe:////./pipe/docker_engine)
INFO org.testcontainers.DockerClientFactory   : Docker host IP address is localhost
INFO org.testcontainers.DockerClientFactory   : Connected to docker: 
  Server Version: 20.10.17
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 25605 MB
INFO 🐳 [testcontainers/ryuk:0.3.4]           : Creating container for image: testcontainers/ryuk:0.3.4
INFO 🐳 [testcontainers/ryuk:0.3.4]           : Container testcontainers/ryuk:0.3.4 is starting: 7985c01359f682cf674c2e6406ae70e9854ade81577590bf5a37b7bdbfbbbe44
INFO 🐳 [testcontainers/ryuk:0.3.4]           : Container testcontainers/ryuk:0.3.4 started in PT1.6241543S
INFO o.t.utility.RyukResourceReaper           : Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
INFO org.testcontainers.DockerClientFactory   : Checking the system...
INFO org.testcontainers.DockerClientFactory   : ✔︎ Docker server version should be at least 1.6.0
INFO 🐳 [mariadb:10.4.12]                     : Creating container for image: mariadb:10.4.12
INFO 🐳 [mariadb:10.4.12]                     : Container mariadb:10.4.12 is starting: 0d038e95a9f48ef7169ce0542626a91dd72fc32f8cc5639e9c59314ca7fd7f13
INFO 🐳 [mariadb:10.4.12]                     : Waiting for database connection to become available at jdbc:mariadb://localhost:63608/test using query 'SELECT 1'
INFO 🐳 [mariadb:10.4.12]                     : Container is started (JDBC URL: jdbc:mariadb://localhost:63608/test)
INFO 🐳 [mariadb:10.4.12]                     : Container mariadb:10.4.12 started in PT23.6436749S
CompanyRepository
CompanyService
CompanyServiceAPI
Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository
CompanyService
CompanyServiceAPI
Integartion-Test
Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository
CompanyService
CompanyServiceAPI
End-2-End-Test
Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

HTTP

CompanyRepository
CompanyService
CompanyServiceAPI
Unit-Test
Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository
CompanyService
CompanyServiceAPI
API-Test
Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository
CompanyService
CompanyServiceAPI
API-Test
Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

mocked

CompanyRepository
CompanyService
CompanyServiceAPI
Company

Tests, which Rock

Layered Service

Lesson

Learned

H2

Repository-Test

Lesson

Learned

DOJO Training

Lesson

Learned

  est    riven    evelopment

T

D

D

DOJO Training

Lesson

Learned

T

D

D

Test schlägt fehl

Test läuft grün

Refactoring

DOJO Training

Lesson

Learned

T

D

D

Test schlägt fehl

Test läuft grün

Refactoring

< 10 min

DOJO Training

Lesson

Learned

DOJO Training

Regel Nummer 1

Es darf erst mit dem Produktionscode angefangen werden wenn es einen fehlerhaften Unit Test dazu gibt.

UncleBob

Regel Nummer 2

Der Unit Test darf nur soviel erweitert werden, bis er fehlschlägt (oder nicht mehr compiliert). 

Regel Nummer 3

Der Produktionscode darf nur um soviel erweitert werden, bis der Unit Test grün läuft.

T

D

D

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

Service B

API

Scenario: pathMatches('/storefront/checkout') && methodIs('get')
          * def response = read('json/checkoutResult.json')

Karate Labs

Test Automation
Made Simple.

Lesson

Learned

Micro Service

Schnittstellen Test

Karate Labs

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Feature: Check Cart has correct Supplier Name

  Scenario: Cart has correct Supplier Name

    Given url host + '/storefront/carts/'
    
    When method GET
    
    Then status 200
    * def cart = get response $.supplierCarts.[0]
    * def supplierName = get cart $.supplierName
    Then match expectedSupplierName == supplierName

cart_test.feature

Service A

Service B

Karate Labs

PortalPurchasingBoardTestSf_cartHasSupplierName.feature

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Service A

Service B

Karate Labs

FixPriceAutomationPortalOrderTestSf
CartTest.java
Map<String, Object> karateArgs = new HashMap<>();
karateArgs.put("productId", testProduct.getProductIdAsString());
karateArgs.put("supplierId", testSupplier.getSupplierId());
karateArgs.put("qty", 1);

Suite suite = new Suite();
Feature feature = Feature.read(new File("cart_test.feature"));
FeatureRuntime featureRuntime = FeatureRuntime.of(suite, feature, vars);
featureRuntime.run();

assertEquals(featureRuntime.result.getErrorMessages(), 0, 
              featureRuntime.result.getFailedCount());

Lesson

Learned

Micro Service

Schnittstellen Test

Scenario: pathMatches('/storefront/checkout') && methodIs('get')
          * def response = read('json/checkoutResult.json')
endpoint.feature
{
  "id":"ALFBQU-211112-461",
  "orderNumber": "NH1-211110-697",
  "customerNumber": "0001",
  "orderLimitExceeded": false,
  "showCustomerOrderId": true,
  "customerOrderIdRequired": false,
  "billingAddress": {
    "id": "9e4bfcc9-9a84-48de-b0ed-befb051b6108",
    "companyName": "ITscope GmbH",
    "addition1": "Team Vertrieb Patrick Auth",
    "addition2": "",
    "street": "Ludwig-Erhard-Allee 202",
    "zipCode": "76131",
    "city": "Karlsruhe",
    "postBox": "",
    "iso3country": "FRA",
    "phone": "123456",
    "fax": "123456",
    "email": "",
    "clientNumber": "",
    "url": "",
    "isPrimaryAddress": false,
    "isDefaultBillingAddress": true,
    "isDefaultDeliveryAddress": true
  }
}

checkoutResult.json

Service A

API

Service A

Service B

Karate Labs

mocked

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Service A

Service B

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

data class Stock(
    val amount: Int,
    val unlimited: Boolean,
    val deliveryDate: LocalDate?,
)

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

<dependency>
    <groupId>de.itscope.catalog</groupId>
    <artifactId>price-stock-api</artifactId>
    <version>1.8.3</version>
</dependency>

pom.xml

Weniger zu testen

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Service A

Service B

import

<dependency>
    <groupId>de.itscope.catalog</groupId>
    <artifactId>price-stock-api</artifactId>
    <version>1.8.3</version>
</dependency>

pom.xml

Kotlin

Typescript

Vue

Backend zu Frontend
win win

Lesson

Learned

Micro Service

Schnittstellen Test

publishing {
    repositories {
        maven {
            url = Publishing.nexusTargetRepo
            isAllowInsecureProtocol = true
            credentials {
                username = Repository.Nexus.Credentials.username
                password = Repository.Nexus.Credentials.password
            }
        }
    }

    publications.create("mavenJava", MavenPublication::class) {
        from(components["java"])
        artifact(sourcesJar.get())
        pom {
            name.set(artifactId)
            developers {
                developer {
                    id.set("its")
                    name.set("ITscope - Developer")
                }
            }
        }
    }
}
build.gradle.kts

Unit

Service

UI

You are Here

UI Tests

Unit

Service

UI

You are Here

UI Tests

Unit

Service

UI

UI Tests

Unit

Service

UI

UI Tests

RedGiant

2.562

Unit

335

Integration

801

Selenium

Unit

Service

UI

UI Tests

0:27

Unit

0:03

Integration

1:05

Selenium

Build Pipeline

#107

#108

#109

#110

#111

#112

Wackler

Wackler

Wackler

Selenium Tests

How
To

How
To

REST
API

doTestSetup()

Selenium Tests

Selenium Tests

-SideApplicationFrameLayout-GlobalSearchContentView-ContextPanelDetailView
-masterDetailLayout-ProductSmallDetailView-ProductSmallHeaderView
-AddToCartWidget-AddToCartMenuBar
getDriver().findElement(By.className("button"));
getDriver().findElement(By.name("In den Warenkorb"));
getDriver().findElement(By.id("..AddToCartMenuBar.."));

TIP

Selenium Tests

TIP

Selenium Tests

// Open Product Page
IExtendedProduct product = TestProductService.getDefault();

FrontendProductPage productPage = application.openPage().productPage(product);

// Check Product Page Header
assertThat(productPage.getHead().getTitle(), hasText(product.getName()));
assertThat(productPage.getHead().getEan(), hasText(product.getEan()));
Application
ProductPage
Head
Title

PageObjectPattern

Ham-Crest-Matcher

Typisierte Test-Daten

TIP

Selenium Tests

resellerApplication.action().createNewCartInPortal("New Cart");
CartRow newCart = portal.getCartTable().getCartByName("New Cart");
newCart.action().addProduct(appleiPadAir);
newCart.action().shareFirstLineItem();
newCart.action().addNegotiationPartner(distributorUser.getUserName());
SearchPage<?> searchPage = openPage().globalSearchNegotiationPage();
SellerResponsePage<?> sellerResponsePage = 
    searchPage.openFirstSellerResponseFromCompany(getCompany());
sellerResponsePage.getFirstRow().setPrice("1.00");
CartColumnTableRow supplierCard = 
    newCart.getCartColumn().getCartTable().getFirstSupplierCard();
supplierCard = selectNegotiationPosition(1, newCart, supplierCard);

PriceRequestTest

TIP

Selenium Tests

Testschritte benennen

// #1 (Buyer) create new cart and open it
resellerApplication.action().createNewCartInPortal("New Cart");

// #2 (Buyer) add Product and start negotiation with distri Also
CartRow newCart = portal.getCartTable().getCartByName("New Cart");
newCart.action().addProduct(appleiPadAir);
newCart.action().shareFirstLineItem();
newCart.action().addNegotiationPartner(distributorUser.getUserName());

// #3 (distri) set a price
SearchPage<?> searchPage = openPage().globalSearchNegotiationPage();
SellerResponsePage<?> sellerResponsePage = 
    searchPage.openFirstSellerResponseFromCompany(getCompany());
sellerResponsePage.getFirstRow().setPrice("1.00");

// #4 Select negotiation price
CartColumnTableRow supplierCard = 
    newCart.getCartColumn().getCartTable().getFirstSupplierCard();
supplierCard = selectNegotiationPosition(1, newCart, supplierCard);

PriceRequestTest

TIP

Selenium Tests

REST
API

doTestSetup()

TIP

Selenium Tests

REST
API

doTestSetup()
getErrorLog()

TIP

Selenium Tests

REST
API

doTestSetup()
getErrorLog()

TIP

Selenium Tests

1:05

Selenium

NEED FOR SPEED

  • Für neue Tests Unit Tests präferiert
  • Selenium Tests nur noch bei Kommentar
  • Stabilisierung der Tests durch bessere Isolation durch Refactoring
  • Feintuning der Parallelität
    Stabilität VS Performance
  • Austausch der DB von
    Live-Snapshot zu Mini-DB
  • Heraustrennung von Bereichen in Micro-Services und Vue-Anwendung

0:24

Selenium

TIP

Selenium Tests

1:05

Selenium

NEED FOR SPEED

0:24

Selenium

TIP

Selenium Tests

Tips

  • Page-Object-Pattern als UI Abstraktion
  • Test-Daten typisieren
  • Hamcrest-Matcher für prägnaten Code
    und lesbare Fehlerausgaben
  • Gute Inline-Dokumentation der Einzelschritte
  • Automatisches Wait + Retry bei Driver Zugriff
  • REST API für Backend Zugriff
  • UI Tests ausdünnen, migrieren oder heraustrennen

Lesson

Learned

Lesson Learned

Lesson Learned

Wähle die richtige Etage
Denke zuerst an die Tests
Regelmäßiges
Learn & Adapt
Passes the tests
Reveals intention
Nehmt euch Zeit
fürs Testen
Was nicht getestet ist,
geht kaputt

feed.tobse.eu

https://slides.com/
tobsefritz/test-pyramid

Folien

Feedback

DANKE