Liebling, ich habe die Testpyramide auf den Kopf gestellt

JUG Karlsruhe

Tobse Fritz

Reich
Gut aussehend
ITscope GmbH
Kotlin Fanboy
Java Entwickler
Clean Coder
Hobby Fotograf
Gaming Nerd
3D Printer
Daddy

https://tobse.eu

JUG Karlsruhe

Lesson
Learned

HOW
TO

Technik
Check

DOS

DONT'S

&

Warum Testen?

Die Test-Pyramide

https://stackshare.io/itscope-gmbh/

SpringBoot

MariaDB

ElasticSearch

Redis

Such-Service

Image-Service

Tracking-Service

Price-Service

Vue.js

RedGiant

Technik
Check

FRONTEND

BACKEND

FRONTEND

BACKEND

TESTS

SOFTWARE QUALITÄT

https://de.wikipedia.org/wiki/ISO/IEC_9126

Was nicht getestet ist,

geht kaputt

velocity-tempaltes

messages

model

Contract

getActualEndDateLegacy: Date

getActualEndDateLegacy: Date

Was nicht getestet ist,

geht kaputt

velocity-tempaltes

messages

model

Contract

getActualEndDate: Date

Was nicht getestet ist,

geht kaputt

velocity-tempaltes

messages

model

Contract

getActualEndDate: Date

<ul>
  #foreach ( $contract in $subs )
    <li>
    #set($aDate = $dateTool.getDate())
    #if ($contract.getActualEndDate())
      #set($aDate = $contract.getActualEndDateLegacy())
    #end
    $contractMessages.message("notify_change_plan_ends", $dateUtils.format($aDate))
    </li>
  #end
</ul>

notify_change.vm

User

Tested

User

Tested

User

Tested

User

Tested

User

Tested

Was nicht getestet ist,

wird missachtet

Was nicht getestet ist,

wird missachtet

@Test
public void allKeysAreTranslatedInEnglish() {
  List<MessageKey> missingEn = newArrayList();
  for (MessageKey key : translationLoader.getTranslation()) {
    if (key.getValues().get(Locale.GERMAN).hasTranslation() &&
        !key.getValues().get(Locale.ENGLISH).hasTranslation()) {
    	   missingEn.add(key);
       }  
  }
  String message = "These keys have no English translation in " + listKeys(missingEn);
  message += "\n Please provide an English translation";
  assertThat(message, missingEn, hasSize(0));
}


java.lang.AssertionError: These keys have no English translation in messages_en.properties:
Account_Api_CollapseAll
Please provide an English translation
Expected: a collection with size <0>
     but: collection size was <1>

Was nicht getestet ist,

wird missachtet

Tests für schnelles
Feedback

Automatisiert

Tests zum Lernen
/ Ausprobieren

Tests?

Tests!

1 Bug

1 Neuer Test

1 Feature

1 Neuer Test

Lesson

Learned

Unit

Service

UI

Automatisiert

langsam

schnell

mehr Integration

mehr Isolation

Die Testpyramide

Unit

Service

UI

Automatisiert

API Tests

Integrations Tests

Component Tests

Die Testpyramide

Unit

Service

UI

Automatisiert

Die Testpyramide

Service wird hochgefahren

Test von Service-Funktionen, bei dem andere Services gemockt sind

User Ähnlichen Case, der durch
mehrere Codestellen
(Klassen, Funktionen) durchführt

Funktionalität einer Service-Methode im Zusammenspiel mit allen anderen Komponenten sicherstellen

Unit

Service

UI

Automatisiert

Die Testpyramide

Unit

Service

UI

Manuelle Tests

Die Testpyramide

Das Testprotokoll

Lesson
Learned

Unit

Service

UI

Manuelle Tests

AUTOMATED

Die Ampel im Büro

4 Fehlgeschlagene Tests

Lesson

Learned

Test Step blockiert merge

Lesson

Learned

Test Step blockiert merge

Lesson

Learned

Frühjahrsputz - für jeden Commit

> Task :price-stock-service:ktlintMainSourceSetCheck FAILED
C:\Users\..\ServerJob.kt:15 Unused import
C:\Users\..\ServerJob.kt:73 Unexpected indentation (18) (should be 16)
C:\Users\..\ServerJob.kt:74 Unexpected indentation (17) (should be 16)
C:\Users\..\ServerJob.kt:74 Exceeded max line length (160) (cannot be auto-corrected)

> gradle ktlint

AUTOMATED

HACK THE

PEOPLE

Tests als Dokumentation

private final MoneyParser parserDE = new DefaultMoneyParser();

@Test
public void parseCurrencyDE() throws Exception {
  assertThat(parserDE.parseCurrency("20,11"), is(new Money("20.11")));
  assertThat(parserDE.parseCurrency("42100,00"), is(new Money("42100.00")));
  assertThat(parserDE.parseCurrency("42100"), is(new Money("42100")));
  assertThat(parserDE.parseCurrency("42.100"), is(new Money("42100")));
  assertThat(parserDE.parseCurrency("42.100,00"), is(new Money("42100.00")));
  assertThat(parserDE.parseCurrency("42.100,11"), is(new Money("42100.11")));
  assertThat(parserDE.parseCurrency("-42.100,11"), is(new Money("-42100.11")));
}

TO

HOW

Tests als Dokumentation



@Test
public void parseCurrencyDE() throws Exception {
  // Preapre
  MoneyParser parserDE = new DefaultMoneyParser();
  
  // Execute
  Money parsed = parserDE.parseCurrency("20,11");
  
  // Verify
  assertThat(parsed, is(new Money("20.11")));
}

Prepare

Execute

Verify

Given

When

Then

TO

HOW

Tests als Dokumentation


@Test
@DisplayName("Parse German currency")
public void parseCurrencyDE() throws Exception {
  // Prepare
  MoneyParser parserDE = new DefaultMoneyParser();
  
  // Execute
  Money parsed = parserDE.parseCurrency("20,11");
  
  // Verify
  assertThat(parsed, is(new Money("20.11")));
}

TO

HOW

Tests als Dokumentation


@Test
@DisplayName("Parse German currency")
public void parseCurrencyDE() throws Exception {
  // Prepare
  MoneyParser parserDE = new DefaultMoneyParser();
  
  // Execute
  Money parsed = parserDE.parseCurrency("20,11");
  
  // Verify
  assertThat(parsed, is(new Money("20.11")));
}

Passes the tests
Reveals intention
No duplication
Fewest elements

Kent Beck
Extreme Programming
Test Driven Development

TO

HOW

Tests als Dokumentation

Passes the tests
Reveals intention
No duplication
Fewest elements

TO

HOW

Tests als Dokumentation

Passes the tests
Reveals intention
No duplication
Fewest elements

@Test
public void someTest() {
  StringBuilder sb = new StringBuilder();
  sb.append("test");
  sb.append((char) 65);
  sb.append((char) 10);
  sb.append((char) 13);
  sb.append((char) 127);
  for (int i = 0; i < 31; i++) {
    sb.append((char) i);
  }
  assertTrue(MessageUtil.foo(sb.toString()).indexOf(10) > -1);
  assertTrue(MessageUtil.foo(sb.toString()).indexOf(13) > -1);
  assertEquals(MessageUtil.foo(sb.toString()).indexOf(3), -1);
  assertEquals(MessageUtil.foo(sb.toString()).indexOf(15), -1);
  assertEquals(MessageUtil.foo(sb.toString()).indexOf(28), -1);
}

TO

HOW

Tests als Dokumentation

Passes the tests
Reveals intention
No duplication
Fewest elements

@Test
public void removeCtrlChars_parameterTest() {
  StringBuilder sb = new StringBuilder();
  sb.append("test");
  sb.append((char) 65);
  sb.append((char) 10);
  sb.append((char) 13);
  sb.append((char) 127);
  for (int i = 0; i < 31; i++) {
    sb.append((char) i);
  }
  assertTrue(MessageUtil.removeCtrlChars(sb.toString()).indexOf(10) > -1);
  assertTrue(MessageUtil.removeCtrlChars(sb.toString()).indexOf(13) > -1);
  assertEquals(MessageUtil.removeCtrlChars(sb.toString()).indexOf(3), -1);
  assertEquals(MessageUtil.removeCtrlChars(sb.toString()).indexOf(15), -1);
  assertEquals(MessageUtil.removeCtrlChars(sb.toString()).indexOf(28), -1);
}

TO

HOW

Tests als Dokumentation

Passes the tests
Reveals intention
No duplication
Fewest elements

@Test
public void removeCtrlCharsKeepsNewLine() {
  String newLine = String.valueOf((char) 10); // new line (/n)
  String cleaned = MessageUtil.removeCtrlChars(newLine);
  assertTrue(MessageUtil.removeCtrlChars(newLine).contains(newLine));
}

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertFalse(cleaned.contains(shiftIn));
}

TO

HOW

Aussagekräftige Fehler

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertFalse(cleaned.contains(shiftIn));
}

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertFalse(Assert.java:65)
	at de.itscope.mv.service.plcrawler.MessageUtilTest.
        removeCtrlCharsRemovesShiftIn(MessageUtilTest.java:39)

Lesson

Learned

Aussagekräftige Fehler

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertThat(cleaned, is(emptyString()));
}

java.lang.AssertionError: 
Expected: is an empty string
     but: was ""

Hamcrest

Lesson

Learned

Aussagekräftige Fehler

@Test
public void removeCtrlCharsRemovesShiftIn() {
  String shiftIn = String.valueOf((char) 15); //  Shift In control (unicode: \u000F)
  String cleaned = MessageUtil.removeCtrlChars(shiftIn);
  assertThat("Shift control symbol should be removed",
		  cleaned, is(emptyString()));
}

java.lang.AssertionError: Shift control symbol should be removed
Expected: is an empty string
     but: was ""

Hamcrest

Lesson

Learned

Randfälle Testen

Ein Software-Tester
läuft in eine Bar und

Jasmine Harpley - ministryoftesting.com

schlendert in die Bar...

gallopiert in die Bar...

rennt in die Bar...

spaziert in die Bar...

TO

HOW

Randfälle Testen

Jokin Aspiazu - ministryoftesting.com

Ein Tester
läuft in eine
Bar und
bestellt
-1 Bier.

TO

HOW

Randfälle Testen

@Test
public void parseEmpty() {
  MoneyParser parserDE = new DefaultMoneyParser();
  assertThrows(ParseException.class, () -> parserDE.parseCurrency(""));
}

@Test
public void parseNull() {
  MoneyParser parserDE = new DefaultMoneyParser();
  assertThrows(ParseException.class, () -> parserDE.parseCurrency(null));
}

@Test
public void parseNull() {
  MoneyParser parserDE = new DefaultMoneyParser();
  assertThat(parserDE.parseCurrency("4.20"), is(new Money("420")));
}

TO

HOW

Unit-Tests - klein und schnell

public class MoneyParserTest

Die 100 schnellsten aus 128 in Modul

max: 0,8 s

∅ 60 ms

Mocks

Lesson
Learned

or no Mocks

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
    //...
  }
 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
    //...
  }
 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
    val status: OrderStatus = placeOrder(product)
    if (status == OrderStatus.OK) {
      emailService.sendMail("You ordered $product")
    }
    return status
  }

  private fun placeOrder(product: Product): OrderStatus {
    return OrderStatus.OK
  }
 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

Mocks

Lesson
Learned

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

@Test
fun testOrderService() {
  val orderService = OrderService(emailService = ???)
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Mocks

or no Mocks

@Test
fun testOrderService() {
  val orderService = OrderService(emailService = object : IEmailService {
    override fun sendMail(message: String) {
        // ignore
    }
  })
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

Lesson

Learned

Mocks

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

@Test
fun testOrderService() {
  val orderService = OrderService(emailService = Mockito.mock(IEmailService::class.java))
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Lesson

Learned

Mocks

or no Mocks

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

class Product(val name: String)

interface IOrderService {
  fun order(product: Product): OrderStatus
}

interface IEmailService {
  fun sendMail(message: String)
}

enum class OrderStatus { OK, ERROR }

class OrderService(private val emailService: IEmailService) : IOrderService {

  override fun order(product: Product): OrderStatus {
  	//...
  } 
}

@Test
fun testOrderService() {
  val orderService = OrderService(emailService = mockk<IEmailService>()))
  val status = orderService.order(Product("IPad"))
  assertEquals(status, OrderStatus.OK)
}

Lesson

Learned

Mocks

mockStatic(SessionUtils.class);
mockStatic(JMX.class);
mockStatic(UI.class);
mockStatic(RequestContexts.class);
mockStatic(Activator.class);
mockStatic(RedgiantPlugin.class);
mockStatic(MetricFactory.class);
mockStatic(RedgiantModelActivator.class);

RedgiantModelActivator mockActivator = mock(RedgiantModelActivator.class);
ApplicationContext mockApplicationContext = mock(ApplicationContext.class);
Environment mockEnvironment = mock(Environment.class);
FrameworkRegistry mockFrameworkRegistry = mock(FrameworkRegistry.class);
DOrderProposalDContainer mockProposalItems = mock(DOrderProposalDContainer.class);
IPartnershipAccountCache mockAccountCache = mock(IPartnershipAccountCache.class);
DCompanySettingsItem mockCompanySettingsItem = mock(DCompanySettingsItem.class);
AbstractProperty mockProperty = mock(AbstractProperty.class);
PushConfiguration mockPushConfiguration = mock(PushConfiguration.class);
AbstractProperty mockTriggerProperty = mock(AbstractProperty.class);
IRequestContext mockRequestContext = mock(IRequestContext.class);
Page mockPage = mock(Page.class);
WebBrowser mockWebBrowser = mock(WebBrowser.class);
Customer mockCustomer = mock(Customer.class);
when(mockCustomer.isShopCustomer()).thenReturn(false);
IAccountCacheService mockAccountCacheService = Mockito.mock(IAccountCacheService.class);
RedgiantPlugin mockRedgiantPlugin = mock(RedgiantPlugin.class);
UserProfile mockProfile = new UserProfile(mock(User.class), mockCustomer);

MOCKS

EVERYWHERE MOCKS

CartOptimizationManagerTest

Lesson

Learned

Mocks von Pojos

@Test
fun mockPojoTest() {
  val product = mock(JProduct::class.java)
  `when`(product.name).thenReturn("mocked")
  
  assertEquals(product.name,  "mocked")
}

Lesson

Learned

Mocks von Pojos

@Test
fun mockPojoTest() {
  val product = mock(JProduct::class.java)
  `when`(product.name).thenReturn("mocked")
  
  assertEquals(product.name,  "mocked")
  
  product.name = "new"
  
  assertEquals(product.name,  "new")
}

Lesson

Learned

Mocks static mit PowerMock

fun isWeekday(): Boolean {
  return when(LocalDateTime.now().dayOfWeek){
    DayOfWeek.SUNDAY,DayOfWeek.SATURDAY -> false
    else -> true
  }
}

Lesson

Learned

Mocks static mit PowerMock

@RunWith(PowerMockRunner.class)
@PrepareForTest(LocalDateTime.class)
public class MockTimeTest {

@Test
public void testLocalDateTime() {
  mockStatic(LocalDateTime.class);
  LocalDateTime now = LocalDateTime.of(2023, 2, 8, 19, 0);
  when(LocalDateTime.now()).thenReturn(now);
  PowerMockito.stub(PowerMockito.method(LocalDateTime.class, "now")).toReturn(now);
  
  LocalDateTime received = LocalDateTime.now();
  
  assertEquals(now, received);
}

Cannot mock/spy class java.time.LocalDateTime
Mockito cannot mock/spy because :
 - final class

Lesson

Learned

@Test
public void testLocalDateTime(){
  try (MockedStatic<?> mocked = mockStatic(LocalDateTime.class)) {
    LocalDateTime now = LocalDateTime.of(2023, 2, 8, 19, 0);
    mocked.when(LocalDateTime::now).thenReturn(now);
    
    LocalDateTime received = LocalDateTime.now();
    
    assertEquals(now, received);
  }
}

Mocks static mit mockito

 A fatal error has been detected by the Java Runtime Environment:

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffaf734ac26, pid=32216, tid=22264

Lesson

Learned

fun isWeekdayDefault(now: LocalDateTime = LocalDateTime.now()): Boolean {
  return when(now.dayOfWeek){
    DayOfWeek.SUNDAY,DayOfWeek.SATURDAY -> true
    else -> false
  }
}

Testen ohne Mocks

Lesson

Learned

Logging in Tests

JA

NEIN

Lesson

Learned

Logging in Tests

H2

Lesson

Learned

Logging in Tests

@Before
public void setUp() {
  TestLogUtil.disableLogForClass(AbstractJacksonWriter.class);
}

public static Log disableLogForClass(Class<?> classWithLogger) {
  Log logMock = PowerMockito.mock(Log.class);
  Whitebox.setInternalState(classWithLogger, logMock);
  return logMock;
}

  Log logMock = PowerMockito.mock(Log.class);
  Whitebox.setInternalState(classWithLogger, logMock);
  return logMock;
}

Lesson

Learned

Logging in Tests

public void testCompanyService(){
  Log log = getLog(ThreadUtils.class);
  CompanyService companyService = new CompanyService();
  Company testCompany = new Company("test");
  companyService.saveCompany(testCompany);
  
  Company loaded = companyService.loadCompanyName("test");
  
  log.debug("Loaded Company: " + loaded);
  
  assertEquals(testCompany, loaded);
}

Lesson

Learned

Logging in Tests

public void testCompanyService(){
  Log log = getLog(ThreadUtils.class);
  CompanyService companyService = new CompanyService();
  Company testCompany = new Company("test");
  companyService.saveCompany(testCompany);
  
  Company loaded = companyService.loadCompanyName("test");
  
  if(log.isDebugEnabled()){
    log.debug("Loaded Company: " + loaded);
  }
  
  assertEquals(testCompany, loaded);
}

Lesson

Learned

Approval-Tests

@Test
public void testResetPasswordDE() {
  EmailDef emailDef = new EmailDefMock(EmailTemplate.Password.reset);

  ResetPasswordDTO dto = new ResetPasswordDTO("Sehr geehrter Herr Müller", 
                                               "mueller@mycomapany.de", 
                                                resetPasswordLink);

  EmailConfiguration emailConfig = new EmailConfigurationMock(dto, Locale.GERMANY);
  EmailConfiguration email = emailDef.createEmail(emailConfig);
  String mail = email.getBody();

  Approvals.verifyHtml(mail);
}

Lesson

Learned

Approval-Tests

Lesson

Learned

Architektur-Tests
mit ArchUnit

RedgiantLinkFactory

RedgiantPlugin

CompanyService

OrderService

Tool-Tip

Architektur-Tests
mit ArchUnit

Tool-Tip

@Test
public void tesRedgiantLinkFactoryNotUsesRedgiantPlugin() {
  JavaClasses redgiantLinkFactory = new ClassFileImporter()
                                       .importClasses(RedgiantLinkFactory.class);
  
  ArchRule rule = ArchRuleDefinition
     .theClass(RedgiantLinkFactory.class)
     .should()
     .onlyDependOnClassesThat()
     .areNotAssignableTo(RedgiantPlugin.class)
     .because("It's not allowed to use Services with the deprecated RedgiantPlugin." +
              " Instead provide them as parameter.");
              
  rule.check(redgiantLinkFactory);
}

RedgiantLinkFactory

RedgiantPlugin

Architektur-Tests
mit ArchUnit

new Label(getContactInformationHtml(contactDetails), ContentMode.HTML);

Tool-Tip

Architektur-Tests
mit ArchUnit

new Label(getContactInformationHtml(contactDetails), ContentMode.HTML);

new HtmlLabel(getContactInformationHtml(contactDetails), SanitizeMode.NORMAL);

Tool-Tip

Service Tests

Unit

Service

UI

You are Here

Unit

Service

UI

You are Here

Service Tests

RedGiant

Unit-Tests

Integrations-Tests

Selenium-Tests

Technik
Check

Service Tests

RedGiant

Unit-Tests

Integrations-Tests

Selenium-Tests

MariaDB

ElasticSearch

Redis

Such-Service

Image-Service

Tracking-Service

integration-test

Technik

Check

Service Tests

RedGiant

Unit-Tests

Integrations-Tests

Selenium-Tests

MariaDB

ElasticSearch

Redis

Such-Service

Image-Service

Tracking-Service

integration-test

AbstractIntegrationTest

createTestEnvironment()
initCompany()
initSession()

Technik

Check

Service Tests

Unit-Tests

Integrations-Tests

Selenium-Tests

integration-test

AbstractIntegrationTest

createTestEnvironment()
initCompany()
initSession()

AllTestsSuite

SofortPaymentTest

EmailPollerTest

erbt

führt aus

JUnit

Technik

Check

Service Tests

Integration-Test-UI

x

Technik

Check

Service Tests

PRO

CONS

Schnelle Tests
Verhalten (fast) wie live
Testsetup für
für neue Test klein

Serverstart nötig
Lokale Ausführung
nur über Web-UI
Kein Test-Scope, da alle Services mitstarten
Test Isolation aufwendig

Lesson

Learned

CompanyRepository

CompanyService

CompanyServiceAPI

Integartion-Test

Company

CompanyService

UserService

CartService

UserService

UserService

TO

HOW

Maria

DB

Spring-Boot Test

Service Tests mit

CompanyRepository

CompanyService

CompanyServiceAPI

Integartion-Test

Company

CompanyService

UserService

CartService

UserService

UserService

TO

HOW

Maria

DB

Spring-Boot Test

Service Tests mit

CompanyRepository

CompanyService

CompanyServiceAPI

Integartion-Test

Company

CompanyService

TO

HOW

Maria

DB

H2

Spring-Boot Test

Service Tests mit

CompanyRepository

CompanyService

CompanyServiceAPI

Integartion-Test

Company

CompanyService

TO

HOW

H2

Spring-Boot Test

Service Tests mit

TO

HOW

Spring-Boot Test

Service Tests mit

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = { StudentJpaConfig.class }, 
                      loader = AnnotationConfigContextLoader.class),
@PropertySource("h2-datasource.properties")
@Transactional
public class InMemorySampleDBTest {

  @Resource
  private StudentRepository studentRepository;

  @Test
  public void givenStudent_whenSave_thenGetOk() {
    Student student = new Student(1, "john");
    studentRepository.save(student);

    Optional<Student> result = studentRepository.findById(1L);
    Student found = result.get();
    assertEquals("john", found.getName());
  }
}

jdbc.driverClassName = org.h2.Driver
jdbc.url = jdbc:h2:mem:myDb

hibernate.dialect = org.hibernate.dialect.H2Dialect
hibernate.hbm2ddl.auto = create

h2-datasource.properties

JUnit

Testcontainer

Service Tests mit

TO

HOW

JUnit

@BeforeTestClass
fun initMariaDB() {
  val dbContainer: JdbcDatabaseContainer<*> = MariaDBContainer(
      DockerImageName.parse("mariadb").withTag("10.4.12"))
}

testImplementation("org.testcontainers:testcontainers-bom:1.17.6")
testImplementation('org.testcontainers:mariadb')

Testcontainer

Service Tests mit

TO

HOW

JUnit

@BeforeTestClass
fun initMariaDB() {
  val dbContainer: JdbcDatabaseContainer<*> = MariaDBContainer(
      DockerImageName.parse("mariadb").withTag("10.4.12"))
       .withExposedPorts(3306)
       .withUsername("user")
       .withPassword("jug")
}

testImplementation("org.testcontainers:testcontainers-bom:1.17.6")
testImplementation('org.testcontainers:mariadb')

Testcontainer

Service Tests mit

TO

HOW

JUnit

Testcontainer

Service Tests mit

TO

HOW

INFO o.t.d.DockerClientProviderStrategy       : Found Docker environment with local Npipe socket (npipe:////./pipe/docker_engine)
INFO org.testcontainers.DockerClientFactory   : Docker host IP address is localhost
INFO org.testcontainers.DockerClientFactory   : Connected to docker: 
  Server Version: 20.10.17
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 25605 MB
INFO 🐳 [testcontainers/ryuk:0.3.4]           : Creating container for image: testcontainers/ryuk:0.3.4
INFO 🐳 [testcontainers/ryuk:0.3.4]           : Container testcontainers/ryuk:0.3.4 is starting: 7985c01359f682cf674c2e6406ae70e9854ade81577590bf5a37b7bdbfbbbe44
INFO 🐳 [testcontainers/ryuk:0.3.4]           : Container testcontainers/ryuk:0.3.4 started in PT1.6241543S
INFO o.t.utility.RyukResourceReaper           : Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
INFO org.testcontainers.DockerClientFactory   : Checking the system...
INFO org.testcontainers.DockerClientFactory   : ✔︎ Docker server version should be at least 1.6.0
INFO 🐳 [mariadb:10.4.12]                     : Creating container for image: mariadb:10.4.12
INFO 🐳 [mariadb:10.4.12]                     : Container mariadb:10.4.12 is starting: 0d038e95a9f48ef7169ce0542626a91dd72fc32f8cc5639e9c59314ca7fd7f13
INFO 🐳 [mariadb:10.4.12]                     : Waiting for database connection to become available at jdbc:mariadb://localhost:63608/test using query 'SELECT 1'
INFO 🐳 [mariadb:10.4.12]                     : Container is started (JDBC URL: jdbc:mariadb://localhost:63608/test)
INFO 🐳 [mariadb:10.4.12]                     : Container mariadb:10.4.12 started in PT23.6436749S

CompanyRepository

CompanyService

CompanyServiceAPI

Company

Tests which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository

CompanyService

CompanyServiceAPI

Integartion-Test

Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository

CompanyService

CompanyServiceAPI

End-2-End-Test

Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository

CompanyService

CompanyServiceAPI

Unit-Test

Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository

CompanyService

CompanyServiceAPI

API-Test

Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

CompanyRepository

CompanyService

CompanyServiceAPI

API-Test

Company

Tests, which Rock

Layered Service

Lesson

Learned

Maria

DB

mocked

CompanyRepository

CompanyService

CompanyServiceAPI

Company

Tests, which Rock

Layered Service

Lesson

Learned

H2

Repository-Test

Lesson

Learned

DOJO Training

Lesson

Learned

est riven evelopment

T

D

DOJO Training

Lesson

Learned

T

D

Test schlägt fehl

Test läuft grün

Refactoring

DOJO Training

Lesson

Learned

T

D

Test schlägt fehl

Test läuft grün

Refactoring

< 10 min

DOJO Training

Lesson

Learned

DOJO Training

Regel Nummer 1

Es darf erst mit dem Produktionscode angefangen werden wenn es einen fehlerhaften Unit Test dazu gibt.

UncleBob

Regel Nummer 2

Der Unit Test darf nur soviel erweitert werden, bis er fehlschlägt (oder nicht mehr compiliert).

Regel Nummer 3

Der Produktionscode darf nur um soviel erweitert werden, bis der Unit Test grün läuft.

T

D

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

Service B

API

Scenario: pathMatches('/storefront/checkout') && methodIs('get')
          * def response = read('json/checkoutResult.json')

Karate Labs

Test Automation
Made Simple.

Lesson

Learned

Micro Service

Schnittstellen Test

Karate Labs

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Feature: Check Cart has correct Supplier Name

  Scenario: Cart has correct Supplier Name

    Given url host + '/storefront/carts/'
    
    When method GET
    
    Then status 200
    * def cart = get response $.supplierCarts.[0]
    * def supplierName = get cart $.supplierName
    Then match expectedSupplierName == supplierName

cart_test.feature

Service A

Service B

Karate Labs

PortalPurchasingBoardTestSf_cartHasSupplierName.feature

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Service A

Service B

Karate Labs

FixPriceAutomationPortalOrderTestSf

CartTest.java

Map<String, Object> karateArgs = new HashMap<>();
karateArgs.put("productId", testProduct.getProductIdAsString());
karateArgs.put("supplierId", testSupplier.getSupplierId());
karateArgs.put("qty", 1);

Suite suite = new Suite();
Feature feature = Feature.read(new File("cart_test.feature"));
FeatureRuntime featureRuntime = FeatureRuntime.of(suite, feature, vars);
featureRuntime.run();

assertEquals(featureRuntime.result.getErrorMessages(), 0, 
              featureRuntime.result.getFailedCount());

Lesson

Learned

Micro Service

Schnittstellen Test

Scenario: pathMatches('/storefront/checkout') && methodIs('get')
          * def response = read('json/checkoutResult.json')

endpoint.feature

{
  "id":"ALFBQU-211112-461",
  "orderNumber": "NH1-211110-697",
  "customerNumber": "0001",
  "orderLimitExceeded": false,
  "showCustomerOrderId": true,
  "customerOrderIdRequired": false,
  "billingAddress": {
    "id": "9e4bfcc9-9a84-48de-b0ed-befb051b6108",
    "companyName": "ITscope GmbH",
    "addition1": "Team Vertrieb Patrick Auth",
    "addition2": "",
    "street": "Ludwig-Erhard-Allee 202",
    "zipCode": "76131",
    "city": "Karlsruhe",
    "postBox": "",
    "iso3country": "FRA",
    "phone": "123456",
    "fax": "123456",
    "email": "",
    "clientNumber": "",
    "url": "",
    "isPrimaryAddress": false,
    "isDefaultBillingAddress": true,
    "isDefaultDeliveryAddress": true
  }
}

checkoutResult.json

Service A

API

Service A

Service B

Karate Labs

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Service A

Service B

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

data class Stock(
    val amount: Int,
    val unlimited: Boolean,
    val deliveryDate: LocalDate?,
)

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

Lesson

Learned

Micro Service

Schnittstellen Test

API

Service A

Service B

import

<dependency>
    <groupId>de.itscope.catalog</groupId>
    <artifactId>price-stock-api</artifactId>
    <version>1.8.3</version>
</dependency>

pom.xml

Weniger zu testen

Lesson

Learned

Micro Service

Schnittstellen Test

Service A

API

Service A

Service B

import

<dependency>
    <groupId>de.itscope.catalog</groupId>
    <artifactId>price-stock-api</artifactId>
    <version>1.8.3</version>
</dependency>

pom.xml

Kotlin

Typescript

Vue

Backend zu Frontend
win win

Lesson

Learned

Micro Service

Schnittstellen Test

publishing {
    repositories {
        maven {
            url = Publishing.nexusTargetRepo
            isAllowInsecureProtocol = true
            credentials {
                username = Repository.Nexus.Credentials.username
                password = Repository.Nexus.Credentials.password
            }
        }
    }

    publications.create("mavenJava", MavenPublication::class) {
        from(components["java"])
        artifact(sourcesJar.get())
        pom {
            name.set(artifactId)
            developers {
                developer {
                    id.set("its")
                    name.set("ITscope - Developer")
                }
            }
        }
    }
}

build.gradle.kts

Unit

Service

UI

You are here

UI Tests

Unit

Service

UI

You are here

UI Tests

Unit

Service

UI

UI Tests

Unit

Service

UI

UI Tests

RedGiant

2.562

Unit

335

Integration

801

Selenium

Unit

Service

UI

UI Tests

0:27

Unit

0:03

Integration

1:05

Selenium

Build Pipeline

#107

#108

#109

#110

#111

#112

Wackler

Selenium Tests

How
To

REST
API

doTestSetup()

Selenium Tests

-SideApplicationFrameLayout-GlobalSearchContentView-ContextPanelDetailView
-masterDetailLayout-ProductSmallDetailView-ProductSmallHeaderView
-AddToCartWidget-AddToCartMenuBar

getDriver().findElement(By.className("button"));
getDriver().findElement(By.name("In den Warenkorb"));
getDriver().findElement(By.id("..AddToCartMenuBar.."));

TIP

Selenium Tests

TIP

Selenium Tests

// Open Product Page
IExtendedProduct product = TestProductService.getDefault();

FrontendProductPage productPage = application.openPage().productPage(product);

// Check Product Page Header
assertThat(productPage.getHead().getTitle(), hasText(product.getName()));
assertThat(productPage.getHead().getEan(), hasText(product.getEan()));

Application

ProductPage

Head

Title

PageObjectPattern

Ham-Crest-Matcher

Typisierte Test-Daten

TIP

Selenium Tests

resellerApplication.action().createNewCartInPortal("New Cart");
CartRow newCart = portal.getCartTable().getCartByName("New Cart");
newCart.action().addProduct(appleiPadAir);
newCart.action().shareFirstLineItem();
newCart.action().addNegotiationPartner(distributorUser.getUserName());
SearchPage<?> searchPage = openPage().globalSearchNegotiationPage();
SellerResponsePage<?> sellerResponsePage = 
    searchPage.openFirstSellerResponseFromCompany(getCompany());
sellerResponsePage.getFirstRow().setPrice("1.00");
CartColumnTableRow supplierCard = 
    newCart.getCartColumn().getCartTable().getFirstSupplierCard();
supplierCard = selectNegotiationPosition(1, newCart, supplierCard);

PriceRequestTest

TIP

Selenium Tests

Testschritte benennen

// #1 (Buyer) create new cart and open it
resellerApplication.action().createNewCartInPortal("New Cart");

// #2 (Buyer) add Product and start negotiation with distri Also
CartRow newCart = portal.getCartTable().getCartByName("New Cart");
newCart.action().addProduct(appleiPadAir);
newCart.action().shareFirstLineItem();
newCart.action().addNegotiationPartner(distributorUser.getUserName());

// #3 (distri) set a price
SearchPage<?> searchPage = openPage().globalSearchNegotiationPage();
SellerResponsePage<?> sellerResponsePage = 
    searchPage.openFirstSellerResponseFromCompany(getCompany());
sellerResponsePage.getFirstRow().setPrice("1.00");

// #4 Select negotiation price
CartColumnTableRow supplierCard = 
    newCart.getCartColumn().getCartTable().getFirstSupplierCard();
supplierCard = selectNegotiationPosition(1, newCart, supplierCard);

PriceRequestTest

TIP

Selenium Tests

REST
API

doTestSetup()

TIP

Selenium Tests

REST
API

doTestSetup()

getErrorLog()

TIP

Selenium Tests

REST
API

doTestSetup()

getErrorLog()

TIP

Selenium Tests

1:05

Selenium

NEED FOR SPEED

Für neue Tests Unit Tests präferiert
Selenium Tests nur noch bei Kommentar
Stabilisierung der Tests durch bessere Isolation durch Refactoring
Feintuning der Parallelität
Stabilität VS Performance
Austausch der DB von
Live-Snapshot zu Mini-DB
Heraustrennung von Bereichen in Micro-Services und Vue-Anwendung

0:24

Selenium

TIP

Selenium Tests

1:05

Selenium

NEED FOR SPEED

0:24

Selenium

TIP

Selenium Tests

Tips

Page-Object-Pattern als UI Abstraktion
Test-Daten typisieren
Hamcrest-Matcher für prägnaten Code
und lesbare Fehlerausgaben
Gute Inline-Dokumentation der Einzelschritte
Automatisches Wait + Retry bei Driver Zugriff
REST API für Backend Zugriff
UI Tests ausdünnen, migrieren oder heraustrennen

Lesson

Learned

Lesson Learned

Wähle die richtige Etage

Denke zuerst an die Tests

Regelmäßiges
Learn & Adapt

Passes the tests

Reveals intention

Nehmt euch Zeit
fürs Testen

Was nicht getestet ist,
geht kaputt

danke.tobse.eu

https://slides.com/
tobsefritz/test-pyramid

Folien

Feedback

Liebling, ich habe die Testpyramide auf den Kopf gestellt

JUG Karlsruhe

Tobse Fritz

JUG Karlsruhe

JUG Karlsruhe

Lesson Learned

HOW TO

Technik Check

DOS

DONT'S

&

Warum Testen?

Die Test-Pyramide

RedGiant

Technik Check

SOFTWARE QUALITÄT

Was nicht getestet ist,

geht kaputt

Was nicht getestet ist,

geht kaputt

Was nicht getestet ist,

geht kaputt

User

Tested

User

Tested

User

Tested

User

Tested

User

Tested

Was nicht getestet ist,

wird missachtet

Was nicht getestet ist,

wird missachtet

Was nicht getestet ist,

wird missachtet

Tests für schnelles Feedback

Tests zum Lernen / Ausprobieren

Tests!

Tests!

Tests!

1 Bug

1 Neuer Test

1 Feature

1 Neuer Test

Lesson

Learned

Unit

Service

UI

Die Testpyramide

Unit

Service

UI

Die Testpyramide

Unit

Service

UI

Die Testpyramide

Unit

Service

UI

Die Testpyramide

Unit

Service

UI

Manuelle Tests

Die Testpyramide

Das Testprotokoll

Lesson Learned

Unit

Service

UI

Manuelle Tests

AUTOMATED

Die Ampel im Büro

Lesson

Learned

Lesson
Learned

HOW
TO

Technik
Check

Technik
Check

Tests für schnelles
Feedback

Tests zum Lernen
/ Ausprobieren

Lesson
Learned

Lesson
Learned

Lesson
Learned

Lesson
Learned

Lesson
Learned

Lesson
Learned