Skip to content

Lab 11

In this lab sessions you will be practicing two advanced software testing concerns: Mutation testing and Mocking. For the exercises in this last lab session, you will be working with the PrimeNumbers project and the MockServerExercise project, both available on Gitlab.

Mutation testing

  • Mutation testing is an automated stress test for existing unit and integration tests.
  • The key thought is to inject errors in a supposed-correct program and assert if at least one tests fails, per injected error.
  • Each variant of the original software injected with an error is called a mutant.
  • If no test is able to detect the artificial program modification, the associated tests are assumed to contain zombies, i.e. pointless of insufficient tests.

Pitest

  • The pitest plugin is a convenient extension to the maven build process.
  • Installation is as simple as adding the following plugin to a pom.xml configuration:
<!-- Mutation test report-->
<!-- HTML report available at:  target/pit-reports/index.html  -->
<plugin>
    <groupId>org.pitest</groupId>
    <artifactId>pitest-maven</artifactId>
    <version>1.17.1</version>
    <executions>
        <execution>
            <id>pit-report</id>
            <!-- optional, this example attached the goal into mvn test phase -->
            <phase>test</phase>
            <goals>
                <goal>mutationCoverage</goal>
            </goals>
        </execution>
    </executions>
</plugin>
  • Afterwards, creation of a mutation test report is triggered by the test phase, i.e. mvn clean test
  • The report is available in target/pit-reports/index.html and can be inspected with any web browser.

Your turn

Sample code setup:

  • Clone the prime number checker project:
git clone https://gitlab.info.uqam.ca/inf2050/PrimeChecker.git
  • Open the project in IntelliJ
  • Run the provided tests, using mvn clean test

All tests will pass, but does that mean your tests are good ? Let's find out:

  • Activate the mutation test plugin, by integrating PIT into your pom.xml.
  • Run the tests again and create a mutation test report: mvn clean test
  • Inspect the mutation test report in your browser.

The report will tell you about a negation mutation that can be added to your code, without the tests noticing.

  • Identify which mutation this is, i.e. which line, which changement.
  • Verify the mutation, by actually mutating the code the exact same way (your code will now count non-primes instead of primes)
  • Run the tests again.
What should happen ?

The test results should still be all passing. The mutation test results said your tests are not able to identify the modification to your source code.

Finally, it is time to fix the problem and improve your tests!

  • Revisit your tests. Identify the issue with your existing tests.
  • Either modify the existing tests or add more tests.
  • Run the mutation tests again and verify the mutation coverage is now 100%.

Mocking

  • In the last lecture you've seen an example of a SUT dependency (database), which needed to be replaced by a Mock to facilitate testing.
  • In the following you are going to apply the same concepts on a different application, the TagCounter.
  • The use case is identical, we have a SUT (class to test), which does just one thing: Count HTML tags.

Example

This HTML sample

<html>
<h1> Hello, World! </h1>
</html>

has 4 tags:

  • <html>
  • <h1>
  • </h1>
  • </html>

Dependency illustration

Unfortunately, we cannot test our TagCounter in isolation. To instantiate, we also need a ServerFileDownloader object, which provides the html code, by downloading if from a real server: Wikipedia

---
title: SUT needs a ServerFileDownloader object
---
classDiagram
    TagCounter *--> ServerFileDownloader: has a

    class ServerFileDownloader {
        <<Class>>
        +getWebpageContent() void
    }

    class TagCounter {
        <<Class>>
        +TagCounter(ServerFileDownloader) TagCounter
        +countTags() int
    }

Your turn

  • Download the prepared ServerMockExercise project.
  • Run the application, familiarize yourself with the existing code.
  • Modify only the test code, to enable mock testing
    • Add Mockito as test dependency to the pom.xml
    • Open the prepared test class: TagCounterTest and use the annotations seen in class to replace the server dependency by a mock dependency.
    • Add a when-thenReturns statement to prepare a test class. Return a minimal webpage, e.g. the one listed above.
    • Add an assert statement to your test, to verify the TagCounter functions correctly.
    • Use a Captor to verify the Mock serverFileDownloader object is correctly invoked.