Testing is hugely important in order to have confidence in a product working as expected and code changes not breaking existing functionality. It’s easy to get stuck in implementation-mode and in fact as we prototyped out the web-service side of Mineservers and figured out our tech stack, we put very little effort into automated testing. In that same time. the service has matured from very basic CRUD to having real business logic which has left a good chunk of our code unexercised beyond basic manual API testing.

Background: Server Voting

One area we desperately needed tests in was server voting.

For a bit of background, Mineservers is our modern, API-driven take on the classic server list website for Minecraft. It is intended to help players find Minecraft servers to join that match their play style and server owners get the word out about what their Minecraft server is all about. One of the ways servers are ordered in the list is by having players vote for the servers they enjoy, and often servers will provide some in-game reward to incentivize voting through Votifier.

In order to keep the playing field level, votes are restricted so that players can’t just vote infinite times, leaving whoever can spam the most as the top server. Many server lists allow one vote per 24 hours, something we feel encourages players to only vote for the largest server that rewards voting the most. Another problem with this approach is that since you can only vote after a full 24 hours has passed since your last vote, your voting time slowly slides back. To fix both issues, we want players to be able to vote for up to 3 servers per day (and only once per server per day), with a universal vote reset time (EG 5AM EST). The exact details of the voting system on Mineservers are not set in stone and we may adjust as we test and get feedback. There are a number of data points we can use to identify a user and find out if they should be allowed to vote. Some of those might be:

  • User IP address
  • Cookie ID
  • Minecraft Username (if provided for voting reward)
  • User/account ID (if authenticated)

So, the logic for “can a user vote?” is basically:

  • Has the user (by the identifiers above) voted fewer than 3 times today?
  • Has the user NOT voted for this server today?

Business logic

Seems relatively simple, but when you translate it into code there is business logic that can go wrong. For example, how do you define today? First, you need to be able to specify a voting reset time. We do this via configuration:

data class VotingConfig(
    val serviceName: String,
    val cutoffTime: OffsetTime,
    val dailyVoteAllowance: Long
)

…where cutoffTime is loaded as OffsetTime.parse(config.propertyOrNull("cutoffTime")?.getString() ?: "05:00-05:00") (defaults to 5AM EST at the moment). Now we need to be able to take the configured cutoff time and represent it as a DateTime of the last reset, so we can count votes that have happened today in our database queries, EG SELECT * FROM votes WHERE createdAt > $lastCutoff.

Initially, that looked like this:

    private val lastCutoff
        get() = LocalDate.now(ZoneOffset.UTC).atTime(votingConfig.cutoffTime).let {

            // if today's vote cutoff time is in the future, the last one was yesterday
            if (it.isAfter(OffsetDateTime.now(ZoneOffset.UTC))) {
                it.minusDays(1)
            } else {
                it
            }
        }.toLocalDateTime()

That’s logic that needs to be tested in a few scenarios so we are confident that we’ll always get the right value. Other testable logic might include:

  • The canVote function which uses the identifiers and lastCutoff values above to return a true/false if the user can vote for the server requested
    • Do the different identifiers find votes as expected?
    • Can you vote for 3 (dailyVoteAllowance) and only 3 (dailyVoteAllowance) servers per day?
    • Can you only vote for a specific server once per day?
    • Is another user able to vote in the same way, unimpacted by the votes of the first user?
  • Votifier configuration/support
    • Do we call the Votifier library to send a vote only when a server is properly configured?
    • Are we sending the right Votifier vote protocol version depending on configuration (V1 vs V2)?
    • What happens if we are unable to connect to send the vote?
  • The vote insertion into the database
    • Are votes only happening when users canVote?
    • What happens when I try to vote for a server ID that doesn’t exist?
    • Does the resulting entity have the appropriate identifiers for later querying?

Testability

The Voting logic for Mineservers is, for the most part, nicely isolated in VotesService.kt. That makes it a lot easier to know exactly what is under test and where problems might happen. That said, as we went to add tests, we realized we needed to make a number of small tweaks to improve testability. Some of those we’ve already done and some are planned for the future.

One example of improved testability is allowing injection of a Clock to let tests control time. In normal operation, the Clock might be Clock.systemUTC() while during tests we could specify Clock.fixed(Instant.parse("2020-08-01T00:00:00Z"), ZoneOffset.UTC). It’s a small change that makes testing the lastCutoff time much easier. The private val lastCutoff from above now becomes:

    private val clock: Clock by inject() // We use Koin for dependency injection

    internal fun getLastCutoff(): LocalDateTime =
        LocalDate.now(clock).atTime(votingConfig.cutoffTime).let {
            // if today's vote cutoff time is in the future, the last one was yesterday
            if (it.isAfter(OffsetDateTime.now(clock))) {
                it.minusDays(1)
            } else {
                it
            }
        }.toLocalDateTime()

Since we’re injecting the dependencies of VotesService, we can construct those instances in our tests. For example:

    fun voteTestModule(votingConfig: VotingConfig, clock: Clock) = module {
        single { LoggerFactory.getLogger("VotesServiceTests") }
        single { votingConfig }
        single { clock }
    }

…then create the service instance with VotesService(). In the future, we should inject the properties when constructing the service instance, rather than injecting inside the service itself. What that means is instead of:

module {
    single { environment.config }
    single { VotingConfig.load(get<ApplicationConfig>().config(VotingConfig.path)) }
    single { Clock.systemUTC() }
    single { VotesService() }
}

// VotesService.kt
class VotesService : KoinComponent {
    private val votingConfig: VotingConfig by inject()
    private val clock: Clock by inject()
    
    // ...
}

We’d have:

module {
    single { environment.config }
    single { VotingConfig.load(get<ApplicationConfig>().config(VotingConfig.path)) }
    single { Clock.systemUTC() }
    single {
        VotesService(
            votingConfig = get(),
            clock = get()
        )
    }
}

// VotesService.kt
class VotesService(
    private val votingConfig: VotingConfig,
    private val clock: Clock
) {
    // ...
}

Which would mean our tests can pass the config and clock directly, without having to construct a Koin module.

Another improvement to make is separating out the interactions with our ORM into a separate data layer. This would allow us to mock the database interaction to really isolate our business logic across the board. Right now, we use MariaDB4j to run an embedded database and are performing real database interaction in the VotesService “unit” tests.

Testing and Its Value

I find that when testing some business logic with clear rules and goals, Gherkin syntax can make tests immediately understandable to anyone reading the results. Perhaps more importantly, when a test fails we know the failed expectation in plain English.

For example, we could write a test case for the business logic described earlier as:

Scenario: Two votes for the same server, same IP
    Given: A server
    When: First Vote
    Then: Vote should have counted
    When: Second Vote
    Then: Duplicate vote fails and exception thrown

This outlines exactly what is being tested and what is expected to happen, without needing to understand how the underlying test is written. To accomplish this, we opted to use the Spek Framework.

Here are the first wave of test scenarios we wrote:

VotesService Tests

These are in no way comprehensive, but it gives a decent starting point for verifying the voting logic in Mineservers and ensuring we don’t break it unintentionally. These tests run on every commit with status/reporting in Pull Requests, so we now have a fighting chance to find and understand changes to the tested logic before code makes it to production.

So what did that get us? What value did it add? Well, aside from the confidence boost and regression prevention, these tests caused us to find and fix a number of issues that would certainly have come up later, most likely at a more inconvenient time. Here are a few of the things that added very tangible value:

  • Fixed that you could vote for non-existent servers
  • Fixed canVote condition where votes without a username or an authenticated user (null values) would trigger equality when not desired (only add query conditions if they are non-null)
  • Fixed server entity’s website field being NOT NULL in the DB schema but nullable in the ORM
  • Better code quality and testability (future tests will be easier to write)

We’re continuing to build out our test suite and I hope I’ve managed to illustrate why testing at every stage is so valuable, as if anyone disagrees.