When programming in Golang, the Mockery project painlessly generates Testify mocks from any interface. But very rarely (and specifically in the AWS Go SDK), you run into situations that Testify doesn’t handle well. If you think that Testify has failed you, then prepare to enter… a minefield of horrors!
The Post-Testify Minefield of Horrors
This “Post-Testify Minefield of Horrors” I speak of is a simple trilemma. You must decide between one of the following tradeoffs:
- Don’t unit test this code – what could go wrong?
- Abandon Testify for this package – hand write ALL the mocks!
- Keep using Testify – rewrite the mocks EVERY time your regenerate your mocks.
None of these are great options. What if there was a way to use Testify mocks 99.99999% of the time, but handcraft only the special cases it doesn’t cover? That’s what I aim to offer you by the end of this post – a way through this minefield.
You Lie! Testify Is All I Need
For AWS APIs which aren’t paginated, Testify works swimmingly, and Mockery does a fantastic job of writing all the tedious mocks for us.
However, as a prime example where Testify seems to break down, consider the DynamoDB package with the AWS Go SDK. Let’s say you want to query one of your DynamoDB tables, but the total amount of data you want to retrieve could become greater than 1 MB. According to the documentation:
If you query or scan for specific attributes that match values that amount to more than 1 MB of data, you’ll need to perform another Query or Scan request for the next 1 MB of data.
To ensure that your application retrieves all the query results, you need to call
DynamoDB.Query() repeatedly with different inputs and outputs. That’s all very simple, but to unit test this code you’ll need a mock implementation of
DynamoDB.Query() which can return different values on each call. Although Mockery shows some examples of this in their README, returning dynamic values seems not to work at time of writing (late October 2015).
So, this ideal solution I’ve promised you should both:
- Use Testify mocks generated painlessly by Mockery for most of the AWS Go SDK
- Allow us to handcraft select pieces of highly dynamic AWS Go SDK code
So how can we achieve both?
The Solution: Separate Interfaces
We’ll write one interface that acts as a contract between our code and AWS Go SDK in general, and we’ll let Mockery generate mocks for this interface. Then, we write a very small dedicated interface for the handcrafted mocks. Any of the non-paginated AWS APIs we use in our code will go through the general interface, but our application will
Query() only through the handcrafted interface. Since Mockery puts its generated code in a subdirectory
./mocks, we can ensure our handcrafted mocks will survive by keeping them out of
The Big, General Interface Interface
DynamoDBer stutters, let’s just call it
DBer. The general interface
DBer should look something like this:
Then to call any method in AWS Go SDK, we always make sure that method has been added to
DBer and use the
DBer interface in our code, something like this:
Don’t try to compile that, but you get the gist. Now, to get all our mock objects for free, we’ll install Mockery. Then, we’ll cd into the code directory and run Mockery:
At the end of the process, I’ll have a shiny new directoy
./mocks containing all the Testify mocks for me. Sweet! Okay, that’s the big
DBer interface. Let’s move on to the other interface.
The Small, Handcrafted Queryer Interface
Queryer only needs to implement the
Query() method already in the DynamoDB package. For good measure, we create a nil and ignored DynamoDB struct object as a
Queryer so that the compiler will verify that we have the method signature of
DynamoDB.Query(). We want to make sure the compiler catches any obvious mistakes (like failing to satisfy an interface) before we ship this code into production. Otherwise, we might theoretically some day update the AWS Go SDK package, have the code compile, and ship this out into production only to discover failures too late.
Wrapping Up Queryer in a Useful Func
Let’s assume your application should always retrieve all the results, we’ll want to create a wrapper method to DRY up and encapsulate all the logic necessary to retrieve all pages of results.
Let’s unpack this a bit. Why both a
paginatedQuery()? The idea here is that we want to easily mock out the method that our application will actually use, so I’ve created a mutable variable
paginatedQueryImpl(). This allows me to change the implementation on the fly in unit tests by just assigning a new lambda function.
However, for my unit tests on
paginatedQuery() itself, I need to ensure that I’m testing the actual implementation. For that kind of assurance, I need to declare a function with the real implementation. Then, in my unit tests I can simply reassign the real implementation to my
paginatedQuery variable as a setup step.
Then, I need a way to test the interaction between
paginatedQueryImpl and DynamoDB, and that’s precisely why it takes a
Queryer argument. This allows us to pass in either real
DynamoDB objects or our handcrafted mocks. The handcrafted mock will simulate all the various test cases that we care about without needing to send any requests over the wire or even have an AWS account.
The implementation of
paginatedQuery() is left as an exercise to the reader, since testing is the primary focus of this post.
Handcrafting the Mock Queryer
There’s a few salient points to discuss here:
- We need the
QueryFuncfield to be a lambda function, because we want to replace it with different implementations in each test case. We don’t want to define a different struct type for every test – that’s just too much boilerplate code.
- Because the
Query()both have the same signature, we can simply passthrough the arguments and return values from the
QueryFunc()and get compile-time checks that we defined our lambdas correctly in the tests.
Let’s Write Some Example Unit Tests
The F# community’s writings about property-based testing have had a profound impact on me. Whenever appropriate, I like to include some randomness in my tests to ensure that a whole range of similar but unpredictable values are tested to prevent an incomplete implementation from passing.
In our test method
TestPaginateQuery_NQueryResultPages, the first five lines randomly pick how many pages of results and total items should exist when this test runs.
The next several lines of code create call counter
c and create a new
mq. This object is passed into our the method we want to test
PaginateQuery(). Now, we have complete control inside the QueryFunc method to calculate whatever return values we want AND to run assertions on the arguments to this method.
My test leaves a lot to be desired (and probably doesn’t even compile), but hopefully it can serve as an inspiration.
Testing Code that Uses PaginateQuery()
I believe the clearest and easiest way to test code that invokes
paginateQuery() is using the mutable/immutable functions I’ve defined earlier. This allows us to stub replace the func during our tests and gives us unfettered access to control in the innerworkings of our code. Let’s see a short example:
In a similar fashion, we can replace
paginatedQuery() with mocks that will return nil errors, improperly formatted data, a slice of results, an empty slice, or any other scenario that could conceivable happen.
Damn Your Smoke and Mirrors Lambdas!
The downside of this approach is that any part of your codebase could overwrite the func pointed to by
paginatedQuery() which seems a bit unsafe. If any unit tests replace it with a mock, you’ll need to reset it to
paginatedQueryImpl() as discussed earlier.
However if global mutable lambdas makes you uncomfortable, you can use Mockery to generate a mock for
DynamoDB.Query() and indirectly control the output of
paginatedQuery(). If you go that route, just don’t use a package variable. Instead, make
paginatedQuery a proper immutable func in its own right. As far as I can tell, which strategy you use is entirely a matter of preference.
Taking it Further
- The Mockery Project is well worth a look if you haven’t seen it before.
- Nearly everything I’ve said here applies equally to any other paginated function in the AWS Go SDK (and that’s a lot of functions). If you use this sort of technique a lot, it might make sense to write a code generator that can DRY up repeated logic and tests around this functions.
- If you’re curious about DynamoDB, see the AWS documentation for Query
- Consider using AWS’s DynamoDB QueryPages() method instead of
Query()in your own code since it handles lots of pagination messiness for you.
- I omitted the definition of the
MySuitestruct in one of the examples above for brevity. See the Testify suites godoc if you need a refresher.
- If you need to rerun just one test several times while you’re debugging or writing tests, use
go test ./path/to/package -run TestFuncwhere “TestFunc” is the name of the test that go knows how to run (ex: the func that starts your Testify suite).
- For integration tests, I recommend trying AWS’s local-dynamo jar. It seems to be maintained and reasonably performant as a drop-in replacement for dynamodb that runs on your own machine or CI build environment.
Golang is a fantastic language for writing unit tests, but it sometimes requires some creativity to test all the code paths you care about. Consider using smaller interfaces to mock out isolated complex dependencies that Testify can’t handle well.
Special Thanks to Julian Cooper for all the ideas about lambdas and small interfaces presented here, to Phil Cluff for allowing me to write about my work, and my wife Emilie Fisher-Fleig for the insightful comments on my drafts.