Now and then, a property test can be easier than an example test. Today, Tanya and I benefited.
There’s this web service. It returns a whole tree of information, some of it useful and some of it is not.
{ “category”: “food”,
“children: [ { “category” : “fruit”,
“children” : […LOTS MORE…],
“updatedAt” : “2014-06-30T16:22:36.440Z”,
“createdAt” : “2014-06-30T16:22:36.440Z”},
{“category” : “vegetables”,
“children” : […EVEN MORE…],
“updatedAt” : “2014-06-25T18:32:36.436Z”,
“createdAt” : “2014-06-25T18:32:36.436Z”}],
“updatedAt” : “2014-06-15T16:32:36.550Z”,
“createdAt” : “2014-03-05T08:12:46.440Z” }
The service is taking a while, mostly because it’s returning half a meg of data. Removing the useless fields will cut that in half.
Being good developers, we want to start with a failing test. An example test might start with inserting data in the database, perhaps after clearing the table out, so we can know what the service should return. That’s a lot of work, when I don’t really care what’s returned, as long as it doesn’t include those updatedAt and createdAt fields.
Currently when we test the function implementing this service, there’s some sample data lying around. If we write a property test instead of an example test, that data is good enough. As long as the service returns some data, and it’s something with children so we can check nested values, it’s good enough. I can test that: (actual code)
(deftest streamline-returned-tree
(testing “boring fields are not returned”
(let [result (method-under-test)]
(is (seq (:children result))))))
This is not a generative test, because it doesn’t generate its own data, and it doesn’t run repeatedly. Yet it is a property test, because it’s asserting a property of the result. The test doesn’t say “expected equals actual.” Instead, it says “The result has children.”
This test passes, and now we can add the property of interest: that the map returned by method-under-test has no :createdAt or :updatedAt keys, at any level. We could find or write a recursive function to dig around in the maps and nested vectors of maps, but that same function would also be useful in the implementation. Duplicating that code in the test is no good.
One of the classic challenges of property testing is finding two different ways to do the same thing. Then we can make assertions without knowing the input. But… nobody said they have to be two smart ways of doing the same thing! I want to be sure there’s no “createdAt blah blah” in the output, so how about we write that nested map to a string and look for “createdAt” in that?
(deftest streamline-returned-tree
(testing “boring fields are not returned”
(let [result (method-under-test)
result-string (pr-str result)]
(is (seq (:children result)))
(is (not (.contains result-string “:createdAt“))))))
This gives us a failing test, and it was a heck of a lot easier to implement than an example test which hard-codes expected results. This test is specific about its purpose. As a bonus, it doesn’t use any strategy we’d ever consider using in the implementation. The print-it-to-a-string idea, which sounded stupid at first, expresses the intention of “we don’t want this stuff included.”
Property tests don’t have to be generative, and they don’t have to be clever. Sometimes it’s the dumb tests that work the best.
Bonus material:
the output of this failing test is
expected: (not (.contains result-string “:createdAt“))
actual: (not (not true))
This “not not true” actual result… yeah, not super useful. clojure-test’s “is” macro responds better to a comparison function than to nested calls. If I define a function not-contains, then I can get:
expected: (not-contains result-string “:createdAt“)
actual: (not (not-contains
“{:children [{:createdAt \”yesterday\”, :category \”fruit\”}], :createdAt \”last week\”, :category \”food\”}”
“:createdAt“))
That’s a little more useful, since it shows what it’s comparing.