Maybe it's too obvious to be mentioned, but you also want to test interactions when the targets of those interactions are not suitable for access during a test. That is, if they would be too slow, return inconsistent results, or perhaps are simply inaccessible from your test environment (this last comes up less with unit tests). Oh, and of course when you need to simulate failure cases during the interactions.
Regardless, I think your point is well taken that you should be verifying state regardless. Even when you are mocking out all interactions, you need to ensure that the code under test properly handles the results of those interactions.
State testing, as you stated, is good for validating the results, often of some function or calculation. State testing is also good for validating edge cases and exceptions (divide by 0).
Interaction testing is good to validate business logic and usage requirements. For example, if a user is added to the system, was a welcoming email sent. Sure, I could mock an SMTP server and test the state of the queue... I could just as easily shim/stub the method for sending email, and validate that it was called.
see more: http://www.sbrickey.com/Tech/Blog/Post/Why_We_Test
Thanks for the article. The one thing I personally cannot agree with is the statement that in most cases we need to test state, not interactions.
Where I work, we do a lot of message processing, where there's very little state (the state is transient an held in a message itself that passes through system and is processed in the meantime). As a result, at most of our objects don't hold state directly. Their only "state" are references to their collaborators with whom they exchange messages with. Their responsibilities is to receive messages (i.e. methods are called on them) and send messages (i.e. invoke methods on other objects). This makes 90% of our tests to be "interaction tests" while having ca. 70% of our methods not even returning any values to assert on.
Anyway, about your example - I'd probably write a test for a rule saying when bubble sort is chosen and when quick sort, but I'd still write a test for each sorting algorithm the same way you wrote the first one (state based). So, the two examples shown are not really alternatives in my mind - the first one specifies what the output of sorting should be, and the second one I'd write earlier to specify when which method should be chosen. Note that if you tested a logic where two sorting algorithms can be chosen and wrote only tests such as the first one, you could actually remove the choice between bubble sort and quick sort and leave just one algorithm - the tests would pass anyway, because each algorithm gives the same end result and this result is the only thing you're asserting.
"Interaction testing" is also the only way when you want to test a framework whose components (real workers) are pluggable. Because you want to test the framework but not the components, so the state is meaningless, only the execution process of the framework is meaningful.
Love this post. This topic of discussion seems to fall in with the Classical TDD vs Behavioral TDD debate, and I've thought that that argument usually just boiled down to what your post is about, testing state vs testing interactions.
When testing layers that don't make direct calls to the database, but perform them through data access objects, I usually mock them out to gain all the benefits of speed, reliability, and nimbleness (in terms of what branch statements I can hit) of unit tests. However, sometimes I wonder if it's better to just let those hit the database.
" Testing state means you're verifying that the code under test returns the right results. "
This is misleading as objects communicates with other objects by sending objects (states). A unit test should also check those sent objects in order for the unit test to be correct and complete.
Always good to see more TotT!
ReplyDeleteMaybe it's too obvious to be mentioned, but you also want to test interactions when the targets of those interactions are not suitable for access during a test. That is, if they would be too slow, return inconsistent results, or perhaps are simply inaccessible from your test environment (this last comes up less with unit tests). Oh, and of course when you need to simulate failure cases during the interactions.
Regardless, I think your point is well taken that you should be verifying state regardless. Even when you are mocking out all interactions, you need to ensure that the code under test properly handles the results of those interactions.
State testing, as you stated, is good for validating the results, often of some function or calculation. State testing is also good for validating edge cases and exceptions (divide by 0).
ReplyDeleteInteraction testing is good to validate business logic and usage requirements. For example, if a user is added to the system, was a welcoming email sent. Sure, I could mock an SMTP server and test the state of the queue... I could just as easily shim/stub the method for sending email, and validate that it was called.
see more: http://www.sbrickey.com/Tech/Blog/Post/Why_We_Test
Hi, Andrew
ReplyDeleteThanks for the article. The one thing I personally cannot agree with is the statement that in most cases we need to test state, not interactions.
Where I work, we do a lot of message processing, where there's very little state (the state is transient an held in a message itself that passes through system and is processed in the meantime). As a result, at most of our objects don't hold state directly. Their only "state" are references to their collaborators with whom they exchange messages with. Their responsibilities is to receive messages (i.e. methods are called on them) and send messages (i.e. invoke methods on other objects). This makes 90% of our tests to be "interaction tests" while having ca. 70% of our methods not even returning any values to assert on.
Anyway, about your example - I'd probably write a test for a rule saying when bubble sort is chosen and when quick sort, but I'd still write a test for each sorting algorithm the same way you wrote the first one (state based). So, the two examples shown are not really alternatives in my mind - the first one specifies what the output of sorting should be, and the second one I'd write earlier to specify when which method should be chosen. Note that if you tested a logic where two sorting algorithms can be chosen and wrote only tests such as the first one, you could actually remove the choice between bubble sort and quick sort and leave just one algorithm - the tests would pass anyway, because each algorithm gives the same end result and this result is the only thing you're asserting.
Great post! And the example is also subtly great.
ReplyDelete"Interaction testing" is also the only way when you want to test a framework whose components (real workers) are pluggable. Because you want to test the framework but not the components, so the state is meaningless, only the execution process of the framework is meaningful.
Love this post. This topic of discussion seems to fall in with the Classical TDD vs Behavioral TDD debate, and I've thought that that argument usually just boiled down to what your post is about, testing state vs testing interactions.
ReplyDeleteWhen testing layers that don't make direct calls to the database, but perform them through data access objects, I usually mock them out to gain all the benefits of speed, reliability, and nimbleness (in terms of what branch statements I can hit) of unit tests. However, sometimes I wonder if it's better to just let those hit the database.
" Testing state means you're verifying that the code under test returns the right results. "
ReplyDeleteThis is misleading as objects communicates with other objects by sending objects (states). A unit test should also check those sent objects in order for the unit test to be correct and complete.