Different Perspectives: Coding with Eric and Noah
Since software development best practices are largely based on communal agreement instead of raw data, we often find ourselves arguing for and against certain practices. This is especially true when it comes to software quality. In fact, as of a couple years ago, only a single university taught software QA as a major in the United States. In this article, two test engineers will debate code reuse in test code.
Matches Development Practices
Eric: If we consider that everyone on the development teams are writing tests then we should try to follow the same best practices we use when writing production code. It is far easier to get buy-in from the team on adding test cases if the developers are going to be doing more of the test implementation and they don’t have to break away from the paradigm they are used to. Code reuse is definitely part of best practices and I expect good developers to be more on board writing tests if they don’t have to knowingly break best practices.
Noah: This is a valid point and one that should definitely be considered if you work in a company where developers are heavily involved in writing tests or in a company that uses automated QA as a feeder for junior development. I do worry that developers working in tests could lead to over extending a function for the sake of saving a few keystrokes in a code base you don’t care about as much. That, however, is based on training or personnel and doesn’t really fit here.
The Cost of Modification Has Been Significantly Reduced
Noah: Imagine a UI test for example, that requires logging in as a user. Your first test logs in a standard user and validates something on the page to ensure we are logged in; simple enough. We can pull the login functionality out and pass in the username and password and now all tests can use it. Next test logs in as an admin who is taken to an admin panel. Now we can’t use the login method to validate logging in as it stands.
If we add an if/else we are covered…until we need to write a test for a sales person, who upon logging in is taken to a sales reporting page. Oh, and what about those customization options that change what a user sees when they log in? As you can see, we can quickly have a wall of if/then statements. Now we’re barely into the site and were already managing complex state and all for the benefit of some future were the locator for password changes?
With modern IDE’s we can use ctrl-F or cmd-F and find all the locations were we’re using that locator. Since all the test logic is in the test, we can make an educated choice about changing that locator; or for the YOLO crowd, we can do a replace all.
Eric: The login issue is a pretty typical problem you run into when you start automating tests where you have to deal with web UI so the scenario is an appropriate example on how the scope of a test step can get overburdened. We can take the approach of adding conditional logic or separate the steps to be more specific to the class of user logging in but we shouldn’t always do it the same way all of the time.
The issue I have with not writing conditional logic to support code reuse is that, at the beginning of the example you provided, we had a method that we trusted to log in the user the same way all of the time. Once we break that away to use and slightly modify elsewhere, we are improving that test code but the original code we may have written months ago is neglected forever. We never pulled forward the tips and tricks we learned along the way in our journey as SDETs to improve test code. What I have experienced has shown me that, especially in the realm of web UI automation, my first implementation is rarely the best and we improve as we add and fix other tests. Without reusing code and taking the time to make sure that conditional logic is elegant and reliable, we could end up with tens, maybe hundreds of iterations of code which does the same thing but one or more iterations could have been written by me a year ago when I wasn’t as competent with the technology and is more prone to flakiness or using something that will be deprecated sooner.
Reusable Code Is More Inclusive
Eric: I would argue with more fervency to reuse code at a unit test level since it sits so close to the implementation of the code that is the SUT because the test code is going to be very similar and less conceptual. That being said, there are good reasons to adopt code reuse farther right on the V model as well (such as in web UI automation). By reusing code, I think we are more inviting for those who may be less experienced to jump in and just drop in the method for doing a given step and work out the more difficult part which is the new assertions we are going to need for the new cases we are adding. I’d rather not spend time trying to get something working from the ground up with someone who is unfamiliar but rather get them past the mundane hurdles and to objectively thinking about what the test is proving than the steps to get to the proof.
Noah: I think there is a belief in this industry that if you legoify code creation, you can make it easy enough to hand over to anyone regardless of skill level. However, the reality is that they hit barriers as soon as the preconceived scenarios no longer work. Then they are often left frustrated and no better off then when they started. Unfortunately, there are no shortcuts to learning to code.
There’s Not an Efficient Way to Determine Ripple Effect
Noah: When we reuse code, in development code, we have tests to validate the effects of the changes we made in that shared codebase. However in test code, most development shops do not have anything to validate the test code. What does that mean for someone making a change in reused test code? Well, it means you must manually validate that the changes you made, output what you expect for not only your test, but every test that uses that shared code. This means that the reduced maintenance and reduced test development time can quickly be negated.
Eric: I think this will often come down to how painful it is to run all of your tests. If you are in a position where you know before executing, which tests are altered by your changes, and your tests don’t take very long to run, then the feedback loop isn’t likely an issue for you and the risk of breaking something else in the test suite is negligible. Sure, you may find that unrelated tests break now and then when you start monkeying around in the code, but there are few things that can counter that problem. Firstly, if you start off with the impression that the code you write now will be reused in maybe not the same context in the future, it forces you to think of new, and sometimes, much more elegant ways to write your test code. Secondly, if there are failing tests due to new test code, it is just as likely that your new code is actually great and the surrounding code in the failing test has a flaw that is now uncovered by your change in which case, you just found more opportunities to harden your test code.
Proper Logging Can Mitigate Complexity
Eric: There are some situations where we need many iterations of the same steps to validate things like fidelity and in those cases, there have been no clear way of breaking the complexity of the test down by moving steps into separate cases or scenarios. In one particular situation, it was recommended that we split the test and have multiple versions of the code for easier readability in output but we found that the issue was just in the way the test was reporting to us which iteration of the test failed. A far simpler solution ended up being overcoming the flaw in our output by simply adding a case ID to the logger. Instead of spending time duplicating our test code, we made a one line change to our test data and tricked the output into giving us better info. I think problems like this are often linked to the complexity that code reuse may introduce but in reality, are different issues.
Noah: This sounds like a self induced problem. Sounds like you tried to save keystrokes by cramming too much into a single test which made it hard to understand where failures where coming from. So rather than solve the problem, code reuse, you bolted on some additional complexity to manage the lack of clarity.
Code Reuse Should Be Reserved for Helper Functions
Noah: Imagine you were in charge of building a bridge. You would probably start by designing something on paper. Many areas of bridge building are the same, but it would be unacceptable to have a sketch of a bridge where the middle section says “please refer to the work I did on that other bridge.” You may, however, hit some predefined key combination in CAD that will automatically insert a joist.
Code reuse should be reserved for similar functionality. You can write little functions that are designed to speed up development but not fundamentally replace areas of your test code.
Eric: I don’t think we disagree on this point, we may just interpret what code reuse really implies for the test suite we are envisioning. I think that when we start splitting down test steps into smaller and smaller bits of logic, those bits of logic become more generic and applicable in more places. If we abstract steps enough then we get to a point where steps may not have any original code but just reference other steps to accomplish one goal. In my ideal scenario, the only thing unique to an entire case or scenario would be the THEN statement or the assertion made at the end of an “it.”
Maintenance Cost Is Potentially Reduced with Code Reuse
Eric: More reuse means less code and less overall opportunities to need to maintain test code. I’ve seen this hurt productivity in the past when we had multiple ways of handling dragging and dropping elements for different browsers and for different types of workflows in the SUT. In this example, we had 4 ways of performing an action. Originally this was one step, but it was duplicated and changed to support a new workflow, now we had two ways to do the same action. The next duplication came from having to add additional browser support, now we had four ways to do the same action. This worked for a while until a breaking change occurred which started failing all four, but instead of having to fix one method of dragging and dropping, we ended up spending way more time fixing four separate instances. Because we dealt with all four issues at the same time, we were able to devise a reusable method to handle all four conditions in a more elegant way, but I can’t help wondering how much time it would have saved us to search for a more elegant solution for all four instances as the need arose rather than being bombarded by them all at once.
Noah: Let me tackle this on two fronts. First, code is not a scarce resource. People argue that code reuse saves time, but if it adds additional complexity, it may not really be worth it; codebases will only grow more complex over time.
Second, if you are spending a bunch of time fixing tests, there is a problem with your tests. You either have tests that aren’t testing at the right level, or you are testing something that changes too often for automation.
Less Effort Required for Review
Eric: At the risk of promoting bad assumptions about code review, code reuse means less of a need for critical review of predefined and working patterns. If I create a step in test code that is entirely composed of other steps which have already been written, reviewed, and proven in a test pipeline, then the reviewer does not need to devote time to critically examining all of the code that performs the test steps. In most cases, it is safe to assume that the steps work fine and we can zero in on our assertions and the immediately preceding step.
Noah: So this actually scares me. The moment we start allowing ourselves slack on using critical thought during code review, is the moment we will get errors, guaranteed. Just because a function has been vetted previous doesn’t mean your current use makes any sense. If I had a vetted function that added two numbers, and I write a test that shoves two strings through that code, without critical review of my code, I just introduced bad code into our code base.
Noah Kruswicki, Software Development Engineer in Test (SDET) III, has been with Accusoft since 2016. With a degree in Computer Science from Lakeland University in Sheboygan, WI, Noah prides himself in his career accomplishments. Noah has led many automation efforts and introduced several new ideas into Accusoft’s testing on teams including SDK, PrizmDoc Cloud, OnTask, and internal development. In his spare time, Noah enjoys watching the Jacksonville Jaguars, Milwaukee Bucks, and the Wisconsin Badgers. Additionally, Noah spends time paddle boarding and is a self-proclaimed local foodie.
Eric Goebel, Software Development Engineer in Test (SDET) III, has worked with all of the PrizmDoc teams since joining Accusoft in 2017. He is currently assisting teams working on PrizmDoc Viewer and PrizmDoc Editor. During college, he focused on game development and brings his advocacy for the player to bare for PrizmDoc users. Eric’s career interests revolve around automation technologies and pursuing effective models for continuous delivery and continuous integration while striving to be a mentor in all things testing. Eric worked to expand testing approaches taken to achieve high quality in our products and our internal pipeline. He has presented on these topics at the local Quality Assurance Association of Tampa Bay. In his spare time, he enjoys video games and creating tools and utilities for games as well as spending time with his family and two dogs.