In-depth interpretation of TDD (reproduced)

The address of the reproduced article, if there is any infringement, please contact to delete: In-depth interpretation of TDD

Reproduced text content:

Structure of this article:

  • What is TDD
  • Why TDD
  • how TDD
  • FAQ
  • learning path
  • further reading

What is TDD

TDD has a broad sense and a narrow sense. It is often referred to as the narrow sense of TDD, which is UTDD (Unit Test Driven Development). TDD in a broad sense is ATDD (Acceptance Test Driven Development), including BDD (Behavior Driven Development) and Consumer-Driven Contracts Development.
The TDD mentioned in this article refers to TDD in a narrow sense, that is, “unit test-driven development”.

TDD is a core practice and technique in agile development and a design methodology. The principle of TDD is to write unit test case code before developing functional code, and the test code determines what product code needs to be written. TDD is a core practice of XP (Extreme Programming). Its main driver is Kent Beck.

TDD has three meanings:

  • Test-Driven Development, test-driven development.
  • Task-Driven Development, task-driven development, to analyze the problem and carry out task decomposition.
  • Test-Driven Design, design improvement under the protection of test. TDD does not directly improve design capabilities, it just gives you more opportunities and guarantees to improve your design.

Why TDD

Traditional Coding VS TDD Coding

Legacy encoding

  • Requirements analysis, can’t figure out the details, let him go, start writing first
  • If you find that the details of the requirements are not clear, please confirm with the business personnel
  • Confirmed several times and finally finished writing all the logic
  • Run it and test it, shit, it really doesn’t work, debug
  • After a long time of debugging, it finally worked
  • Turn to testing, QA detects bugs, debugs, and patches
  • Finally, the code works
  • When I saw that the code was rotten like shit, I didn’t dare to move it. If I moved it, I had to manually test it, let QA test it, and have to work overtime…

TDD coding style

  • Decompose tasks first and separate concerns (demo later)
  • Column Example, use instantiated requirements to clarify the details of the requirements
  • Write tests, only focus on the requirements, the input and output of the program, and don’t care about the intermediate process
  • Write the implementation, regardless of other requirements, just use the simplest way to meet the current small requirements
  • Refactoring, using methods to eliminate bad smells in the code
  • After writing, test it manually. There is basically no problem. If there is a problem, make up a use case and fix it
  • Transfer test, small problems, supplementary use cases, fixes
  • The code is clean and the use cases are complete, submit with confidence

Benefits of TDD

Reduce developer burden
Through a clear process, let us only focus on one point at a time, with less thinking burden.

Protection net
The advantage of TDD is that it covers complete unit tests and provides a protective net for the product code, allowing us to easily accept changes in requirements or improve code design.
So if your project requirements are stable, you can do it all at once, and if there are no subsequent changes, you will enjoy less benefits from TDD.

Clarify requirements ahead of time
Writing tests first can help us think about the requirements and clarify the details of the requirements in advance, instead of discovering unclear requirements halfway through the code.

Quick Feedback
There are many people who say that when TDD, the amount of my code increases, so the development efficiency decreases. However, if there is no unit test, you have to test manually, and you have to spend a lot of time preparing data, starting the application, jumping to the interface, etc., and the feedback is very slow. To be precise, fast feedback is the benefit of unit testing.

How to TDD

TDD

The basic process of TDD is: red, green, refactoring.
A more detailed process is:

  • write a test case
  • run test
  • Write just enough implementations to make the tests pass
  • run test
  • Identify bad smells and modify the code with tricks
  • run test

You may ask, I write a test case, it will obviously fail, should I run it?
Yes. You may think that there are only two cases of success and failure in the test. However, there are countless kinds of failures. Only by running the test can you ensure that the current failure is the failure you expected.
Everything is to make the program meet expectations, so that when an error occurs, the error can be quickly located (it must be caused by the code that was just modified, because the code was still in line with my expectations a minute ago).
In this way, a lot of time in debugging code is saved.

The Three Rules of TDD

  1. No production code is allowed except to make a failing unit test pass
  2. In a unit test, only write what is just enough to cause a failure (compilation errors are also failures)
  3. Only allow writing production code that is just enough to make a failing unit test pass

What happens if it is violated?
In violation of the first article, the product code is written first, so what is the purpose of this code to achieve? How do you make sure it actually works?
Violation of the second article, writing multiple failed tests, if the test fails to pass for a long time, it will increase the pressure on the developer. In addition, the test may be refactored, which will increase the modification cost of the test.
In violation of Article 3, the product code implements functions beyond the current test, then this part of the code is not protected by the test, and it is not known whether it is correct, and manual testing is required. Maybe this is a non-existent requirement, which increases the complexity of the code out of thin air. If it is an existing requirement, then the subsequent tests will pass directly after writing, destroying the rhythm of TDD.

I think its essence is:
Separate concerns and wear one hat at a time
In the process of our programming, there are several concerns: requirements, implementation, and design.
TDD gives us three clear steps, each of which focuses on one aspect.
Red: Write a failed test. It is a description of a small requirement. You only need to care about input and output. At this time, you don’t need to care about how to implement it.
Green: Focus on realizing the current small requirement in the fastest way, and don’t care about other requirements, and don’t care how terrible the quality of the code is.
Refactoring: There is no need to think about requirements, nor is there pressure to implement. You only need to find out the bad smell in the code, and use a method to eliminate it, so that the code becomes a clean code.

Attention Control
Human attention can be actively controlled or passively attracted. Switching attention back and forth consumes more energy and makes thinking less complete.
Using TDD development, we need to take the initiative to control attention. When writing a test, we find that a class is not defined, and the IDE prompts a compilation error. If you create this class at this time, your attention will not be on the demand, and you have already switched to In terms of implementation, we should concentrate on writing this test, think about whether it expresses the requirements, and then start to eliminate compilation errors after confirming that it is correct.

Why can’t many people do TDD?

Will not split tasks reasonably
Before TDD, tasks should be split, and a large requirement should be split into multiple small requirements.
You can also split multiple functions.

Can’t write tests
What is an effective unit test? Many people write tests, but they don’t even know what they are testing, and they may not even have assertions. They can be verified by console output and visual comparison.
A good unit test should conform to several principles:

  • Simple, only test one requirement
  • Conforms to the Given-When-Then format
  • high speed
  • contains assertions
  • can be repeated

Can’t write just the implementation
Many people can’t focus on the current needs when writing and implementing, and accidentally realize other needs, which destroys the sense of rhythm.
When it is realized, it will not be taken in small steps.

Will not refactor
I don’t know what Clean Code is, I can’t see Smell, and I haven’t refactored in time. When I want to refactor, it’s already difficult to start.
I don’t know how to eliminate Smell with a proper “hands-on”.

Infrastructure
For a specific technology stack, the unit test infrastructure is not well established, resulting in the inability to focus on test cases when writing tests.

Example

Write a program to count the frequency of each word in a text file words.txt.
To keep things simple, assume:

  • words.txt contains only lowercase letters and spaces
  • Each word contains only lowercase letters
  • Words are separated by one or more spaces

As an example, suppose words.txt contains the following:

the day is sunny the the
the sunny is is

Your program should output the following, sorted in reverse order of frequency:

the 4
is 3
sunny 2
day 1

Please don’t read on yet, think about what you would do.
(Think for 3 minutes…)

When a novice gets such a requirement, he will write all the code into a main() method. The pseudocode is as follows:

main() {
    // read the file
    ...
    // separate words
    ...
    // group
    ...
    // sort in reverse order
    ...
    // concatenate string
    ...
    // Print
    ...
}

The idea is very clear, but it is often written in one go, and finally runs, but the output does not meet expectations, and then breakpoint debugging starts.

This code does not have any encapsulation. This is why many people jump out immediately when they hear that some companies limit a method to no more than 10 lines, saying that this is impossible, what can 10 lines do, our business logic is very complicated…
What’s wrong with this code?

  • not testable
  • not reusable
  • Difficult to locate the problem

Okay, let’s do TDD, how do you write the test code that reads the file and outputs the console?
Of course, we can isolate IO through Mock and Stub, but is it really necessary?

Kent Beck was asked this question:

Can you really measure everything? Will even getters and setters be tested?

Kent Beck said: The company hired me to realize business value, not to write test code.
So I only write test code where I don’t have confidence.

For our program, reading files and printing to the console are calling system APIs, so we can be very confident. The least confident thing is the business logic in the middle that needs to be written by itself.
So we can do some encapsulation of the program. “The Way of Clean Code” says that methods can be extracted wherever there are comments, and the method names are used instead of comments:

main() {
    String words = read_file('words.txt')
    String[] wordArray = split(words)
    Map<String, Integer> frequency = group(wordArray)
    sort(frequency)
    String output = format(frequency)
    print(output)
}

In this way, can we write unit tests for split, group, sort, format alone?
Of course, their input and output are very clear.

Wait, you might say, isn’t it Test Driven Design? How did you start designing? good question!

Does TDD need to be designed in advance?

Kent Beck does not design in advance, he will choose the simplest use case, write it directly, and pass the test with the simplest code. Gradually add tests, make the code complex, and use refactoring to drive the design.
In this requirement, what is the simplest scenario?
That is, the file content is empty, and the output is also empty.

Of course, for complex problems, you may need to add new use cases while writing, but for such simple topics, you can basically figure out in advance which use case to drive and which product code to use.
The following use cases can probably be imagined:

  • “” => “”
  • “he” => “he 1”, a word that drives the format string code
  • “he is” => “he 1\r\\
    is 1”, two different words, drive out the code for splitting words
  • “he he is” => “he 2\r\\
    is 1”, with the same word, drive out the grouping code
  • “he is is” => “is 2\r\\
    he 1”, drive out the sorting code after grouping
  • “he is” => “he 1\r\\
    is 1”, multiple spaces, perfect code for splitting words

Martin Fowler’s point of view is that before we wrote code, we had to do Big Front Up Design, and we had to design all the details before we started writing code.
And after we have the refactoring tool, the pressure of design is much less, because of the protection of the test code, we can refactor the implementation at any time. But that doesn’t mean we don’t need to design in advance. Designing in advance allows us to discuss with others and iterate a few times before starting to write code. Iterating on paper is always faster than changing the code.
I personally agree with Martin Fowler’s approach, first design in my head (of course, my brain is not enough, so I draw on paper), and then start writing after a few iterations. In this way, I will still use the simplest implementation to pass the test. But when refactoring, there is a direction, and the efficiency is higher.

Going back to this program, I found that the current encapsulation is not at an abstract level, and a more ideal design would be:

break down tasks

main() {
    String words = read_file('words.txt')
    String output = word_frequency(words)
    print(output)
}

word_frequency(words) {
    String[] wordArray = split(words)
    Map<String, Integer> frequency = group(wordArray)
    sort(frequency)
    return format(frequency)
}

At this time, there are two options. Some people like top-down, some people like bottom-up. I personally prefer the former.

From now on, just follow the red-green-refactor cycle.
Most TDD is not done well, that is, there is no previous process of task decomposition and listing of examples.
If you want to watch the TDD process, you can refer to my live broadcast.
Or if needed, I can also record a video on this topic.

FAQ

Why do we have to write the test first, is it okay to make up the test later?

OK, but after writing the implementation, write the test immediately, and use the test to verify the implementation. If you test manually first, debug the code, and then supplement the unit test, you will feel very tasteless and increase the workload.
You can enjoy fast feedback whether you test first or last, but if you test first, you can enjoy another benefit, reducing rework with intent-driven programming. Because your test code is the client (caller) of the product code, you can write your ideal look (method name, parameters, return value, etc.) in the test code, and then implement the product code, compared to writing the implementation first and then writing Test, the former has less rework.

I just wrote a test, but I haven’t written the implementation yet. Knowing that running the test will definitely report an error, why do you want to run it?

In fact, the results of the test are not only passed or failed, because there are many possibilities when it fails. So run the test knowing that it will fail, the purpose is to see if the expected error is reported.

It’s good to take small steps, but do you really need such small steps?

If you take too many steps, it is easy to tear the eggs.
When practicing, you need to develop the habit of taking small steps, and you can freely switch the size of your steps when you are working.
When you are confident, you can take bigger steps, and when you are not confident, you can immediately switch to the mode of taking smaller steps. If you can only take big steps, it is difficult to take small steps.

Will the test code become a burden to maintain?

The TDD process is also followed during maintenance. First, modify the test code to look like a requirement change, let the test fail, and then modify the product code to make it pass.
This way you are not maintaining test cases, but utilizing test cases.

Why fast implementation?

In fact, the binary search method is used to isolate the problem. After passing the test through hardcode, it is basically determined that there is no problem with the test. At this time, the product code is implemented. If the test fails, it is a problem with the product code.
So small steps are mainly to isolate the problem, that is, you can say goodbye to Debug ?.

Why should test code be simple?

If a test fails and the fix is to change the test code instead of the production code, then the test code is poorly written.
When the test code is simple enough, if a test fails, there is enough confidence to conclude that it must be a problem with the production code.

When is TDD not suitable?

If you are doing exploratory technical research (Spike), do not need long-term maintenance, and the cost of building a test infrastructure is high, then it is better to test manually.
In addition, there are “legacy systems with extremely poor testability” and systems that “use test-unfriendly technology stacks”. Doing TDD may outweigh the benefits.

Learning Path

  1. “Effective Unit Testing”
  2. “The Way to Clean Code”
  3. “Refactoring”
  4. Transformation Priority Premise
  5. 《Test-Driven Development by Example》
  6. 《Growing Object-Oriented Software, Guided by Tests》

Extended reading

  • Let’s talk about TDD again
  • Why is TDD so difficult?
  • TDD Topics