The Programmer's Apprenticeship - 04: Reflections on Testing

Word count: 1.4kReading time: 8 min

 2018/09/27

This is the fourth post in the reading notes series for How to Stop Sucking and Be Awesome Instead (published in Chinese as 《程序员的修炼——从优秀到卓越》). The author, Jeff Atwood, is one of the founders of Stack Overflow. His articles cover a wide range of topics. He is a seasoned programmer, manager, and entrepreneur. This book discusses many things beyond programming. Whether you are a junior engineer or a senior engineer, this book is worth reading. As your experience grows, every time you re-read this book, you will gain new insights. Just as the Chinese subtitle, “From Good to Great,” suggests, the author points out the path for you, but whether you can succeed depends on your own cultivation.

I will excerpt some wonderful remarks from the book and sometimes add my own insights or experiences. The outline of the reading notes is consistent with the outline of the book itself. This is also a method I learned from another source and have been using: “How to Read a Book”. I record it for my own frequent review and for readers’ reference. Below is the The Programmer’s Apprenticeship reading note series:

Reading note (reviewed October 2026): This post retains the structure of the 2018 reading notes. The source essays date from 2005–2009; the additions below fill in the hardware case and distinguish legacy tools from modern error handling.

Unit Testing Is Necessary

One value of unit testing is that it makes you think about how to verify code. In Good Test, Bad Test (2005), Jeff criticizes developers who enter a few values, click some buttons, and hand the code to QA if no unhandled exception appears. His claim that most developers do not test is commentary from that period, not a current industry statistic. An absence of exceptions does not establish correct output or behavior.

Unit testing forces you to think about a series of difficult but unavoidable questions for the code you just wrote:

How should I test this code?
What kind of tests should I perform?
What is the normal situation?
What are the possible exceptions?
How many external dependencies do I have?
What system failures might I encounter?

In practice, tests need explicit expectations and assertions, with meaningful boundary and failure cases. Interactions with external dependencies may also need integration or end-to-end tests; unit tests alone cannot establish that the whole system works. Google’s code review guidance likewise asks reviewers to check whether tests are appropriate and whether their assertions would fail when the behavior breaks.

Sometimes It’s a Hardware Problem

The original case is Sometimes It’s a Hardware Problem (August 11, 2006). A dual Opteron 250 server with 8 GB of RAM, Windows Server 2003 x64, and Virtual Server 2005 R2 kept rebooting. BIOS and driver updates resolved a memory-node warning but did not stop the reboots. Comparing configurations with Memtest86+ and Prime95 showed stability with 4 GB and failures with 8 GB. Passing the more CPU-focused small FFT test did not rule out a memory-related problem.

After the author changed the BIOS memory speed from 200 MHz to 166 MHz, Memtest86+ and the Prime95 blend test passed. That result supports investigating the memory configuration and its interaction with the platform; it does not prove that a particular DIMM was defective or guarantee permanent stability. For a similar investigation today, record hardware, firmware, drivers, workload, and reproduction conditions, then compare one change at a time. The 2006 settings are not a prescription to underclock memory or immediately update the BIOS.

We shouldn’t always point the finger at software. Sometimes we need to go back and check the hardware and its configuration.

Exception-Driven Development

Developers should review exception and error reports continuously, while also listening to user feedback. In Exception-Driven Development (April 16, 2009), Jeff describes Stack Overflow’s custom fork of ELMAH at the time; this does not establish its current production configuration. The original ELMAH 1.2 SP2 targets ASP.NET on .NET Framework through HTTP modules and handlers. Those integration steps do not transfer directly to ASP.NET Core.

The author asks whether fixing a bug users will never encounter is worth the time. In practice, having no reports rarely establishes that a failure can never occur. The source essay also explicitly supports test-driven development and says the approaches can coexist. Tests verify expected behavior; production feedback reveals cases the test environment missed. Each serves a different purpose.

The essay recommends iterating on production exceptions. Applied today, that feedback process needs a few qualifications:

Complete the tests and review relevant to the change, then control the rollout according to product risk.
Use exceptions, error rates, latency, and user feedback to investigate, recording versions, triggering conditions, and scope of impact.
Prioritize by severity and impact, make the necessary fixes, and add regression tests. The original “three most serious problems” is an iteration example, not a rule requiring architectural changes each time.
After deployment, check whether user-visible outcomes improve, then repeat. Fewer log entries alone do not prove that a problem is gone.

Exception logs are only one signal. Incorrect content or excessive latency can be a user-visible failure even with HTTP 200; error counts also need request volume as a denominator. Google SRE’s monitoring chapter identifies latency, traffic, errors, and saturation as four important signals. Feedback can improve a system, but a few iterations cannot guarantee rock-solid reliability.

Version update: ASP.NET Core 10 / .NET 10. The official error-handling documentation covers exception-handling middleware and IExceptionHandler. Starting in .NET 10, handled exceptions for which TryHandleAsync returns true suppress middleware diagnostics, such as logs and metrics, by default. This differs from .NET 8/9 and can be changed with SuppressDiagnosticsCallback. Review that policy during an upgrade: no log entry does not necessarily mean no error. The Developer Exception Page is for the Development environment and may expose stack traces, query parameters, cookies, and request headers. Do not expose these diagnostics in production, and restrict access to centralized logs.

The following Douban blurbs are historical excerpts; their traffic figures are not 2026 statistics.

How to Stop Sucking and Be Awesome Instead is a compilation of the best articles from the Coding Horror blog. The book is divided into 8 chapters, covering topics such as time management, programming methods, web design, testing, user needs, the Internet, game programming, and technical reading. The topics selected by the author are all pain points in a programmer’s career. Many articles have high click-through rates and reply rates on blogs and the Internet. —— from Douban

Jeff Atwood founded the Coding Horror blog (codinghorror.com) in 2004 to record his thoughts and bits and pieces of his software development experience. Today, the blog has nearly 100,000 visits per day. Readers participate in comments, and various views and wisdom collide passionately there. —— from Douban

The writing style of How to Stop Sucking and Be Awesome Instead is humorous, understanding, and caring; it is suitable for programmers at all stages from novice to veteran, and also suitable for students of computer science and related majors who are about to become programmers. How to Stop Sucking and Be Awesome Instead can help readers pay more attention to the human nature and humanistic factors of technical work, thereby achieving a successful turning point in their programmer career. —— from Douban

About Me && Blog

Below is my personal introduction and related links. I look forward to exchanging ideas with fellow professionals. “When three walk together, one can always be my teacher!”