到目前为止,我已经避免了测试多线程代码的噩梦,因为它似乎是一个太大的雷区。我想知道人们是如何测试依赖于线程的代码以获得成功执行的,或者人们是如何测试那些仅在两个线程以给定方式交互时才会出现的问题的?
对于今天的程序员来说,这似乎是一个非常关键的问题,恕我直言,将我们的知识集中在这个问题上是很有用的。
到目前为止,我已经避免了测试多线程代码的噩梦,因为它似乎是一个太大的雷区。我想知道人们是如何测试依赖于线程的代码以获得成功执行的,或者人们是如何测试那些仅在两个线程以给定方式交互时才会出现的问题的?
对于今天的程序员来说,这似乎是一个非常关键的问题,恕我直言,将我们的知识集中在这个问题上是很有用的。
当前回答
确实很难!在我的(c++)单元测试中,我按照使用的并发模式将其分解为几个类别:
Unit tests for classes that operate in a single thread and aren't thread aware -- easy, test as usual. Unit tests for Monitor objects (those that execute synchronized methods in the callers' thread of control) that expose a synchronized public API -- instantiate multiple mock threads that exercise the API. Construct scenarios that exercise internal conditions of the passive object. Include one longer running test that basically beats the heck out of it from multiple threads for a long period of time. This is unscientific I know but it does build confidence. Unit tests for Active objects (those that encapsulate their own thread or threads of control) -- similar to #2 above with variations depending on the class design. Public API may be blocking or non-blocking, callers may obtain futures, data may arrive at queues or need to be dequeued. There are many combinations possible here; white box away. Still requires multiple mock threads to make calls to the object under test.
题外话:
在我所做的内部开发人员培训中,我教授了并发的支柱和这两种模式,作为思考和分解并发问题的主要框架。显然还有更先进的概念,但我发现这组基础知识可以帮助工程师摆脱困境。正如上面所描述的,它还会导致代码更具单元可测试性。
其他回答
有一篇关于这个主题的文章,在示例代码中使用Rust作为语言:
https://medium.com/@polyglot_factotum/rust-concurrency-five-easy-pieces-871f1c62906a
总而言之,诀窍在于编写并发逻辑,使其对涉及多个执行线程的非确定性具有健壮性,使用通道和condvars等工具。
然后,如果这就是您构建“组件”的方式,那么测试它们的最简单方法是使用通道向它们发送消息,然后阻塞其他通道以断言组件发送某些预期的消息。
链接到的文章完全使用单元测试编写。
对于Java,请参阅JCIP的第12章。有一些具体的例子,可以编写确定性的多线程单元测试,以至少测试并发代码的正确性和不变量。
用单元测试“证明”线程安全要危险得多。我相信在各种平台/配置上进行自动化集成测试会更好。
如果你正在测试简单的new Thread(runnable).run() 您可以模拟Thread来按顺序运行可运行对象
例如,如果被测试对象的代码像这样调用一个新线程
Class TestedClass {
public void doAsychOp() {
new Thread(new myRunnable()).start();
}
}
然后模拟new Threads并按顺序运行runable参数会有所帮助
@Mock
private Thread threadMock;
@Test
public void myTest() throws Exception {
PowerMockito.mockStatic(Thread.class);
//when new thread is created execute runnable immediately
PowerMockito.whenNew(Thread.class).withAnyArguments().then(new Answer<Thread>() {
@Override
public Thread answer(InvocationOnMock invocation) throws Throwable {
// immediately run the runnable
Runnable runnable = invocation.getArgumentAt(0, Runnable.class);
if(runnable != null) {
runnable.run();
}
return threadMock;//return a mock so Thread.start() will do nothing
}
});
TestedClass testcls = new TestedClass()
testcls.doAsychOp(); //will invoke myRunnable.run in current thread
//.... check expected
}
听着,要做到这一点并不容易。我正在做一个本来就是多线程的项目。事件来自操作系统,我必须并发地处理它们。
处理测试复杂的多线程应用程序代码的最简单方法是:如果它太复杂而无法测试,那么您做错了。如果您有一个单独的实例,其中有多个线程作用于它,并且您无法测试这些线程相互踩在一起的情况,那么您的设计需要重做。它既简单又复杂。
有许多方法可以为多线程编程,以避免线程同时通过实例运行。最简单的方法是使所有对象都是不可变的。当然,这通常是不可能的。因此,您必须在设计中确定线程与同一实例交互的地方,并减少这些地方的数量。通过这样做,您可以隔离多线程实际发生的几个类,从而降低测试系统的总体复杂性。
但是您必须意识到,即使这样做,您仍然不能测试两个线程相互践踏的每一种情况。要做到这一点,您必须在同一个测试中并发地运行两个线程,然后准确地控制它们在任何给定时刻执行的行。你能做的就是模拟这种情况。但这可能需要您专门为测试编写代码,这充其量是迈向真正解决方案的半步。
测试代码是否存在线程问题的最好方法可能是对代码进行静态分析。如果您的线程代码没有遵循有限的线程安全模式集,那么您可能会遇到问题。我相信VS中的代码分析确实包含了一些线程的知识,但可能不多。
看,就目前的情况来看(可能还会持续很长一段时间),测试多线程应用程序的最佳方法是尽可能降低线程代码的复杂性。最小化线程交互的区域,尽可能地进行测试,并使用代码分析来识别危险区域。
近年来,在为几个项目编写线程处理代码时,我多次遇到过这个问题。我提供了一个迟来的答案,因为大多数其他答案虽然提供了替代方案,但实际上并没有回答关于测试的问题。我的答案是针对多线程代码没有替代方案的情况;为了完整性,我将讨论代码设计问题,但也将讨论单元测试。
编写可测试的多线程代码
首先要做的是将生产线程处理代码与所有执行实际数据处理的代码分开。这样,数据处理就可以作为单线程代码进行测试,多线程代码所做的唯一事情就是协调线程。
The second thing to remember is that bugs in multithreaded code are probabilistic; the bugs that manifest themselves least frequently are the bugs that will sneak through into production, will be difficult to reproduce even in production, and will thus cause the biggest problems. For this reason, the standard coding approach of writing the code quickly and then debugging it until it works is a bad idea for multithreaded code; it will result in code where the easy bugs are fixed and the dangerous bugs are still there.
相反,在编写多线程代码时,必须抱着一种从一开始就避免编写错误的态度来编写代码。如果您已经正确地删除了数据处理代码,线程处理代码应该足够小——最好只有几行,最坏也就几十行——这样您就有机会在不编写错误的情况下编写它,当然也不会编写很多错误,如果您了解线程,请慢慢来,并且小心。
为多线程代码编写单元测试
一旦尽可能仔细地编写了多线程代码,仍然值得为该代码编写测试。测试的主要目的与其说是测试高度依赖于时间的竞争条件错误(不可能重复测试这种竞争条件),不如说是测试防止这种错误的锁定策略是否允许多个线程按预期进行交互。
To properly test correct locking behavior, a test must start multiple threads. To make the test repeatable, we want the interactions between the threads to happen in a predictable order. We don't want to externally synchronize the threads in the test, because that will mask bugs that could happen in production where the threads are not externally synchronized. That leaves the use of timing delays for thread synchronization, which is the technique that I have used successfully whenever I've had to write tests of multithreaded code.
If the delays are too short, then the test becomes fragile, because minor timing differences - say between different machines on which the tests may be run - may cause the timing to be off and the test to fail. What I've typically done is start with delays that cause test failures, increase the delays so that the test passes reliably on my development machine, and then double the delays beyond that so the test has a good chance of passing on other machines. This does mean that the test will take a macroscopic amount of time, though in my experience, careful test design can limit that time to no more than a dozen seconds. Since you shouldn't have very many places requiring thread coordination code in your application, that should be acceptable for your test suite.
Finally, keep track of the number of bugs caught by your test. If your test has 80% code coverage, it can be expected to catch about 80% of your bugs. If your test is well designed but finds no bugs, there's a reasonable chance that you don't have additional bugs that will only show up in production. If the test catches one or two bugs, you might still get lucky. Beyond that, and you may want to consider a careful review of or even a complete rewrite of your thread handling code, since it is likely that code still contains hidden bugs that will be very difficult to find until the code is in production, and very difficult to fix then.