Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Improve Unit Tests #160
Improve Unit Tests #160
Comments
Also minor coding style adjustments. Relates #160.
It's not possible to use the logger from the unit test framework inside test cases. To this end, one can use the macros ERROR, INFO, and VERBOSE. Relates #160.
This makes it consistent with BOOST_ASSERT. Moreover, we can now use CAF_REQUIRE in the unit test framework with the same semantics as BOOST_REQUIRE. Relates #160.
|
The testing framework violates the CAF style guide:
We also need "API compatibility" to the boost testing framework for Boost.Actor (i.e. patching one using find/replace operations only to get the other). I noticed at least one incompatibility: Boost defines Why no Once it actually uses the CAF coding style (as opposed to the VAST coding style |
Why not CAF_CHECK_EQUAL(x, y);
CAF_CHECK_NOT_EQUAL(x, y);
CAF_CHECK_GREATER_THAN(x, y)as opposed to: CAF_CHECK(x == y);
CAF_CHECK(x != y);
CAF_CHECK(x > y);I hope that's convincing enough. |
|
Sorry, your argument is not convincing because you miss the point:
For example, instead of Although header-only has its benefits, I would much rather have it compiled, to be honest. Re-compiling the unit test boilerplate code every time only increases build times unnecessarily. Boost offers both, so maybe you can adopt this behavior if you insist of a header-only use for your purposes. :) On more thing: In Boost, you have to register your test suites manually to set timeouts: http://www.boost.org/doc/libs/1_35_0/libs/test/example/unit_test_example_03.cpp. Just in case you want to have the timeout configurable in the future. I don't mind to omit this for CAF and simply use a default timeout. Your watchdog setup doesn't work on all platforms, though. For MSVC I had to replace the "detached thread with abort in it" approach with condition variables and mutexes (see https://github.com/actor-framework/actor-framework/blob/develop/unit_testing/test.cpp). Luckily it's a straightforward fix. |
|
Have you actually looked at the output of I'm fine with going back to compiled if we really have to, but do you really think it's going to matter much? It's not a big library, a handful of classes with only 2 compiled ones, and the include footprint would be just reduced by one header file ( I'll fix the timeout right now. |
Uh... that's quite a lot of under-the-hood-magic. I would prefer a Regarding compile time: yes, I do think it matters. Compilation time already is a problem with CAF. Anything that helps reducing it is more than welcome. Maybe you can move the implementation to an |
|
And sorry for being nitpicky. :) |
|
Sure, the
So 209 would be straight text replacement. So the remaining 42 (!) tests may need manual inspection to see whether the I like the idea of the implementation header, will go with that. |
|
I've address all of your comments and squashed the commits into a single one. |
|
Sorry for a noob question, but do we really need rolling out a new unit testing library? I've checked a ping-pong test and seen nothing but quite ordinary assertions and unreachable code markers. Why not just use something like Catch, for example? It is a simple one-file header-only library which can be linked to the project using (to your choice):
I understand that a lot of work is already done by @mavam, but I don't really want CAF to take the path of Boost or Qt. I hope one day C++ will have a better infrastructure, with some commonly accepted wide-spread libraries instead of lots of reinvented wheels, and I believe that Catch have a great chances to be among them.
Catch can do that with a single assertion macro (to some extent, of course, as it uses some very interesting C++ template magic internally). |
|
Thanks for sharing your thoughts on how to improve CAF. Unfortunately there appears to be a disconnect in our goals and what you seem to advocate. First, we want to minimize the amount of dependencies to external projects. Ideally CAF only requires a compiler, and nothing else. A simple unit test library does not need to have several thousands line of code (Catch has close to 10k and the Boost Test headers alone twice as much). With the current iteration of CAF test, we have a few hundred lines of maintainable, slim C++11 code. When we need to make a change, we just do it quickly. We can choose between a shared/static library as well as a header-only version. This makes it very attractive for downstream projects that depend already on CAF. Second, we like the Boost community and want both their attention and users. There are multiple efforts underway to make it possible for CAF to replace several subsystems of CAF with Boost. This is why CAF test is API-compatible to Boost Test, albeit to a subset. As an aside, we don't want tie our hands with specific build frameworks. Biicode appears to get a lot of traction and we're about to finish the integration, but don't want to depend on it. This leaves us with git submoduels and CMake. We use the former only for optional parts of CAF, and the test framework is part of the core. This also makes external projects less attractive.
The current porting takes place in a different branch. The branch you looked at presumably just contains a brief example illustrating its use.
So does CAF test, by the way, just with much less boilerplate. What are the real advantages of Catch? |
|
Thank you for that detailed answer. I get your opinion and agree with most of your points.
I've mentioned Catch as one of possible and easy to get running option, but I'm going to feel absolutely comfortable if CAF is going to eventually adopt Boost Test.
While that sounds quite reasonable, that would you do if you would replace |
|
We're not going to switch to Boost.Test for CAF. We are working on a "boostified" version of CAF called Boost.Actor that we propose to the Boost community. For the Boost version, we're going to re-use Boost components like Boost.Test for instance. That's why we need the API compatibility. The feedback from the Boost community so far was mixed, so we don't know if CAF/Boost.Actor will ever be accepted as a library. In any case, CAF will remain and will be available as standalone version (just like ASIO coexists with Boost.ASIO). |
|
I close this issue since we are already working on a pull request. |
The unit tests in CAF could really use some cleanup and a more fine-grained structure. Several tests in fact check multiple things instead of only test one particular feature. @mavam will probably donate his powerful unit testing framework from VAST, which would be a huge step in the right direction.