The Boost Unit Test Adapter extension has been developed for the Visual Studio 2012, Visual Studio 2013 and Visual Studio 2015 Test Explorer window to allow integrated test execution of C++ projects utilizing the Boost Unit Test Framework.
In case of any issues please file an...
I tried version 0.1 and it didn't even load the test list. Just downloaded and installed version 0.2, after few attempts it finally loaded the test list, but when I try to run all tests or any selection or debug a test it fails with the following message in output: ------ Run test started ------ An exception occurred while invoking executor 'executor://boosttestexecutor/v1': Object reference not set to an instance of an object. ========== Run test finished: 0 run (0:00:00.3314) ==========
For those suffering from an object reference not set error, you can download this extension via your browser then double click on it to launch the VSIX Installer. The extension will install correctly without error.
To uninstall you can use the Extensions & Updates dialog successfully.
I've been enjoying this extension and it was key in my decision to use Boost.Test and TurtleMock at my company. I have an interest in ensuring this is a reliable and sustained project but unfortunately I haven't been able find any kind of publicly available homepage, issue tracker, or source code. Does any of this exist or is this page the only public resource?
That said, I do have an issue to report, fortunately it is easy to fix. The problem is that after creating a new Boost Unit Test Project the project fails to build. The problem appears to be an unset Platform Toolset.
Thank you for this extension, since we started with boost.test I've been using it almost every day.
Visual Studio 2013
Boost Test Runner Adapater 0.9.4
First of all, great extensions, it is really useful.
I am trying to integrate the boost test adapter to our build system. The unit tests are running fine in our developers machines.
The issue is when i follow (https://msdn.microsoft.com/en-us/library/ms253138.aspx) to add the adapter to the build system I get an 'Object reference not set to an instance of an object'. My assembly file specification points to the .exe generated by the boost test project. Does anyone has gone trough the process of integrating with TFS? Any thoughts?
This is a fantastic plugin and I was using it heavily on Visual Studio 2013. Now I have moved to 2014 for the support of C++14 and other stuff. It will be really really help to evaluate if we get this plugin support for evaluating.
Hello, Is there any momentum on supporting Visual Studio 2015? Having this plug-in will help in evaluating Visual Studio 2015 for my company's use. I am fine to take a developer release to test as well. Please let me know.
I tested your adapter and it looks promising. All our tests are discovered but when I want to run tests it just outputs:
-- Run test started --
and then a few seconds later
-- Run test finished: 0 run
Since we use a pretty complex build system it would be nice to have a log file where your adapter tells me a bit more about what went wrong. Is there such a file and if yes where is it? Or do you have any idea what could be the problem?
Okay so I adjusted some environment variables and now most of the tests run. However I have one group of tests which use the BOOST_AUTO_TEST_CASE_TEMPLATE macro. These tests are nicely detected and even list a test case for each type which is great. But when I want to run them there is an error:
Boost Unit Test Adapter: Error while executing tests: StartIndex cannot be less than zero.
Parameter name: startIndex
I tried to reproduce the issue you described and can give you the following hints:
Using the following example code I was able to successfully execute the unit tests:
typedef boost::mpl::list<int, long, double> testTypes;
BOOST_AUTO_TEST_CASE_TEMPLATE( TypeTest, T, testTypes )
BOOST_REQUIRE_EQUAL( sizeof(T), unsigned(4));
Also, if you wrap the test case within a test suite, it works fine.
But it is important that you avoid using ifdefs because the parser, which is responsible for finding test cases within a source code file, is not able to handle ifdefs correctly.
If, for example, you use the following code snippet (the XXX is not set in the project properties)
// source code from above
the tests cases will be displayed in the test explorer (which is wrong), but they are not been "compiled into" the executable (which is correct), resulting in the error you described.
This is a known limitation of the adapter. We are working on a solution for that issue.
Thank you for the quick answer. Although the test file does not contain any ifdefs and I removed all ifdefs from the test fixture the same errors still occur. I looked through some of the included files and there are many ifdefs so I am not completely sure if this is the problem or not. Also all the testcases listed in the testexplorer are executed if I run the executable directly.
I will try again when you added support for ifdefs.
I get the 'StartIndex cannot be less than zero' error, when attempting to run a unit test that has nested test suites, example:
I already mentioned it earlier: the adapter does NOT support nested test suites. The reason for that limitation is that Visual Studio does not support displaying test trees with more than 1 child in the Test Explorer window, i.e. neither nested test suites (boost adapter) nor nested test classes (native test framework) can be displayed.
BOOST_AUTO_TEST_CAST_TEMPLATE doesn't work if the mpl list is a list of classes.
Say I have a class Foo
the test automatically generated name will come out as
But the parser reads it as
and then I get the dreaded "StartIndex cannot be < 0" when I try to run the test.
I have some tests that run for as much as an hour (stress testing) I have never seen one of them report in the test explorer taking more than 2 minutes (sometimes it says 1) and yet the logs for the test clearly show that it ran all the way through.
Would it be possible to have the full output of a test case?
If not then the first failure would be more helpful than last.
Test Name: my
Test FullName: mySuitemy
Test Source: <hide>.cpp : line 41
Test Outcome: Failed
Test Duration: 0:00:00.0000001
Result Message: three
Result StackTrace: <hide>.cpp line 45
Is there a way to get all the failures?
Even if the first two are not critical, there may be a reason for the order and it makes sense to fix the first one first, doesn't it?
I have just a few tests in my project (300+) and the performance of running them is prohibitively bad - when running as a post build step I don't even notice the tests executing as it takes <1s and using this extension with run all tests option the same tests take over a minute.
Another performance issue is that the whole solution is build rather than the unit test projects - in my case this means a very extended link duration which again does not exist when running the tests as a post build step.
Is there any chance for this to improve? Am I doing anything wrong?
I'm using the latest Boost Unit Test Adapter version (0.9.2 ) with VS2013 update 4. I can't run any test successfully. I always got the following error:
Boost Unit Test Adapter: Error while executing tests: The given key was not present in the dictionary.
The following test could not be executed: -> Can_do_something
Also, it would be nice to group tests by "project" and then by "traits". When working with large solution with multiple test projects, only one grouping level is not enough.
for grouping we rely on the functionality of the test explorer in Visual Studio. Here it is not possible to group in different levels. So, we can only offer the grouping modes Visual Studio offers.
Regarding the problem with your unit test "Can_do_something", I need more information: can you post the beginning of the test, so how the BOOST_TEST_CASE macro is used and how it is embedded in the file:
- Are there any #ifdefs used?
- Do you use more than one lever of test suites?).
I don't need the actual test code, only the scaffolding of the test within the test file.
Sorry for the delay. I just tested with 0.9.3 and I have the same issue.
There's no special #ifdefs.
The project is a console project built with VS2013
The exact output is:
------ Run test started ------
Executing: -> should_handle_empty_stop_time
Error: The given key was not present in the dictionary.
========== Run test finished: 0 run (0:00:00,2230223) ==========
The test scaffolding:
I didn't notice there was such a project type. I tested and it works.
However, I have dozens of unit tests projects and recreating each one is not an option. Could you tell how this project type differs from standard console project so I can update my projects. I looked into a vcxproj created with your template but I can't find the differences.
Sorry the the delayed response.
I just tried your code scaffold within a normal console project (exe, default settings when creating it). It works fine, so all 4 unit tests are shown in the test explorer and can be executed successfully. Expecially the stdafx files needed to be modified compared to the default state after creating a new project:
- add the header to Boost library (stdafx.h)
- define boost test module (stdafx.cpp)
#define BOOST_TEST_MODULE MyTest
These settings don't need to be edited when using the provided "Boost Unit Test Project". But except that there is no obvious difference between a normal console project and the project we provide. So you don't need to replace the existing projects by the "special" ones.
If you add a new file to your test project, I recommend using the provided "Boost Unit Test File" template. It makes the start easier, I think.
I have this same problem.
my project is of boost type and the only change i had to do is to make it a multi-byte character set project. each of my files has the structure:
TEST_SUITE( X )
|-- TEST_CASE( A1 )
|-- TEST_CASE( A2 )
|-- TEST_CASE( A3 )
And the only extra thing I have tried is to delete the testCaseCollection.xml that contains this dictionary so it gets regenerated on compilation.
Please help me with this situation, because i dont know what more to do.
I looked in the testCaseCollection.xml file and found that my issue is due to the fact that the test name format is incorrect: instead of "testSuite/testName" it's saved as "testSuitetestName". Adding the missing slash allowed my to run the test.