I tried version 0.1 and it didn't even load the test list. Just downloaded and installed version 0.2, after few attempts it finally loaded the test list, but when I try to run all tests or any selection or debug a test it fails with the following message in output: ------ Run test started ------ An exception occurred while invoking executor 'executor://boosttestexecutor/v1': Object reference not set to an instance of an object. ========== Run test finished: 0 run (0:00:00.3314) ==========
For those suffering from an object reference not set error, you can download this extension via your browser then double click on it to launch the VSIX Installer. The extension will install correctly without error.
To uninstall you can use the Extensions & Updates dialog successfully.
All test suites are displayed in the category "Traits" and not in "Class". Compared to the built-in unit test framework where all tests are wrapped in classes the Boost unit test framework does not use classes but functions. Therefore "Traits" seemed to be the better place for showing test suites (and all tests within a suite).
I am trying to run the unit test adapter with tests using fixtures.
When using per test case fixtures (http://www.boost.org/doc/libs/1_44_0/libs/test/doc/html/utf/user-guide/fixture/per-test-case.html) my test cases are somehow not displayed in the VS Test Explorer.
Here is the source code I used for testing:
ObjectUnderTest = new CObjectUnderTest();
BOOST_TEST_MESSAGE( "setup fixture" );
BOOST_TEST_MESSAGE( "teardown fixture" );
BOOST_CHECK(ObjectUnderTest->IsAnything() == true);
I also tried to use test suite level fixtures (http://www.boost.org/doc/libs/1_44_0/libs/test/doc/html/utf/user-guide/fixture/test-suite-shared.html). In this case I got following error when running the test:
An exception occurred while invoking executor 'executor://boosttestexecutor/v1': StartIndex cannot be less than zero.
Do you have any advice how to get the test adapter running with tests that use fixtures?
I'm trying to make the latest boost test adapter (0.7) work with VS2012 Update 4. I'm running W8.1 update 1 x64 and I am building against boost 1.53. My goal is to make the tests work with TFS 2012 build.
I have trouble making the adapter work reliably in visual studio and I can't make it work at all manually;
I've now created a minimal test project with a single suite with a single failing test, it will sometimes work in visual studio from the test explorer but I cannot make it work through vstest.console.exe in any form, I've tried X64 and X86 they both give the same error;
Error: An exception occurred while invoking executor 'executor://boosttestexecutor/v1': Object reference not set to an i
nstance of an object.
What I'm trying to run is the following (in the correct output dir of course)
vstest.console BoostUnitTests.exe /Platform:X86 /UseVsixExtensions:true /InIsolation
vstest.console BoostUnitTests.exe /Platform:X64 /UseVsixExtensions:true /InIsolation
Any hints on what I'm doing wrong or how to make this work or get some kind of diagnostics output? I've tried DebugView/FusLogVw etc, but I've come up with nothing that helps.
Please advice, I'm at the verge of building my own adapter now :)
I've just discovered that the adapter is in fact crudely parsing the C++ files to discover tests, I was under the impression that the adapter utilized the generated exe to list/execute tests. I'm starting to understand why this does not work now. I also noticed that this parsing is directly implemented in the test adapter and there is apparently no intermediate files (e.g. xml) or tiers that would make this more modular, is that correct? I.e. the adapter will always depend on the source code to be available during discovery?
After doing a bit of research, may I offer my humble suggestions to work around some of the issues I discovered;
1. AFAIC understand the adapter depends on the VS IDE to work;
- Allow a "preprocessor" mode, where a pre-generated test database is loaded from e.g. an XML file, this will allow "discovering" and executing tests via vstest.console etc. and also enable automatic tests in TFS builds, with a bit of work.
2. Separate the "preprocessor" test discoverer into an assembly which can be run separately as well as from the adapter.
- Allow pre-generated test database to be generated both from the IDE and possibly manually or from e.g. msbuild.
When Boost Test gets the ability to list/discover tests in the near future, the pre-generated file can be directly generated/converted. Apparently this is already implemented according to the author;
first of all, thank you for your elaborate comment on the Boost Unit Test extension for Visual Studio.
When reading through your comment, I wondered why you want to use the tool vstest.console.exe for executing your Boost unit tests. This is not necessary since all Boost tests are deployed in an executable which can run without any other tool such as vstest.console.
For additional test output, please refer to
I'm aware of the fact that I can run the boost unit test executable to get the output directly, but Team Foundation Server builds already has support for running automated tests through the Visual Studio Test Runner in the default build template and has a special UI section for tests. Other (.Net) unit test projects and VS C++ unit test project support this mechanism already, and therefore integrate with TFS build automatically. The boost unit test adapter would IMHO be more useful if it also supported this use-case.
Running the exe directly from a TFS build process template and make it integrate with the build as other unit tests does would require creating a unit test adapter that would parse the output from the exe, which your adapter already does :)
When I create tests, they are not loaded by the adapter, nor do they appear in the Test Explorer.
The test project is created in an empty solution and the only change is that BoostUnitTest.cpp contains the line BOOST_CHECK(true). The project links against libboost_unit_test_framework-vc110-gd-1_55.lib, and the executable can run with no issue (reports two test cases, both tests succeed). The configuration is Debug|x64, and the only changes from default are the Boost library to use, and library and include directories. I am using Visual Studio 2012 Premium Update 4 and version 0.6.2 of the adapter. I have tried installing the extension both through the Visual Studio GUI and using the downloaded vsix.
On a side note, when I create a new boost test project, I get the following error:
"The file targetver.h could not be found within the project templates. continuing to run, but the resulting project may not build properly."
This is the test output:
------ Discover test started ------
========== Discover test finished: 0 found (0:00:00,0858069) ==========
I'm getting the error below while the adapter is trying to load tests from solution (multiple test projects), subsequently only part of tests are loaded.
An exception occurred while test discoverer 'BoostTestDiscoverer' was loading tests. Exception: An item with the same key has already been added.
Could you advise me a work around? Is it possible to tackle the problem in a bugfix release? ;)
Wherever I have a boost test which is effectively disabled by putting inside #ifdef SOMETHING, tests execution breaks with an error below.
An exception occurred while invoking executor 'executor://boosttestexecutor/v1': The given key was not present in the dictionary.
Sample minimal code to reproduce the problem:
BOOST_CHECK( true );
This is a known issue.
Currently, it is not possible to inactivate a test case by using the #if(n)(def)...#endif pattern.
Although the code is displayed as inactive in Visual Studio, the unit test parser does not ignore code within this pattern. If you need to inactivate the code, you have to do it manually by using /*...*/.
We will fix this issue as soon as possible.