Those aren't bugs until you report them as such!!!
Seriously, don't assume we know about problems. They may not be as general as you imagine. Of course, you'll need to be fairly specific in a bug report.
When writing "learning tests" it is incredibly helpful to be able to write the output of method calls and details of member variables. This can be done when using the NUnit GUI runner by simply outputting to the Console (eg Console.WriteLine).
When I use the MSTest unit testing framework I can use the methods of Trace and Debug (inside the System.Diagnostics namespace) to write output which is collected and presented in an Output element of the test results.
The NUnit Test Adapter offers either of these. So any output that I require for developing my learning tests (or any other type of test where I want see output) is ignored, forcing me to either use MSTest or the NUnit GUI runner, thus negating the need for this Test Adapter.
Working OK in VS2012 so far. Works with simple tests or test cases generated using TestCaseSourceAttribute. I haven't tested TestCaseAttribute but at this point I am confident it works just fine too.
My only wish would be that the optional description of a test could be visible in Test Explorer somewhere, but I am unsure if this would be the responsibility of this adapter or if Visual Studio simply doesn't allow for it.
Oh, and one other wish, but I think that's for NUnit itself: A way to give a test a name, like a TestNameAttribute, or a named parameter in the TestAttribute.
Showing the description would be a shared responsibility: VS to provide a way to tell it and us to pass the description.
You probably know that there's a way to specify a name for a test case, e.g. on the TestCaseAttribute. We never thought about needing it for a simple Test. Please file an (NUnit) bug for this.
The old binaries work but you can't build it from source under the RC without a few changes. I'll do a new release that's buildable as time allows and follow it up with another a 1.0 release after RTM.
See the thread at: http://social.msdn.microsoft.com/Forums/en-US/tfsbuild/thread/9ac71e24-8f8f-42c0-ac8e-857a70085014/
The unanswered question is whether the Test Adapter needs a change to accommodate the change made in VS (Microsoft.VisualStudio.TestPlatform.Utilities.dll), or whether the introduced bug is unrelated to the test adapter at all.
MS is aware of the problem (though currently playing the 20-questions game), but should the problem be logged as a bug here?
I have a similar problem as @houlgap. After installing Update 2 to Visual Studio 2012 Ultimate, I no longer see *any* NUnit tests in my Test Explorer window. None. Zero. Zip.
I'm having a problem where I switched from MSTest to NUnit using this adapter. Running tests works fine, but debugging a test does not work -- no breakpoint is hit and it doesn't appear that the test runs at all. I can debug an MSTest test in the same solution.
I tried creating a small example solution and debugging works fine, so it is something specific to this solution but I don't know what.
The output window doesn't appear to have any useful messages in it that I can see.
Any idea what to try?
I'm getting the following exception each time Test Discovery finishes:
Exception thrown discovering tests in d:\Sources\MyProject\MAIN\Source\UI.Tests\bin\Debug\MyProject.UI.Tests.dll
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
Server stack trace:
at System.ModuleHandle.ResolveType(RuntimeModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at NUnit.Core.TestRunner.Load(TestPackage package)
at NUnit.Util.TestDomain.Load(TestPackage package)
at NUnit.VisualStudio.TestAdapter.NUnitTestDiscoverer.DiscoverTests(IEnumerable`1 sources, IDiscoveryContext discoveryContext, IMessageLogger logger, ITestCaseDiscoverySink discoverySink)
It seems that the Visual Studio Reference Assembly folders are being ignored...
Note: I'm a newbie w/ the Test Adapter and somewhat with NUnit as well (previously used mstest).
Question: can the unit test methods with the "Ignore" attribute be excluded from the run list by default, or at least require another check before running (just in case)? Or is there a way to do this already? thank you.
replying to my own post... looks like items w/ Ignore will appear in the Skipped Tests section, as expected, at least when I selectively click on items to run. so probably a non-issue/false alarm.
overall, seems like a very good/useful VS/nunit extension. my compliments to the developer. i'm using vs2012 (update 2) w/ nunit 2.6.2.
We are building VS2012 projects, but still using TFS 2010 server. On our build server, TFS 2010 build service and VS2012 are installed. Can NUnit Test Adapter be used to integrate the automatic testing with the build process in this environment?
I see this message in the Tests output window:
------ Discover test started ------
Assembly not supported: D:\Development\Clients\Leica\Workspaces\shared\CLIP\main\TempBin\x86\XXX_x86.dll
An exception occurred while test discoverer 'NUnitTestDiscoverer' was loading tests. Exception: Object reference not set to an instance of an object.
========== Discover test finished: 0 found (0:00:00.8630494) ==========
My solution contains native C++ projects (XXX is such a project), mixed-mode assemblies in C++/CLI and managed C# NUnit tests.
The message is displayed simply to inform you that the assembly is not supported. It doesn't indicate any kind of a problem. Originally, the adapter simple let this error pass silently, but that led to problems when an assembly that _should_ contain tests was bypassed. It would be better to only show the message when it is meaningful, but there's no way for the adapter to know that.
I would like to turn this feature off, while developing tests. If I have multiple tests, I would like to run only the one I am developing, but this runs all tests every time I rebuild the one I am working on.
Any suggestions?
When using the NUnit Gui, you are able to re-run the last set of tests executed. Such an option could be made available with the adapter.
However, since the adapter merely runs what Test Explorer tells it to run, such a feature enhancement would have to be made in Test Explorer rather than the adapter.
Let me start off by saying, I am not very knowledgable with NUnit. I have a set of NUnit tests that I have adopted. I am running VS 2012 Ultimate. I installed the NUnit Test Adapter via extensions and updates. I have NUnit 2.6.2 installed on my system (Win 7 Pro). I can run the tests in NUnit, but I cannot run them in VS Test Runner via the Test Explorer, they do not appear in the Explorer window. In addition, The IntelliSense does not pick up the attributes. I don't know what I am doing wrong and I cannot find any data source on the internet that would explain what I am missing.
The goal here is to run these tests in a Build process eventually, and I am having trouble just running them local before attempting to run them via a build config.
Never mind, the mistake was mine. I was compiling a solution above the solution that contained the tests, so basically, I had the wrong solution loaded for the test work.
vs professional: 11.0.51106.01 update 1, nunit test adapter beta 4: 0.94, nunit: 2.5.10
Should we be able to see why something has failed? ie. "expected 2 was 1"? I am working around this by adding AssertionException to my debugged exceptions so it breaks on throw.
In my test output window I see:
------ Run test started ------
========== Run test finished: 1 run (0:00:02.2012201) ==========
When debugging I see in the debug output:
A first chance exception of type 'NUnit.Framework.AssertionException' occurred in nunit.framework.DLL
I've just installed Beta 3-2 in VS2012 Premium. Is it possible to specify an alternate folder for a solution that contains the test assemblies? I'm looking for something similar to the folder override option that ReSharper's Unit Test running functionality provides.
Thanks in advance
Probably not - since VS provides us with the path to each test assembly. I can look into it more closely if you provide more info about what you want. Googling 'resharper alternate folder' didn't give me anything useful.
Resharper provides the option to "Run tests from" a different directory. Here is the link to the help details - http://www.jetbrains.com/resharper/webhelp/Reference__Options__Tools__Unit_Testing.html
In short, our build process copies project assemblies to a custom directory structure so they don't end up where Visual Studio expects to find them. There are a number of reasons we do this however, some of those reaons have been forgotten. I was looking to try out the new VS20120 Premium testing features with our existing process. It could also be time to review our build process.
I'll check that reference out. Thanks.
In fact, NUnit itself (not the adapter) is built in the same way you describe. The result is that I can't use the adapter to run the NUnit tests. I've been thinking of this as simply a limitation of the TestPlatform itself and it's not a problem for me anyway, since I prefer to run NUnit tests with NUnit (eating my own dogfood).
But the approach you suggest might expand the usefulness of the adapter, so let's take a look at it!
Thanks for looking into this.
I've made changes to one of the VS solution's projects in order to get the NUnit tests to run. I had to change it so all the project references had "copy local" set to true for the tests to be found by NUnit (all the dependencies need to be in the assembly output directory).
We have a large number of projects in our solutions and having "copy local" set to false produces faster build times as the referenced assemblies aren't copied multiple times to the various project output directories. Having the ability to point NUnit to another directory for tests would allow "copy local" to remain false. I would need to have an AfterBuild step to copy all the resulting assemblies to the common test directory.
&size=large&version=c367bd67-6b11-4707-a9a5-d32e4563605b){kind=avatar}
View Large
