xUnit.net runner for Visual Studio 2012

Hi,

I an new to VS and xunit. I had created two tests in two separate files, when tried to run 'All Tests' only test present in one file got executed, other test didn't execute, can someone please let me know how to do that

Is this possible to see Console output/Trace output from the test run? 

I would like to test my NHbiernate queries and see the resulting SQL. Also I would like to see progress of db creation before test run

Click on the test in Test Explorer. The details window at the bottom of Test Explorer will include a link titled "Output". Click this, and you'll see any console/trace/debug output that was captured during the test run.

Is it also possible to see Debug.WriteLine output? And is it possible to see any of this output while I'm paused on a breakpoint while trying to debug the unit tests?

Unfortunately, the design of xUnit.net v1.x is that it buffers all output until the test is complete (a bad decision that we're fixing in v2). However, I'm not 100% sure how the VS test runner would surface live output, so when v2 is getting close to ready, we will need to reevaluate the runner code and see what can be done (for example, detecting that you're running inside of a debugger and dump everything to the Debug console live).

How can I use assembly binding redirects with this test runner? What config file would I put them in? Where? Please advise.

Honestly, I've never investigated the rules.

When xUnit.net creates the AppDomain for running your tests in, it uses the config file for the DLL by default. So, if you add an App.config file to your test project, during compilation that will get copied into the appropriate config file (for example, if your assembly is MyTests.dll, it will be get named MyTests.dll.config). The only thing I'm unsure about is whether .NET will take binding redirects from this config file, or only from the config file that launched the process.

Thank you for the information. Any idea what the config file that launched the process might be, or how to find out? MSTest.exe.config?

Actually, never mind. Putting my binding redirect in the App.config for my test project seems to have done the trick after all. Thanks again!

(Accidentally posted this as a reply to the FsUnit discussion when it should be its own thing)

This test runner seems to copy the assembly to a temporary directory. I've always thought this was an awkward approach as opposed to running the app in place but so be it. The bigger issue is that I can't seem ot get any other files (including files that are set to copy always) to copy to the target directory.

This is a huge problem when needing to tests with static assets. (Or when using ApprovalTests.)

Is there a solution to this? Am I misunderstanding something?

This is the behavior of shadow copying in the .NET framework. The behavior is described here: http://msdn.microsoft.com/en-us/library/ms404279.aspx

My understanding is that any file copied into the binary folder ("bin\Debug", "bin\Release", etc.) or any sub-folder will also be copied when doing shadow copying. Any folder which lives outside of this folder hierarchy will not be copied.

In general, our recommendation for test content is to use embedded resources rather than reading files from disk.

When I try to Build/Rebuild the 41 project solution, the test explorer acts like it is searching for tests (green progress bar fluxing across the top of the console window), but then it fails "An unexpected error occurred. See the "Output" window for more info.  When I check I find the log:

------ Discover test started ------
The request has taken more than 10 seconds to respond, aborting it.
Exception has been thrown by the target of an invocation.
========== Discover test finished: 1 found (0:00:12.981298) ==========

I have over 3400 xUnit tests which I typically test with TestDriven.NET (VS2010 .Net 4.0).  I was hoping to try out this tool, but can't seem to get it to respond.  Is there anyway to increase the response timeout (30 seconds)?

Specs:
Windows 7 Pro
Visual Studio 2012
Projects: .NET 4.5
xUnit 1.9.1

Thanks in advance,

Michael

Sorry, I don't know how the increase the timeout. This is apparently something that is being enforced by Visual Studio, because xUnit.net does not contain any discovery timeout mechanism.

Hey, thanks a lot for making this extension!

I'm having a problem using it with tests written in FsUnit.xUnit. The problem is that F# allows to use spaces, punctuation chars in identifier names (using the double backtick syntax).

To reproduce, create an "F# Console Application" project and install FsUnit.xUnit via NuGet. This will add FsUnitSample.fs with test samples to your project. (make sure you move it up over the Program.fs in solution explorer, otherwise it won't compile)

The tests are visible in the test explorer, but when you try to run them you get
    Catastrophic failure: System.ArgumentException: Type ConsoleApplication1.Tests.Given a LightBulb that has had its state set to true.when I ask whether it is On it answers true could not be found

("Given a LightBulb that has had its state set to true" is a class name, and "when I ask whether it is On it answers true." is a method name.)

Thanks for the bug report. I've opened an issue on the xUnit.net site and will investigate.

http://xunit.codeplex.com/workitem/9830

This test runner seems to copy the assembly to a temporary directory. I've always thought this was an awkward approach as opposed to running the app in place but so be it. The bigger issue is that I can't seem ot get any other files (including files that are set to copy always) to copy to the target directory.

This is a huge problem when needing to tests with static assets. (Or when using ApprovalTests.)

Is there a solution to this? Am I misunderstanding something?

Just updated to the new version earlier today and my unit tests are hanging randomly (some run, others freeze). Resharper seems to run them just fine.

Any ideas?

Hmm... Nothing changed in the runner behavior, only in the addition of traits support. :( Is the code that's hanging publicly available code? Does it repro on all your projects, or just one project in particular?

Hello,

First, thanks a lot for this tool it worked great until then :)

My problem is that when I run my tests I get an error message in the output window :

------ Run test started ------
Updating the layout...

Copying files: Total <1 mb to layout...

Registering the application to run from layout...

Deployment complete. Full package name: "3f477dba-d39c-46e4-b413-6dc2235688b8_1.0.0.0_neutral__m0we3rjtmsxmt"

An exception occurred while invoking executor 'executor://xunit.codeplex.com/VsTestRunner': An item with the same key has already been added.
========== Run test finished: 0 run (0:00:04,7364736) ==========

I didn't do anything particular, the tests just suddenly stopped working.

Here is what did to try to solve the problem :
- Clean the solution
- Try to remove the package with PowerShell but it doesn't exists.
- Try to delete a key in the registry, but it gave me an error
- Reboot the computer
- Change the package GUID in order to change the package full name

Unfortunally none of this worked, I'm kind of stuck :/
Any ideas ?

OK, the problem come from a badly named test, which had the same name than another one.
My bad. 

Phenomenal tooling for VS.net 2012... would be cool if when you do the structure like:

<pre>
wrapper class 
{
    method class1
    {
    }
    method class2
    {
    }
}
</pre>

that somehow, the method names are shown on the screen.

Again phenomenal tooling, and for being free {as in beer, right?}, you can't go wrong.

realizing this may not be clear...

what I'd like to see in the window would be 
WrapperClass.MethodClass1.Method1
WrapperClass.MethodClass1.Method2
WrapperClass.MethodClass2.Method1

so that, using your technique as you showed it on the Tekpub video, it's easier to understand long term.  May be a good thing to do, but I'm not sure.  Your call.

In a later iteration, we will be adding some configuration options to modify the list behavior (today it's fixed as "method name").

I'm using Visual Studio 2012 Express for Web (11.0.50522.1 RCREL) and I get the error "This extension is not installable on any currently installed products". But it seems Express does have a Test Explorer and works with MSTest, so I'm wondering if it's possible to get this working?

I did try hacking this .vsix and adding
<Edition>VWDExpress</Edition>
to <SupportedProducts>, but sadly that didn't work! It did seem to install, but the test explorer still doesn't recognise any xunit tests.

No, it does not support Express.

The limitation is not technical. It is a Microsoft rule that no third party add-ins are allowed to run inside of Express edition. So if you want unit testing inside of Express, you're stuck with MSTest.

This is frustrating!!  I want to use VS Express for my open-source project, and not require that contributors have a specific version of VS; and I'd also prefer xUnit over MSUnit (everybody's doing it...)

I understand if you don't want to break your employer's rules, but there are a number of other 3rd party add-ins in VS Express - including Script#, also by a Microsoft employee.  I'm not trying to bust Nikhil, I just think this rule should change, for the good of the ecosystem.

I've been lobbying hard to get this overturned (for the good of the whole unit testing community -- including NUnit, not just us), but it has so far met with extremely firm "no"s. I expect that voices from the community would have more weight here than internal people, to be honest.