| Info |
|---|
Before reading this page, it is highly recommended you read the builds page. |
| Table of Contents |
|---|
What is the Unit Tests Page?
The Unit Tests page allows you to see the results of any Testing Actions that were completed during the build. The Unit Tests page can be located by viewing an individual build. Once unit tests have been added to a stage, all results for those tests can be found on this page.
The Unit Tests page is broken up into two sections (Unit Tests page is shown below). At the top of the page is a summary of all your tests which displays how many tests completed successfully, failed etc. Below the summary is a full list of every test including how long the test took to run, the test result, etc.
Note that for running builds, unit tests are not finalised until the current stage has completed. Be
When navigating to the Unit Tests tab and assuming some tests have run, you'll see a web page similar to the one pictured below. The page is split into two, the top part is the test result header and the table underneath lists the tests.
Note: Unit tests are created at the end of each stage they were run in. Be sure to refresh the page once the build has completed to get an accurate representation of unit tests in the result header.
...
Unit Tests
...
Test Result Header
Summary
The results of all unit tests are grouped into several categories and these categories are displayed in the Unit Tests Summary at the top of the Unit Tests page. All categories (except New Failures) will only be shown if you have at least one test that falls into that category. For example if all of your tests succeeded you will not see the Errors category in the Unit Tests Summary.
Each category in the summary can be clicked which will filter the table below it to only show tests in that category.
Each type of unit test result in the test result header is clickable and will filter the table below it to show only those types of tests. The result header will vary depending on the results of the tests. To keep things simple we've chosen to hide/show certain unit test results in the result header. Unit tests can have many different results so if there are no tests for a certain result we hide it (except for New Failures). Regardless of the test outcomes we always show New Failures as the default view. The complete list of test result headers are:
Total, Passed, Fixed, Failed, New Failures, Error, Inconclusive and , Not Run and Shelved.
...
Display By
...
The Display By drop down changes the way the unit test data is viewed.
Test View
Tests
All unit tests can be displayed individually by selecting Tests in the Display By dropdown. By displaying the individual tests, you can drill down and check why a certain test failed by clicking the test's name. If this test has failed previously then the error message will include a link to the build where this build first failed.
When displaying by tests, all
...
columns are sortable but only name, fixture, namespace and assembly are taken into account when the search feature is used.
...
Fixture/Namepace/Assembly View
An alternate way of viewing unit test data is to show the test fixtures, namespaces or assemblies. These 3 are hierarchical so assembly will show just the assembly name, namespace will show assembly and namespace and fixture shows all three. These views have a duration column which is a sum of all the tests underneath it, but substitute the test view's result field for a "Total" field which is a count of the number of tests in the view type. The follow image shows all Failed tests displayed by Fixture name.
Filter Breadcrumb
The filter breadcrumb is located to the right of the "Display By" drop down box. The Filter Breadcrumb is constructed as you start clicking links or selecting a view type. While viewing the unit test data you can click on a Fixture, Namespace and Assembly. If there's no previous filter, then clicking one of these will filter the current set of results by that type regardless of it's parent. For example, clicking on a Fixture named "WindowsTests" will filter the results and show tests that belong to the Fixture name "WindowsTests" regardless of which Namespace or Assembly they belong to. If you want to view test in the Fixture "WindowsTests" for a specific Assembly/Namespace then you first select the Assembly, then the Namespace, then finally the Fixture. Filters take into account the current view type, the current test result type selected in the test result header and whatever search text is entered in the search box. The following image is an example of a filter that can be best explain as "Show me all newly failed tests for the fixture ExpectExceptionTest under the namespace of NUnit.Core.Tests in the assembly nunit.core.tests.dll.
Display By Fixture/Namespace/Assembly
Unit tests can be grouped and viewed by the assembly, namespace or fixture. When grouping, test results are not shown. The information displayed when grouping includes the total amount of time it took to run every test in that group and the total number of tests in that group.
If your tests are grouped and you shelve a group, you will be shelving every test within that group. See the Shelving section below for more information.
Shelving Tests
Unit tests can be shelved so that their results are ignored the next time a test is executed. This is useful when you have conditions on your stages that stop the build when there are failed tests. All shelved tests are ignored by the build process so any shelved tests that fail will not count towards your failed builds. You can choose to shelve a test, an entire fixture and.or an entire namespace. When shelving a test, you can specify whether the tests should be shelved for just this configuration or for every configuration in the current project.
Shelving Individual Tests
Individual tests can be shelved by selecting Display By Tests and clicking the shelve button next to the test you want to shelve. This will only shelve a single test.
Shelving Entire Fixtures, Namespace and Assemblies
Entire fixtures, namespace and assemblies can also be shelved. If you select fixture, namespace or assembly in the Display by dropdown, you can shelve an entire group by clicking Shelve. For example, if you Display by Namespace, clicking shelve next to the appropriate namespace will shelve every test in that namespace.
Shelving Dialog
When you shelve a test, the Shelve Test dialog will appear with the following properties:
Comment
The comment allows you to associate any additional information when shelving a test.
Apply to all tests in
This property allows you to set whether this test should be shelved for just the current configuration or for all configurations in the current project.
Unshelving Tests
All tests, fixtures, namespaces and assemblies can be unshelved by clicking the unshelve button next to the appropriate tests.
Continua allows you to shelve Fixture/Namespaces but that translates into shelving all tests underneath them. If a test has been shelved via its fixture or namespace then unshelving the test will unshelve all tests in that fixture/namespace. For example, if you're viewing tests via the test view and they've all been shelved as a result of their fixture being shelved then you'll get a pop up warning you of what the action of unshelving will do. In the following scenario, a fixture has been shelved and we're trying to unshelve an individual test in that fixture.
I just shelved a single test and it shelved more than one!
A test is identified by its Assembly, Namespace, Fixture and Test Name. When using some unit test applications like NUnit, it's possible for a test fixture to have multiple tests executed with the same test name. In Continua, they'll show up as separate tests but will all be treated as the same test when shelving. This means shelving a test that has multiple tests with the same name will also shelve those. Don't be alarmed when you shelve a test that has 10 other tests with the same name in the same fixture and see all of those 10 tests shelved as well.