Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

When navigating to the Unit Tests tab and assuming some tests have run, you'll see a web page similar to the one pictured below.

 

(((((((( PIC HERE ))))))))

 

The page is split into two, the top part is the summary field and the table underneath lists the tests. The summary field will vary depending on the results of the tests. To keep things simple we've chosen to hide/show certain unit test results in the summary field. Unit tests can have many different results so if there are no tests for a certain result we hide it (except for New Failures). Regardless of the test outcomes we always show New Failures as the default view.

...

Info

Before reading this page, it is highly recommended you read the builds page.

Table of Contents

What is the Unit Tests Page?

The Unit Tests page allows you to see the results of any Unit Testing Actions that were completed during the build. The Unit Tests page can be located by viewing an individual build. Once unit tests have been added to a stage, all results for those tests can be found on this page.

The Unit Tests page is broken up into two sections (Unit Tests page is shown below). At the top of the page is a summary of all your tests which displays how many tests completed successfully, failed etc. Below the summary is a full list of every test including how long the test took to run, the test result, etc.

Image Added

Note that for running builds, unit tests are not finalised until the current stage has completed. Be sure to refresh the page once the build has completed to get an accurate representation of unit tests in the summary fieldresult header.

 


...

Unit Tests Summary

Using "Display By"

The Display By drop down changes the way tests are viewed.

Test View

...

The results of all unit tests are grouped into several categories and these categories are displayed in the Unit Tests Summary at the top of the Unit Tests page. All categories (except New Failures) will only be shown if you have at least one test that falls into that category. For example if all of your tests succeeded you will not see the Errors category in the Unit Tests Summary.

Each category in the summary can be clicked which will filter the table below it to only show tests in that category. 

The complete list of test result headers are:

Total, Passed, Fixed, Failed, New Failures, Error, Inconclusive, Not Run and Shelved.


Display By Tests

All unit tests can be displayed individually by selecting Tests in the Display By dropdown. By displaying the individual tests, you can drill down and check why a certain test failed by clicking the test's name. If this test has failed previously then the error message will include a link to the build where this build first failed.

When displaying by tests, all columns are sortable but only name, fixture, namespace and assembly are taken into account when the search feature is used.

...

((( PIC test view))))

 

Fixture/Namepace/Assembly Views

When test view isn't selected you view tests by fixture, namespace or assembly. These 3 are hierarchical so assembly will show just the assembly name, namespace will show assembly and namespace and fixture shows all three. These views have a duration column but substitute the result field for a "Total" field which is a count of the number of tests in them.

 

((( PIC fixture view )) 

Using Summary Fields

Each unit test result in the summary field is clickable and will filter the table below it to show only those types of tests. Clicking a summary field will also take into account the current filter of test results


Display By Fixture/Namespace/Assembly

Unit tests can be grouped and viewed by the assembly, namespace or fixture. When grouping, test results are not shown. The information displayed when grouping includes the total amount of time it took to run every test in that group and the total number of tests in that group.

If your tests are grouped and you shelve a group, you will be shelving every test within that group. See the Shelving section below for more information.

Image Added


Shelving Tests

Unit tests can be shelved so that their results are ignored the next time a test is executed. This is useful when you have conditions on your stages that stop the build when there are failed tests. All shelved tests are ignored by the build process so any shelved tests that fail will not count towards your failed builds. You can choose to shelve a test, an entire fixture and.or an entire namespace. When shelving a test, you can specify whether the tests should be shelved for just this configuration or for every configuration in the current project.

Shelving Individual Tests

Individual tests can be shelved by selecting Display By Tests and clicking the shelve button next to the test you want to shelve. This will only shelve a single test.

Shelving Entire Fixtures, Namespace and Assemblies

Entire fixtures, namespace and assemblies can also be shelved. If you select fixture, namespace or assembly in the Display by dropdown, you can shelve an entire group by clicking Shelve. For example, if you Display by Namespace, clicking shelve next to the appropriate namespace will shelve every test in that namespace.

Shelving Dialog

Image Added

When you shelve a test, the Shelve Test dialog will appear with the following properties:

Comment

The comment allows you to associate any additional information when shelving a test.

Apply to all tests in

This property allows you to set whether this test should be shelved for just the current configuration or for all configurations in the current project.


Unshelving Tests

All tests, fixtures, namespaces and assemblies can be unshelved by clicking the unshelve button next to the appropriate tests.

Continua allows you to shelve Fixture/Namespaces but that translates into shelving all tests underneath them. If a test has been shelved via its fixture or namespace then unshelving the test will unshelve all tests in that fixture/namespace. For example, if you're viewing all passed tests for a certain assembly then clicking on the Error summary field will show all error'ed tests for that assembly. It will also use the table's search field to filter results and further filters the tests based on what type of view you're currently on (Tests, Fixtures, Namespace, Assembly).

 

Building up a filter

A filter will take into account the current view type (Tests, Fixture, Namespace, Assembly)

 

 

 

 

 

 

 

tests via the test view and they've all been shelved as a result of their fixture being shelved then you'll get a pop up warning you of what the action of unshelving will do. In the following scenario, a fixture has been shelved and we're trying to unshelve an individual test in that fixture.


I just shelved a single test and it shelved more than one!

A test is identified by its Assembly, Namespace, Fixture and Test Name. When using some unit test applications like NUnit, it's possible for a test fixture to have multiple tests executed with the same test name. In Continua, they'll show up as separate tests but will all be treated as the same test when shelving. This means shelving a test that has multiple tests with the same name will also shelve those. Don't be alarmed when you shelve a test that has 10 other tests with the same name in the same fixture and see all of those 10 tests shelved as well.