Let’s clarify a bit what Play Mode and Edit Mode test means from the Unity Test Framework perspective:
## Edit Mode tests
**Edit Mode** tests (also known as Editor tests) are only run in the Unity Editor and have access to the Editor code in addition to the game code.
With Edit Mode tests it is possible to test any of your [Editor extensions](https://docs.unity3d.com/Manual/ExtendingTheEditor.html) using the [UnityTest](./reference-attribute-unitytest.md) attribute. For Edit Mode tests, your test code runs in the [EditorApplication.update](https://docs.unity3d.com/ScriptReference/EditorApplication-update.html) callback loop.
> **Note**: You can also control entering and exiting Play Mode from your Edit Mode test. This allow your test to make changes before entering Play Mode.
Edit Mode tests should meet one of the following conditions:
* They should have an [assembly definition](./workflow-create-test-assembly.md) with reference to *nunit.framework.dll* and has only the Editor as a target platform:
```assembly
"includePlatforms": [
"Editor"
],
```
* Legacy condition: put tests in the project’s [Editor](https://docs.unity3d.com/Manual/SpecialFolders.html) folder.
## Play Mode tests
You can run **Play Mode** tests as a [standalone in a Player](./workflow-run-playmode-test-standalone.md) or inside the Editor. Play Mode tests allow you to exercise your game code, as the tests run as [coroutines](https://docs.unity3d.com/ScriptReference/Coroutine.html) if marked with the `UnityTest` attribute.
Play Mode tests should correspond to the following conditions:
* Have an [assembly definition](./workflow-create-test-assembly.md) with reference to *nunit.framework.dll*.
* Have the test scripts located in a folder with the .asmdef file.
* The test assembly should reference an assembly within the code that you need to test.
```assembly
"references": [
"NewAssembly"
],
"optionalUnityReferences": [
"TestAssemblies"
],
"includePlatforms": [],
```
## Recommendations
### Attributes
Use the [NUnit](http://www.nunit.org/)`Test` attribute instead of the `UnityTest` attribute, unless you need to [yield special instructions](./reference-custom-yield-instructions.md), in Edit Mode, or if you need to skip a frame or wait for a certain amount of time in Play Mode.
### References
# Edit Mode vs. Play Mode tests
Let’s clarify a bit what Play Mode and Edit Mode test means from the Unity Test Framework perspective:
## Edit Mode tests
**Edit Mode** tests (also known as Editor tests) are only run in the Unity Editor and have access to the Editor code in addition to the game code.
With Edit Mode tests it is possible to test any of your [Editor extensions](https://docs.unity3d.com/Manual/ExtendingTheEditor.html) using the [UnityTest](./reference-attribute-unitytest.md) attribute. For Edit Mode tests, your test code runs in the [EditorApplication.update](https://docs.unity3d.com/ScriptReference/EditorApplication-update.html) callback loop.
> **Note**: You can also control entering and exiting Play Mode from your Edit Mode test. This allow your test to make changes before entering Play Mode.
Edit Mode tests should meet one of the following conditions:
* They should have an [assembly definition](./workflow-create-test-assembly.md) with reference to *nunit.framework.dll* and has only the Editor as a target platform:
```assembly
"includePlatforms": [
"Editor"
],
```
* Legacy condition: put tests in the project’s [Editor](https://docs.unity3d.com/Manual/SpecialFolders.html) folder.
## Play Mode tests
You can run **Play Mode** tests as a [standalone in a Player](./workflow-run-playmode-test-standalone.md) or inside the Editor. Play Mode tests allow you to exercise your game code, as the tests run as [coroutines](https://docs.unity3d.com/ScriptReference/Coroutine.html) if marked with the `UnityTest` attribute.
Play Mode tests should correspond to the following conditions:
* Have an [assembly definition](./workflow-create-test-assembly.md) with reference to *nunit.framework.dll*.
* Have the test scripts located in a folder with the .asmdef file.
* The test assembly should reference an assembly within the code that you need to test.
```assembly
"references": [
"NewAssembly"
],
"optionalUnityReferences": [
"TestAssemblies"
],
"includePlatforms": [],
```
## Recommendations
### Attributes
Use the [NUnit](http://www.nunit.org/)`Test` attribute instead of the `UnityTest` attribute, unless you need to [yield special instructions](./reference-custom-yield-instructions.md), in Edit Mode, or if you need to skip a frame or wait for a certain amount of time in Play Mode.
### References
It is possible for your Test Assemblies to reference the test tools in `UnityEngine.TestRunner` and `UnityEditor.TestRunner`. The latter is only available in Edit Mode. You can specify these references in the `Assembly Definition References` on the Assembly Definition.
It is possible to extend the Unity Test Framework (UTF) in many ways, for custom workflows for your projects and for other packages to build on top of UTF.
These extensions are a supplement to the ones already offered by [NUnit](https://github.com/nunit/docs/wiki/Framework-Extensibility).
Some workflows for extending UTF include:
*[How to split the build and run process for standalone Play Mode tests](./reference-attribute-testplayerbuildmodifier.md#split-build-and-run-for-player-mode-tests)
*[How to run tests programmatically](./extension-run-tests.md)
*[How to get test results](./extension-get-test-results.md)
# Extending Unity Test Framework
It is possible to extend the Unity Test Framework (UTF) in many ways, for custom workflows for your projects and for other packages to build on top of UTF.
These extensions are a supplement to the ones already offered by [NUnit](https://github.com/nunit/docs/wiki/Framework-Extensibility).
Some workflows for extending UTF include:
*[How to split the build and run process for standalone Play Mode tests](./reference-attribute-testplayerbuildmodifier.md#split-build-and-run-for-player-mode-tests)
*[How to run tests programmatically](./extension-run-tests.md)
*[How to get test results](./extension-get-test-results.md)
*[How to retrieve the list of tests](./extension-retrieve-test-list.md)
You can receive callbacks when the active test run, or individual tests, starts and finishes. You can register callbacks by invoking `RegisterCallbacks` on the [TestRunnerApi](./reference-test-runner-api.md) with an instance of a class that implements [ICallbacks](./reference-icallbacks.md). There are four `ICallbacks` methods for the start and finish of both the whole run and each level of the test tree.
## Example
An example of how listeners can be set up:
> **Note**: Listeners receive callbacks from all test runs, regardless of the registered `TestRunnerApi` for that instance.
``` C#
public void SetupListeners()
{
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RegisterCallbacks(new MyCallbacks());
}
private class MyCallbacks : ICallbacks
{
public void RunStarted(ITestAdaptor testsToRun)
{
}
public void RunFinished(ITestResultAdaptor result)
{
}
public void TestStarted(ITestAdaptor test)
{
}
public void TestFinished(ITestResultAdaptor result)
{
if (!result.HasChildren && result.ResultState != "Success")
> **Note**: The registered callbacks are not persisted on domain reloads. So it is necessary to re-register the callback after a domain reloads, usually with [InitializeOnLoad](https://docs.unity3d.com/Manual/RunningEditorCodeOnLaunch.html).
# How to get test results
You can receive callbacks when the active test run, or individual tests, starts and finishes. You can register callbacks by invoking `RegisterCallbacks` on the [TestRunnerApi](./reference-test-runner-api.md) with an instance of a class that implements [ICallbacks](./reference-icallbacks.md). There are four `ICallbacks` methods for the start and finish of both the whole run and each level of the test tree.
## Example
An example of how listeners can be set up:
> **Note**: Listeners receive callbacks from all test runs, regardless of the registered `TestRunnerApi` for that instance.
``` C#
public void SetupListeners()
{
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RegisterCallbacks(new MyCallbacks());
}
private class MyCallbacks : ICallbacks
{
public void RunStarted(ITestAdaptor testsToRun)
{
}
public void RunFinished(ITestResultAdaptor result)
{
}
public void TestStarted(ITestAdaptor test)
{
}
public void TestFinished(ITestResultAdaptor result)
{
if (!result.HasChildren && result.ResultState != "Success")
> **Note**: The registered callbacks are not persisted on domain reloads. So it is necessary to re-register the callback after a domain reloads, usually with [InitializeOnLoad](https://docs.unity3d.com/Manual/RunningEditorCodeOnLaunch.html).
It is possible to provide a `priority` as an integer as the second argument when registering a callback. This influences the invocation order of different callbacks. The default value is zero. It is also possible to provide `RegisterCallbacks` with a class instance that implements [IErrorCallbacks](./reference-ierror-callbacks.md) that is an extended version of `ICallbacks`. `IErrorCallbacks` also has a callback method for `OnError` that invokes if the run fails to start, for example, due to compilation errors or if an [IPrebuildSetup](./reference-setup-and-cleanup.md) throws an exception.
It is possible to use the [TestRunnerApi](./reference-test-runner-api.md) to retrieve the test tree for a given test mode (**Edit Mode** or **Play Mode**). You can retrieve the test tree by invoking `RetrieveTestList` with the desired `TestMode` and a callback action, with an [ITestAdaptor](./reference-itest-adaptor.md) representing the test tree.
## Example
The following example retrieves the test tree for Edit Mode tests and prints the number of total test cases:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
It is possible to use the [TestRunnerApi](./reference-test-runner-api.md) to retrieve the test tree for a given test mode (**Edit Mode** or **Play Mode**). You can retrieve the test tree by invoking `RetrieveTestList` with the desired `TestMode` and a callback action, with an [ITestAdaptor](./reference-itest-adaptor.md) representing the test tree.
## Example
The following example retrieves the test tree for Edit Mode tests and prints the number of total test cases:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
Run tests by calling `Execute` on the [TestRunnerApi](./reference-test-runner-api.md), and provide some execution settings that consists of a [Filter](./reference-filter.md). The `Filter` specifies what tests to run.
### Example
The following is an example of how to run all **Play Mode** tests in a project:
``` C#
var testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();
It is possible to specify a more specific filter by filling out the fields on the `Filter` class in more detail.
Many of the fields allow for multiple values. The runner tries to match tests against at least one of the values provided and then runs any tests that match.
### Example
In this example, the API runs tests with full names that fit either of the two names provided:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
If using multiple different fields on the filter, then it matches against tests that fulfill all the different fields.
### Example
In this example, it runs any test that fits either of the two test names, and that also belongs to a test assembly that fits the given name.
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.Execute(new ExecutionSettings(new Filter()
{
assemblyNames = new [] {"MyTestAssembly"},
testNames = new [] {"MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest"}
}));
```
## Multiple constructor filters
The execution settings take one or more filters in its constructor. If there is no filter provided, then it runs all **Edit Mode** tests by default. If there are multiple filters provided, then a test runs if it matches any of the filters.
### Example
In this example, it runs any tests that are either in the assembly named `MyTestAssembly` or if the full name of the test matches either of the two provided test names:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
Run tests by calling `Execute` on the [TestRunnerApi](./reference-test-runner-api.md), and provide some execution settings that consists of a [Filter](./reference-filter.md). The `Filter` specifies what tests to run.
### Example
The following is an example of how to run all **Play Mode** tests in a project:
``` C#
var testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();
It is possible to specify a more specific filter by filling out the fields on the `Filter` class in more detail.
Many of the fields allow for multiple values. The runner tries to match tests against at least one of the values provided and then runs any tests that match.
### Example
In this example, the API runs tests with full names that fit either of the two names provided:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
If using multiple different fields on the filter, then it matches against tests that fulfill all the different fields.
### Example
In this example, it runs any test that fits either of the two test names, and that also belongs to a test assembly that fits the given name.
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.Execute(new ExecutionSettings(new Filter()
{
assemblyNames = new [] {"MyTestAssembly"},
testNames = new [] {"MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest"}
}));
```
## Multiple constructor filters
The execution settings take one or more filters in its constructor. If there is no filter provided, then it runs all **Edit Mode** tests by default. If there are multiple filters provided, then a test runs if it matches any of the filters.
### Example
In this example, it runs any tests that are either in the assembly named `MyTestAssembly` or if the full name of the test matches either of the two provided test names:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
> **Note**: Specifying different test modes or platforms in each `Filter` is not currently supported. The test mode and platform is from the first `Filter` only and defaults to Edit Mode, if not supplied.
The Unity Test Framework (UTF) enables Unity users to test their code in both **Edit Mode** and **Play Mode**, and also on target platforms such as [Standalone](https://docs.unity3d.com/Manual/Standalone.html), Android, iOS, etc.
This package provides a standard test framework for users of Unity and developers at Unity so that both benefit from the same features and can write tests the same way.
UTF uses a Unity integration of NUnit library, which is an open-source unit testing library for .Net languages. For more information about NUnit, see the [official NUnit website](http://www.nunit.org/) and the [NUnit documentation on GitHub](https://github.com/nunit/docs/wiki/NUnit-Documentation).
> **Note**: UTF is not a new concept or toolset; it is an adjusted and more descriptive naming for the toolset otherwise known as Unity Test Runner, which is now available as this package.
# Installing Unity Test Framework
To install this package, follow the instructions in the [Package Manager documentation](https://docs.unity3d.com/Packages/com.unity.package-manager-ui@latest/index.html).
> **Note**: Search for the Test Framework package. In Unity 2019.2 and higher, you may need to enable the package before use.
# Using Unity Test Framework
To learn how to use the Unity Test Framework package in your project, read the [manual](./manual.md).
# Technical details
## Requirements
This version of the Unity Test Framework is compatible with the following versions of the Unity Editor:
* 2019.2 and later.
## Known limitations
Unity Test Framework version 1.0.18 includes the following known limitations:
* The `UnityTest` attribute does not support WebGL and WSA platforms.
* The `UnityTest` attribute does not support [Parameterized tests](https://github.com/nunit/docs/wiki/Parameterized-Tests)(except for `ValueSource`).
* The `UnityTest` attribute does not support the `NUnit`[Repeat](https://github.com/nunit/docs/wiki/Repeat-Attribute) attribute.
* Nested test fixture cannot run from the Editor UI.
* When using the `NUnit`[Retry](https://github.com/nunit/docs/wiki/Retry-Attribute) attribute in PlayMode tests, it throws `InvalidCastException`.
## Package contents
The following table indicates the root folders in the package where you can find useful resources:
| August 23, 2019 | Applied feedback to the documentation |
| July 25, 2019 | Documentation updated to include features in version 1.1.0 |
| July 11, 2019 | Documentation updated. Matches package version 1.0.18 |
| May 27, 2019 | Documentation created. Matches package version 1.0.14 |
# About Unity Test Framework
The Unity Test Framework (UTF) enables Unity users to test their code in both **Edit Mode** and **Play Mode**, and also on target platforms such as [Standalone](https://docs.unity3d.com/Manual/Standalone.html), Android, iOS, etc.
This package provides a standard test framework for users of Unity and developers at Unity so that both benefit from the same features and can write tests the same way.
UTF uses a Unity integration of NUnit library, which is an open-source unit testing library for .Net languages. For more information about NUnit, see the [official NUnit website](http://www.nunit.org/) and the [NUnit documentation on GitHub](https://github.com/nunit/docs/wiki/NUnit-Documentation).
> **Note**: UTF is not a new concept or toolset; it is an adjusted and more descriptive naming for the toolset otherwise known as Unity Test Runner, which is now available as this package.
# Installing Unity Test Framework
To install this package, follow the instructions in the [Package Manager documentation](https://docs.unity3d.com/Packages/com.unity.package-manager-ui@latest/index.html).
> **Note**: Search for the Test Framework package. In Unity 2019.2 and higher, you may need to enable the package before use.
# Using Unity Test Framework
To learn how to use the Unity Test Framework package in your project, read the [manual](./manual.md).
# Technical details
## Requirements
This version of the Unity Test Framework is compatible with the following versions of the Unity Editor:
* 2019.2 and later.
## Known limitations
Unity Test Framework version 1.0.18 includes the following known limitations:
* The `UnityTest` attribute does not support WebGL and WSA platforms.
* The `UnityTest` attribute does not support [Parameterized tests](https://github.com/nunit/docs/wiki/Parameterized-Tests)(except for `ValueSource`).
* The `UnityTest` attribute does not support the `NUnit`[Repeat](https://github.com/nunit/docs/wiki/Repeat-Attribute) attribute.
* Nested test fixture cannot run from the Editor UI.
* When using the `NUnit`[Retry](https://github.com/nunit/docs/wiki/Retry-Attribute) attribute in PlayMode tests, it throws `InvalidCastException`.
## Package contents
The following table indicates the root folders in the package where you can find useful resources:
When writing tests, it is possible to avoid duplication of code by using the [SetUp and TearDown](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) methods built into [NUnit](http://www.nunit.org/). The Unity Test Framework has extended these methods with extra functionality, which can yield commands and skip frames, in the same way as [UnityTest](./reference-attribute-unitytest.md).
## Action execution order
The actions related to a test run in the following order:
* Any [SetUp](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) attributes
*[Action attributes](https://nunit.org/docs/2.6/actionAttributes.html) have their `BeforeTest` method invoked
* Attributes implementing of [IWrapTestMethod](https://github.com/nunit/docs/wiki/ICommandWrapper-Interface)
***The test itself runs**
*[Action attributes](https://nunit.org/docs/2.6/actionAttributes.html) have their `AfterTest` method invoked
* Any method with the [TearDown](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) attribute
* Tests with [UnityTearDown](#unitysetup-and-unityteardown) methods in their test class
* Any [OuterUnityTestAction](#outerunitytestaction) has its `AfterTest` invoked
The list of actions is the same for both `Test` and `UnityTest`.
## UnitySetUp and UnityTearDown
The `UnitySetUp` and `UnityTearDown` attributes are identical to the standard `SetUp` and `TearDown` attributes, with the exception that they allow for [yielding instructions](reference-custom-yield-instructions.md). The `UnitySetUp` and `UnityTearDown` attributes expect a return type of [IEnumerator](https://docs.microsoft.com/en-us/dotnet/api/system.collections.ienumerator?view=netframework-4.8).
### Example
```c#
publicclassSetUpTearDownExample
{
[UnitySetUp]
publicIEnumeratorSetUp()
{
yieldreturnnewEnterPlayMode();
}
[Test]
publicvoidMyTest()
{
Debug.Log("This runs inside playmode");
}
[UnitySetUp]
publicIEnumeratorTearDown()
{
yieldreturnnewExitPlayMode();
}
}
```
## OuterUnityTestAction
`OuterUnityTestAction` is a wrapper outside of the tests, which allows for any tests with this attribute to run code before and after the tests. This method allows for yielding commands in the same way as `UnityTest`. The attribute must inherit the `NUnit` attribute and implement `IOuterUnityTestAction`.
When writing tests, it is possible to avoid duplication of code by using the [SetUp and TearDown](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) methods built into [NUnit](http://www.nunit.org/). The Unity Test Framework has extended these methods with extra functionality, which can yield commands and skip frames, in the same way as [UnityTest](./reference-attribute-unitytest.md).
## Action execution order
The actions related to a test run in the following order:
* Any [SetUp](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) attributes
*[Action attributes](https://nunit.org/docs/2.6/actionAttributes.html) have their `BeforeTest` method invoked
* Attributes implementing of [IWrapTestMethod](https://github.com/nunit/docs/wiki/ICommandWrapper-Interface)
***The test itself runs**
*[Action attributes](https://nunit.org/docs/2.6/actionAttributes.html) have their `AfterTest` method invoked
* Any method with the [TearDown](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) attribute
* Tests with [UnityTearDown](#unitysetup-and-unityteardown) methods in their test class
* Any [OuterUnityTestAction](#outerunitytestaction) has its `AfterTest` invoked
The list of actions is the same for both `Test` and `UnityTest`.
## UnitySetUp and UnityTearDown
The `UnitySetUp` and `UnityTearDown` attributes are identical to the standard `SetUp` and `TearDown` attributes, with the exception that they allow for [yielding instructions](reference-custom-yield-instructions.md). The `UnitySetUp` and `UnityTearDown` attributes expect a return type of [IEnumerator](https://docs.microsoft.com/en-us/dotnet/api/system.collections.ienumerator?view=netframework-4.8).
### Example
```c#
publicclassSetUpTearDownExample
{
[UnitySetUp]
publicIEnumeratorSetUp()
{
yieldreturnnewEnterPlayMode();
}
[Test]
publicvoidMyTest()
{
Debug.Log("This runs inside playmode");
}
[UnitySetUp]
publicIEnumeratorTearDown()
{
yieldreturnnewExitPlayMode();
}
}
```
## OuterUnityTestAction
`OuterUnityTestAction` is a wrapper outside of the tests, which allows for any tests with this attribute to run code before and after the tests. This method allows for yielding commands in the same way as `UnityTest`. The attribute must inherit the `NUnit` attribute and implement `IOuterUnityTestAction`.
In **Edit Mode** tests it is possible to yield instructions that can result in a domain reload, such as entering or exiting **Play Mode** (see [Custom yield instructions](./reference-custom-yield-instructions.md)). When a domain reload happens, all non-Unity actions (such as `OneTimeSetup` and `Setup`) are rerun before the code, which initiated the domain reload, continues. Unity actions (such as `UnitySetup`) are not rerun. If the Unity action is the code that initiated the domain reload, then the rest of the code in the `UnitySetup` method runs after the domain reload.
This attribute is an alternative to the standard `Ignore` attribute in [NUnit](http://www.nunit.org/). It allows for ignoring tests only under a specified condition. The condition evaluates during `OnLoad`, referenced by ID.
## Example
The following example shows a method to use the `ConditionalIgnore` attribute to ignore a test if the Unity Editor is running macOS:
[Test, ConditionalIgnore("IgnoreInMacEditor", "Ignored on Mac editor.")]
public void TestNeverRunningInMacEditor()
{
Assert.Pass();
}
}
```
# ConditionalIgnore attribute
This attribute is an alternative to the standard `Ignore` attribute in [NUnit](http://www.nunit.org/). It allows for ignoring tests only under a specified condition. The condition evaluates during `OnLoad`, referenced by ID.
## Example
The following example shows a method to use the `ConditionalIgnore` attribute to ignore a test if the Unity Editor is running macOS:
The presence of this attribute causes the **Test Runner** to expect every single log. By default, the Test Runner only fails on error logs, but `TestMustExpectAllLogs` fails on warnings and info level messages as well. It is the same as calling the method [LogAssert.NoUnexpectedReceived](./reference-custom-assertion.md#static-methods) at the bottom of every affected test.
## Assembly-wide usage
You can apply this attribute to test assemblies (that affects every test in the assembly), fixtures (affects every test in the fixture), or on individual test methods. It is also inherited from base fixtures.
The `MustExpect` property (`true` by default) lets you enable or disable the higher level value.
# TestMustExpectAllLogs attribute
The presence of this attribute causes the **Test Runner** to expect every single log. By default, the Test Runner only fails on error logs, but `TestMustExpectAllLogs` fails on warnings and info level messages as well. It is the same as calling the method [LogAssert.NoUnexpectedReceived](./reference-custom-assertion.md#static-methods) at the bottom of every affected test.
## Assembly-wide usage
You can apply this attribute to test assemblies (that affects every test in the assembly), fixtures (affects every test in the fixture), or on individual test methods. It is also inherited from base fixtures.
The `MustExpect` property (`true` by default) lets you enable or disable the higher level value.
For example when migrating an assembly to this more strict checking method, you might attach `[assembly:TestMustExpectAllLogs]` to the assembly itself, but then whitelist failing fixtures and test methods with `[TestMustExpectAllLogs(MustExpect=false)]` until you have migrated them. This also means new tests in that assembly would have the more strict checking.
You can use the `TestPlayerBuildModifier` attribute to accomplish a couple of different scenarios:
## Modify the Player build options for Play Mode tests
It is possible to change the [BuildPlayerOptions](https://docs.unity3d.com/ScriptReference/BuildPlayerOptions.html) for the test **Player**, to achieve custom behavior when running **Play Mode** tests. Modifying the build options allows for changing the target location of the build as well as changing [BuildOptions](https://docs.unity3d.com/ScriptReference/BuildOptions.html).
To modify the `BuildPlayerOptions`, do the following:
* Implement the `ITestPlayerBuildModifier`
* Reference the implementation type in a `TestPlayerBuildModifier` attribute on an assembly level.
playerOptions.options|=BuildOptions.SymlinkLibraries;// Enable symlink libraries when running on iOS
}
playerOptions.options|=BuildOptions.AllowDebugging;// Enable allow Debugging flag on the test Player.
returnplayerOptions;
}
}
```
> **Note:** When building the Player, it includes all `TestPlayerBuildModifier` attributes across all loaded assemblies, independent of the currently used test filter. As the implementation references the `UnityEditor` namespace, the code is typically implemented in an Editor only assembly, as the `UnityEditor` namespace is not available otherwise.
## Split build and run
It is possible to use the Unity Editor for building the Player with tests, without [running the tests](./workflow-run-playmode-test-standalone.md). This allows for running the Player on e.g. another machine. In this case, it is necessary to modify the Player to build and implement a custom handling of the test result.
By using `TestPlayerBuildModifier`, you can alter the `BuildOptions` to not start the Player after the build as well as build the Player at a specific location. Combined with [PostBuildCleanup](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup), you can automatically exit the Editor on completion of the build.
If the Editor is still running after the Play Mode tests have run, the Player tries to report the results back, using [PlayerConnection](https://docs.unity3d.com/ScriptReference/Networking.PlayerConnection.PlayerConnection.html), which has a reference to the IP address of the Editor machine, when built.
To implement a custom way of reporting the results of the test run, let one of the assemblies in the Player include a [TestRunCallback](./reference-attribute-testruncallback.md). At `RunFinished`, it is possible to get the full test report as XML from the [NUnit](http://www.nunit.org/) test result by calling `result.ToXml(true)`. You can save the result and then save it on the device or send it to another machine as needed.
# TestPlayerBuildModifier attribute
You can use the `TestPlayerBuildModifier` attribute to accomplish a couple of different scenarios:
## Modify the Player build options for Play Mode tests
It is possible to change the [BuildPlayerOptions](https://docs.unity3d.com/ScriptReference/BuildPlayerOptions.html) for the test **Player**, to achieve custom behavior when running **Play Mode** tests. Modifying the build options allows for changing the target location of the build as well as changing [BuildOptions](https://docs.unity3d.com/ScriptReference/BuildOptions.html).
To modify the `BuildPlayerOptions`, do the following:
* Implement the `ITestPlayerBuildModifier`
* Reference the implementation type in a `TestPlayerBuildModifier` attribute on an assembly level.
playerOptions.options|=BuildOptions.SymlinkLibraries;// Enable symlink libraries when running on iOS
}
playerOptions.options|=BuildOptions.AllowDebugging;// Enable allow Debugging flag on the test Player.
returnplayerOptions;
}
}
```
> **Note:** When building the Player, it includes all `TestPlayerBuildModifier` attributes across all loaded assemblies, independent of the currently used test filter. As the implementation references the `UnityEditor` namespace, the code is typically implemented in an Editor only assembly, as the `UnityEditor` namespace is not available otherwise.
## Split build and run
It is possible to use the Unity Editor for building the Player with tests, without [running the tests](./workflow-run-playmode-test-standalone.md). This allows for running the Player on e.g. another machine. In this case, it is necessary to modify the Player to build and implement a custom handling of the test result.
By using `TestPlayerBuildModifier`, you can alter the `BuildOptions` to not start the Player after the build as well as build the Player at a specific location. Combined with [PostBuildCleanup](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup), you can automatically exit the Editor on completion of the build.
If the Editor is still running after the Play Mode tests have run, the Player tries to report the results back, using [PlayerConnection](https://docs.unity3d.com/ScriptReference/Networking.PlayerConnection.PlayerConnection.html), which has a reference to the IP address of the Editor machine, when built.
To implement a custom way of reporting the results of the test run, let one of the assemblies in the Player include a [TestRunCallback](./reference-attribute-testruncallback.md). At `RunFinished`, it is possible to get the full test report as XML from the [NUnit](http://www.nunit.org/) test result by calling `result.ToXml(true)`. You can save the result and then save it on the device or send it to another machine as needed.
It is possible for the test framework to invoke callbacks as the current test run progresses. To do this, there is a `TestRunCallback` attribute which takes the type of `ITestRunCallback` implementation. You can invoke the callbacks with [NUnit](http://www.nunit.org/)`ITest` and `ITestResult` classes.
At the `RunStarted` and `RunFinished` methods, the test and test results are for the whole test tree. These methods invoke at each node in the test tree; first with the whole test assembly, then with the test class, and last with the test method.
From these callbacks, it is possible to read the partial or the full results, and it is furthermore possible to save the XML version of the result for further processing or continuous integration.
Debug.Log($"Result of {result.Name}: {result.ResultState.Status}");
}
}
}
```
# TestRunCallback attribute
It is possible for the test framework to invoke callbacks as the current test run progresses. To do this, there is a `TestRunCallback` attribute which takes the type of `ITestRunCallback` implementation. You can invoke the callbacks with [NUnit](http://www.nunit.org/)`ITest` and `ITestResult` classes.
At the `RunStarted` and `RunFinished` methods, the test and test results are for the whole test tree. These methods invoke at each node in the test tree; first with the whole test assembly, then with the test class, and last with the test method.
From these callbacks, it is possible to read the partial or the full results, and it is furthermore possible to save the XML version of the result for further processing or continuous integration.
Debug.Log($"Result of {result.Name}: {result.ResultState.Status}");
}
}
}
```
> **Note:** The `TestRunCallback` does not need any references to the `UnityEditor` namespace and is thus able to run in standalone Players, on the **Player** side.
Use this attribute to define a specific set of platforms you want or do not want your test(s) to run on.
You can use this attribute on the test method, test class, or test assembly level. Use the supported [RuntimePlatform](https://docs.unity3d.com/ScriptReference/RuntimePlatform.html) enumeration values to specify the platforms. You can also specify which platforms to test by passing one or more `RuntimePlatform` values along with or without the include or exclude properties as parameters to the [Platform](https://github.com/nunit/docs/wiki/Platform-Attribute) attribute constructor.
The test(s) skips if the current target platform is:
- Not explicitly specified in the included platforms list
| `RuntimePlatform[] exclude` | List the platforms you do not want to have your tests run on. |
| `RuntimePlatform[] include` | A subset of platforms you need to have your tests run on. |
# UnityPlatform attribute
Use this attribute to define a specific set of platforms you want or do not want your test(s) to run on.
You can use this attribute on the test method, test class, or test assembly level. Use the supported [RuntimePlatform](https://docs.unity3d.com/ScriptReference/RuntimePlatform.html) enumeration values to specify the platforms. You can also specify which platforms to test by passing one or more `RuntimePlatform` values along with or without the include or exclude properties as parameters to the [Platform](https://github.com/nunit/docs/wiki/Platform-Attribute) attribute constructor.
The test(s) skips if the current target platform is:
- Not explicitly specified in the included platforms list
`UnityTest` attribute is the main addition to the standard [NUnit](http://www.nunit.org/) library for the Unity Test Framework. This type of unit test allows you to skip a frame from within a test (so background tasks can finish) or give certain commands to the Unity **Editor**, such as performing a domain reload or entering **Play Mode** from an **Edit Mode** test.
In Play Mode, the `UnityTest` attribute runs as a [coroutine](https://docs.unity3d.com/Manual/Coroutines.html). Whereas Edit Mode tests run in the [EditorApplication.update](https://docs.unity3d.com/ScriptReference/EditorApplication-update.html) callback loop.
The `UnityTest` attribute is, in fact, an alternative to the `NUnit`[Test attribute](https://github.com/nunit/docs/wiki/Test-Attribute), which allows yielding instructions back to the framework. Once the instruction is complete, the test run continues. If you `yield return null`, you skip a frame. That might be necessary to ensure that some changes do happen on the next iteration of either the `EditorApplication.update` loop or the [game loop](https://docs.unity3d.com/Manual/ExecutionOrder.html).
## Edit Mode example
The most simple example of an Edit Mode test could be the one that yields `null` to skip the current frame and then continues to run:
```C#
[UnityTest]
public IEnumerator EditorUtility_WhenExecuted_ReturnsSuccess()
{
var utility = RunEditorUtilityInTheBackgroud();
while (utility.isRunning)
{
yield return null;
}
Assert.IsTrue(utility.isSuccess);
}
```
## Play Mode example
In Play Mode, a test runs as a coroutine attached to a [MonoBehaviour](https://docs.unity3d.com/ScriptReference/MonoBehaviour.html). So all the yield instructions available in coroutines, are also available in your test.
From a Play Mode test you can use one of Unity’s [Yield Instructions](https://docs.unity3d.com/ScriptReference/YieldInstruction.html):
-[WaitForFixedUpdate](https://docs.unity3d.com/ScriptReference/WaitForFixedUpdate.html): to ensure changes expected within the next cycle of physics calculations.
-[WaitForSeconds](https://docs.unity3d.com/ScriptReference/WaitForSeconds.html): if you want to pause your test coroutine for a fixed amount of time. Be careful about creating long-running tests.
The simplest example is to yield to `WaitForFixedUpdate`:
`UnityTest` attribute is the main addition to the standard [NUnit](http://www.nunit.org/) library for the Unity Test Framework. This type of unit test allows you to skip a frame from within a test (so background tasks can finish) or give certain commands to the Unity **Editor**, such as performing a domain reload or entering **Play Mode** from an **Edit Mode** test.
In Play Mode, the `UnityTest` attribute runs as a [coroutine](https://docs.unity3d.com/Manual/Coroutines.html). Whereas Edit Mode tests run in the [EditorApplication.update](https://docs.unity3d.com/ScriptReference/EditorApplication-update.html) callback loop.
The `UnityTest` attribute is, in fact, an alternative to the `NUnit`[Test attribute](https://github.com/nunit/docs/wiki/Test-Attribute), which allows yielding instructions back to the framework. Once the instruction is complete, the test run continues. If you `yield return null`, you skip a frame. That might be necessary to ensure that some changes do happen on the next iteration of either the `EditorApplication.update` loop or the [game loop](https://docs.unity3d.com/Manual/ExecutionOrder.html).
## Edit Mode example
The most simple example of an Edit Mode test could be the one that yields `null` to skip the current frame and then continues to run:
```C#
[UnityTest]
public IEnumerator EditorUtility_WhenExecuted_ReturnsSuccess()
{
var utility = RunEditorUtilityInTheBackgroud();
while (utility.isRunning)
{
yield return null;
}
Assert.IsTrue(utility.isSuccess);
}
```
## Play Mode example
In Play Mode, a test runs as a coroutine attached to a [MonoBehaviour](https://docs.unity3d.com/ScriptReference/MonoBehaviour.html). So all the yield instructions available in coroutines, are also available in your test.
From a Play Mode test you can use one of Unity’s [Yield Instructions](https://docs.unity3d.com/ScriptReference/YieldInstruction.html):
-[WaitForFixedUpdate](https://docs.unity3d.com/ScriptReference/WaitForFixedUpdate.html): to ensure changes expected within the next cycle of physics calculations.
-[WaitForSeconds](https://docs.unity3d.com/ScriptReference/WaitForSeconds.html): if you want to pause your test coroutine for a fixed amount of time. Be careful about creating long-running tests.
The simplest example is to yield to `WaitForFixedUpdate`:
For more information, see [Command line arguments](https://docs.unity3d.com/Manual/CommandLineArguments.html).
## Commands
### batchmode
Runs Unity in batch mode and ensures no pop-up windows appear to eliminate the need for any human intervention.
### forgetProjectPath
Don't save your current **Project** into the Unity launcher/hub history.
### runTests
Runs tests in the Project.
### testCategory
A semicolon-separated list of test categories to include in the run. If using both `testFilter` and `testCategory`, then tests only run that matches both.
### testFilter
A semicolon-separated list of test names to run, or a regular expression pattern to match tests by their full name.
### testPlatform
The platform you want to run tests on. Available platforms are **EditMode** and **PlayMode**.
> **Note**: If unspecified, tests run in Edit Mode by default.
Platform/Type convention is from the [BuildTarget](https://docs.unity3d.com/ScriptReference/BuildTarget.html) enum. Supported platforms are:
* StandaloneWindows
* StandaloneWindows64
* StandaloneLinux64
* StandaloneOSX
* iOS
* Android
* PS4
* XboxOne
### assemblyNames
A semicolon-separated list of test assemblies to include in the run.
### testResults
The path where Unity should save the result file. By default, Unity saves it in the Project’s root folder.
### playerHeartbeatTimeout
The time, in seconds, the editor should wait for heartbeats after starting a test run on a player. This defaults to 10 minutes.
### runSynchronously
If included, the test run will run tests synchronously, guaranteeing that all tests runs in one editor update call. Note that this is only supported for EditMode tests, and that tests which take multiple frames (i.e. `[UnityTest]` tests, or tests with `[UnitySetUp]` or `[UnityTearDown]` scaffolding) will be filtered out.
### testSettingsFile
Path to a *TestSettings.json* file that allows you to set up extra options for your test run. An example of the *TestSettings.json* file could look like this:
```json
{
"scriptingBackend":2,
"Architecture":null,
"apiProfile":0
}
```
#### apiProfile
The .Net compatibility level. Set to one of the following values:
- 1 - .Net 2.0
- 2 - .Net 2.0 Subset
- 3 - .Net 4.6
- 5 - .Net micro profile (used by Mono scripting backend if **Stripping Level** is set to **Use micro mscorlib**)
- 6 - .Net Standard 2.0
#### appleEnableAutomaticSigning
Sets option for automatic signing of Apple devices.
#### appleDeveloperTeamID
Sets the team ID for the apple developer account.
#### architecture
Target architecture for Android. Set to one of the following values:
* None = 0
* ARMv7 = 1
* ARM64 = 2
* X86 = 4
* All = 4294967295
#### iOSManualProvisioningProfileType
Set to one of the following values:
* 0 - Automatic
* 1 - Development
* 2 - Distribution iOSManualProvisioningProfileID
#### scriptingBackend
Set to one of the following values:
- Mono2x = 0
- IL2CPP = 1
# Running tests from the command line
It’s pretty simple to run a test project from the command line. Here is an example in Windows:
For more information, see [Command line arguments](https://docs.unity3d.com/Manual/CommandLineArguments.html).
## Commands
### batchmode
Runs Unity in batch mode and ensures no pop-up windows appear to eliminate the need for any human intervention.
### forgetProjectPath
Don't save your current **Project** into the Unity launcher/hub history.
### runTests
Runs tests in the Project.
### testCategory
A semicolon-separated list of test categories to include in the run. If using both `testFilter` and `testCategory`, then tests only run that matches both.
### testFilter
A semicolon-separated list of test names to run, or a regular expression pattern to match tests by their full name.
### testPlatform
The platform you want to run tests on. Available platforms are **EditMode** and **PlayMode**.
> **Note**: If unspecified, tests run in Edit Mode by default.
Platform/Type convention is from the [BuildTarget](https://docs.unity3d.com/ScriptReference/BuildTarget.html) enum. Supported platforms are:
* StandaloneWindows
* StandaloneWindows64
* StandaloneLinux64
* StandaloneOSX
* iOS
* Android
* PS4
* XboxOne
### assemblyNames
A semicolon-separated list of test assemblies to include in the run.
### testResults
The path where Unity should save the result file. By default, Unity saves it in the Project’s root folder.
### playerHeartbeatTimeout
The time, in seconds, the editor should wait for heartbeats after starting a test run on a player. This defaults to 10 minutes.
### runSynchronously
If included, the test run will run tests synchronously, guaranteeing that all tests runs in one editor update call. Note that this is only supported for EditMode tests, and that tests which take multiple frames (i.e. `[UnityTest]` tests, or tests with `[UnitySetUp]` or `[UnityTearDown]` scaffolding) will be filtered out.
### testSettingsFile
Path to a *TestSettings.json* file that allows you to set up extra options for your test run. An example of the *TestSettings.json* file could look like this:
```json
{
"scriptingBackend":2,
"Architecture":null,
"apiProfile":0
}
```
#### apiProfile
The .Net compatibility level. Set to one of the following values:
- 1 - .Net 2.0
- 2 - .Net 2.0 Subset
- 3 - .Net 4.6
- 5 - .Net micro profile (used by Mono scripting backend if **Stripping Level** is set to **Use micro mscorlib**)
- 6 - .Net Standard 2.0
#### appleEnableAutomaticSigning
Sets option for automatic signing of Apple devices.
#### appleDeveloperTeamID
Sets the team ID for the apple developer account.
#### architecture
Target architecture for Android. Set to one of the following values:
Use this class to compare two `Color` objects. `ColorEqualityComparer.Instance` has default calculation error value set to 0.01f. To set a test specific error value instantiate a comparer instance using the [one argument constructor](#constructors).
| `bool Equals(Color expected, Color actual);` | Compares the actual and expected `Color` objects for equality using `Utils.AreFloatsEqualAbsoluteError` to compare the `RGB` and `Alpha` attributes of `Color`. Returns `true` if expected and actual objects are equal otherwise, it returns `false`. |
Use this class to compare two `Color` objects. `ColorEqualityComparer.Instance` has default calculation error value set to 0.01f. To set a test specific error value instantiate a comparer instance using the [one argument constructor](#constructors).
| `bool Equals(Color expected, Color actual);` | Compares the actual and expected `Color` objects for equality using `Utils.AreFloatsEqualAbsoluteError` to compare the `RGB` and `Alpha` attributes of `Color`. Returns `true` if expected and actual objects are equal otherwise, it returns `false`. |
If you need to compare Vectors using the overloaded operator == (see [Vector2.operator ==](https://docs.unity3d.com/ScriptReference/Vector2-operator_eq.html), [Vector3.operator ==](https://docs.unity3d.com/ScriptReference/Vector3-operator_eq.html), and [Vector4.operator ==](https://docs.unity3d.com/ScriptReference/Vector4-operator_eq.html)) you should use the respective comparer implementations:
- Vector2ComparerWithEqualsOperator
- Vector3ComparerWithEqualsOperator
- Vector4ComparerWithEqualsOperator
The interface is the same as for other [equality comparers](./reference-custom-equality-comparers.md) except the public [constructor](./reference-custom-equality-comparers.md#constructors)`error` parameter is inapplicable in this case.
If you need to compare Vectors using the overloaded operator == (see [Vector2.operator ==](https://docs.unity3d.com/ScriptReference/Vector2-operator_eq.html), [Vector3.operator ==](https://docs.unity3d.com/ScriptReference/Vector3-operator_eq.html), and [Vector4.operator ==](https://docs.unity3d.com/ScriptReference/Vector4-operator_eq.html)) you should use the respective comparer implementations:
- Vector2ComparerWithEqualsOperator
- Vector3ComparerWithEqualsOperator
- Vector4ComparerWithEqualsOperator
The interface is the same as for other [equality comparers](./reference-custom-equality-comparers.md) except the public [constructor](./reference-custom-equality-comparers.md#constructors)`error` parameter is inapplicable in this case.
Use this class to compare two float values for equality with [NUnit](http://www.nunit.org/) constraints. Use `FloatEqualityComparer.Instance` comparer to have the default error value set to 0.0001f. For any other error, use the [one argument constructor](#constructors) to create a comparer.
Use this class to compare two float values for equality with [NUnit](http://www.nunit.org/) constraints. Use `FloatEqualityComparer.Instance` comparer to have the default error value set to 0.0001f. For any other error, use the [one argument constructor](#constructors) to create a comparer.
Use this utility to compare two [Quaternion](https://docs.unity3d.com/ScriptReference/Quaternion.html) objects for equality with [NUnit](http://www.nunit.org/) assertion constraints. Use the static instance `QuaternionEqualityComparer.Instance` to have the default calculation error value set to 0.00001f. For any other custom error value, use the [one argument constructor](#constructors).
| `bool Equals(Quaternion expected, Quaternion actual)` | Compares the `actual` and `expected``Quaternion` objects for equality using the [Quaternion.Dot](https://docs.unity3d.com/ScriptReference/Quaternion.Dot.html) method. |
Use this utility to compare two [Quaternion](https://docs.unity3d.com/ScriptReference/Quaternion.html) objects for equality with [NUnit](http://www.nunit.org/) assertion constraints. Use the static instance `QuaternionEqualityComparer.Instance` to have the default calculation error value set to 0.00001f. For any other custom error value, use the [one argument constructor](#constructors).
| `bool Equals(Quaternion expected, Quaternion actual)` | Compares the `actual` and `expected``Quaternion` objects for equality using the [Quaternion.Dot](https://docs.unity3d.com/ScriptReference/Quaternion.Dot.html) method. |
