Adaug proiectul Unity

This commit is contained in:
2026-11-01 23:39:55 +02:00
parent cb14d9d37d
commit 863eac8f2c
23463 changed files with 5412601 additions and 0 deletions
@@ -0,0 +1,25 @@
artifacts/**
build/**
.build_script/**
node_modules/**
Documentation/ApiDocs/**
Documentation~/ApiDocs/**
.DS_Store
.npmrc
.npmignore
.gitignore
CONTRIBUTING.md
CONTRIBUTING.md.meta
QAReport.md
QAReport.md.meta
build.sh
build.sh.meta
build.bat
build.bat.meta
*.zip*
upm-ci~/**
.Editor/**
.yamato/**
CODEOWNERS
TestRunnerOptions.json
.editorconfig
@@ -0,0 +1,833 @@
# Changelog
## [6.6.0] - 2026-05-15
### Fixed
- Unsampled sample groups are now reported with zeroed statistics, and the active test is always cleared so a failing test can no longer fail the rest of the run. [UUM-150776]
- Performance test results are now saved to the results file as each test finishes instead of only when the whole run completes, so results collected before a crash, a test timeout or another premature shutdown of a test run are no longer lost. The results file is also written through a temporary file so an interrupted write cannot leave it truncated.
- Repeat test-player builds no longer assign a fresh GUID to the PerformanceTestRunInfo.json/PerformanceTestRunSettings.json assets on every build. Previously, deleting these assets' `.meta` files as part of post-build cleanup caused Unity to assign a new GUID on the next build, which the data pipeline treated as a change and triggered a full asset rebuild even when nothing else changed. The `.meta` files are now cached in `Library/` between builds and restored before the assets are rewritten, scoped to builds with test assemblies. (UUM-132359)
- Post-build cleanup no longer deletes the Assets/Resources folder if it contains user content. Previously, an aborted test build could leave a stale cleanup flag behind, causing a later build to recursively delete the entire folder.
- Cleanup of performance test artifacts is now best-effort: files or folders that cannot be deleted (e.g. transiently locked by another process) log a warning instead of throwing, which would previously fail the build when cleanup runs during build preprocessing.
- The internal cleanup flag tracking whether the package created the Assets/Resources folder moved from EditorPrefs to SessionState, making it per-project and per-session. Previously the flag was shared across all Unity projects on the machine and could go stale across editor restarts.
### Changed
- com.unity.test-framework.performance is now a core package included in the Unity Editor. Changes to the core package version will be part of the Editor release notes.
- When `perfTestResults` is not specified but the Unity Test Framework `testResults` argument is, the performance results file is now also saved next to the NUnit results file. A command line run collects that directory, so results of a run that did not finish can still be picked up from it.
- PerformanceTestRunInfo.json and PerformanceTestRunSettings.json are no longer generated during player builds that do not include test assemblies, as they are only read by performance tests running inside the player. Stale files left behind by an aborted test build are removed before such builds instead of being included in them.
### Added
- A performance regression Threshold property for the Performance attribute. It is only reported when the value is greater than 0.0 and is to be used by the performance reporting endpoint to compare against historical data.
## [3.4.2] - 2026-04-27
### Added
- Made the package compatible with Fast Enter Playmode
## [3.4.1] - 2026-04-17
### Fixed
- Order dependency JSON content to not rewrite it to disk when contents have not actually changed.
## [3.4.0] - 2026-03-26
- Internal fixes only
## [3.3.0] - 2026-03-18
- Internal fixes only
## [3.2.1] - 2026-03-12
### Fixed
- Only call AssetDatabase.Refresh() in Cleanup() when something actually got deleted [BUTFP-29](https://issuetracker.unity3d.com/product/unity/issues/guid/BUTFP-29)
- Avoid boxing from enum.Equals in ConvertSamples.
- Prevent unnecessary asset recompilation by skipping result writing when the content is unchanged.
### Changed
- Added assert when sample is measured for a non-existing ProfilerMarker.
- Use ProfilerRecorder API to measure profiler markers.
- Updated GC.Alloc measurements (GC() function of Measure.Method) to collect allocation count on the running thread only.
- SampleGroup.GetRecorder is now marked as [Obsolete] and will be removed in a future update.
## [3.2.0] - 2025-09-16
### Added
- Improved render threading mode reporting
## [3.1.0] - 2025-03-28
### Added
- Added an optional command-line argument "perfTestResults" to control the target location of performance test run
results file.
### Fixed
- Warmup cycles no longer record GC measurements.
- Setup and Cleanup cycles no longer contribute to GC measurements.
## [3.0.3] - 2023-09-11
### Fixed
- Fixed issue where exception in OnTestEnded callback would result in EndTest method not finalising properly
### Changed
- Temporarily removed "Open Script" from Performance Benchmark Window
- Some clarifications in documentation were added ("Extension" naming changed to "Package", Package limitations
clarified)
## [3.0.2] - 2023-06-29
### Changed
- Added additional InternalsVisibleTo attribute for internal test assembly
## [3.0.1] - 2023-06-23
### Changed
- Removed false "unityRelease" field from package.json
## [3.0.0] - 2023-06-05
### Added
- "Open Source Code" menu item to test results
- Test Filter to filter results by test name
### Changed
- Items in test results are grouped by classname
- Make Sample Groups to be shown only when clicking on items with them
- "New Data available" label re-located
- Replaced CamelCase headers with regular text headers in csv report
- Replaced TRUE/FALSE with Yes/No in the "Increase Is Better" column in csv report
- Renamed the default report name to PerformanceTestResults
- Documentation updates to reflect API changes
### Fixed
- Median calculation in case of even number of samples
- Deviation division by zero
- Exception thrown after clicking Cancel button (during Test Result export)
- The issue where the last column was always empty in csv report
## [3.0.0-pre.2] - 2023-04-06
### Added
- Help button, which redirects the user to documentation website
- Clear Results button, which clears all Performance test results
- Updated CI to support more Unity versions and expand test coverage
### Changed
- Updated the style of toolbar buttons to match that of the Test Runner window for consistency
- Export button is now disabled instead of hidden when there are no Performance test results
- Made AutoRefresh toggle retain its state after closing and reopening the window
- Moved the Performance Test Report from 'Window/Analysis' to 'Window/General' near Test Runner for better accessibility
### Removed
- Build project from CI
### Fixed
- Issue where running tests with the Test Report window open would cause the error message "The object of type '
Material' has been destroyed" to appear
- Issues where incorrect labels were displayed at certain scenarios
- Issues flagged by SonarQube
## [3.0.0-pre.1] - 2023-03-02
### Added
- Merged 2.8.1 changes that weren't reflected in 2.10.0 release
### Fixed
- Fixing issues from SonarQube check
- Updating obsolete API's that stopped working with recent Unity versions
- Clarified how to add package in Unity project in documentation
## [2.10.0] - 2021-11-01
### Added
- Support for dynamic measurement count in Measure.Method and Measure.Frames
## [2.9.0] - 2021-04-14
### Added
- Support for overriding measurement count
## [2.8.1] - 2021-03-16
### Removed
- Setting up date when building player. It will set up at the beginning of the run.
## [2.8.0] - 2021-03-16
### Added
- Overloads to measurements for overriding sample unit
### Fixed
- Cases where cleanup throws an exception
## [2.7.0] - 2021-02-19
### Changed
- Reduce metadata overhead when running locally by caching dependencies
- Restructured documentation
### Removed
- The need for link.xml
### Fixed
- Method measurement IterationsPerMeasurement
## [2.6.0] - 2021-01-12
### Added
- Build configuration support
## [2.5.1] - 2021-01-05
### Fixed
- Serialization for Performance Test Report window
## [2.5.0] - 2020-12-29
### Added
- Domain reload support
### Changed
- Switched from Newtonsoft.Json to Unity json module
## [2.4.1] - 2020-11-05
### Changed
- Metadata collection was made public
## [2.4.0] - 2020-09-16
### Added
Performance Test Report window updates:
- CSV export option.
- Monitoring of results file timestamp to support auto refresh when a new file is found.
- Display of timestamp of last loaded results file.
- Option to sort test report window by order the tests ran in (index). This is now the default.
- Min and max to the table.
- Improved titles and tooltips on columns
### Changed
- Upgraded json dependency to release version
- Reduced overhead introduced when running tests
## [2.3.1] - 2020-07-01
### Fixed
- Overhead introduced with Measure.Method no longer calculates execution time of Setup and Cleanup changes
## [2.3.0] - 2020-06-17
### Fixed
- Measure.Method overhead
- Measure.Method no longer calculates execution time of Setup and Cleanup
- Overwritten test name will be displayed with method name in Test Result viewer
## [2.2.0] - 2020-05-26
### Added
- Support for custom metadata
## [2.1.0] - 2020-05-14
### Added
- Flexible horizontal splitter for report window
### Fixed
- Date format
## [2.0.9] - 2020-03-23
### Fixed
- Profiler measurements for method measurements
- Throw exceptions when measuring NaN
## [2.0.8] - 2020-02-20
### Fixed
- Fix profiler marker capture when changing scenes in editor tests
- Only shift samplegroups for UI
## [2.0.7] - 2020-02-14
### Fixed
- Results parsing
## [2.0.6] - 2020-01-13
### Fixed
- Development player field
## [2.0.5] - 2020-01-13
### Changed
- Disallow multiple performance attributes
- Disallow empty samplegroup name
- Assign samplegroup name to frames measurements
## [2.0.4] - 2019-12-05
### Changed
- Update json package to support AOT platforms
## [2.0.3] - 2019-11-20
### Added
- New fields to data format BuildTarget, StereoRenderingPath
## [2.0.2] - 2019-11-20
### Changed
- Increased test serialization version
## [2.0.1] - 2019-11-20
### Fixed
- Player callbacks when no tests were executed
## [2.0.0] - 2019-11-19
### Added
- Tests to package testables
### Changed
- Refactored data format, reduced nesting
- Slight refactor on measurement API
- Shift sample units when printing results
- Switched to newtosoft json package
### Removed
- Unused fields
- Deprecated attributes
### Fixed
- Resources cleanup meta files
## [1.3.1] - 2019-11-05
### Fixed
- Warning after cleaning resources
- Test suite when running in the editor
## [1.3.0] - 2019-08-26
### Changed
- Switch to errors from exceptions when parsing results
- Increase minimum unity version to 2019.3
### Removed
- Metadata collectors tests
## [1.2.6] - 2019-08-22
### Changed
- Categorize performance tests as performance
- ProfilerMarkers can now be called with string params
- Switch measuring frames and methods to stopwatch
### Removed
- Profiler section on docs as the feature was removed
## [1.2.5] - 2019-06-17
### Added
- Test publish for CI
## [1.2.4] - 2019-06-17
### Added
- Test publish for CI
## [1.2.3] - 2019-06-14
### Changed
- Updated changelog
## [1.2.2] - 2019-06-13
### Added
- Support for domain reload
## [1.2.1] - 2019-06-07
### Fixed
- Bug that would cause player build failures
## [1.2.0] - 2019-05-23
### Changed
- Increase unity version to 2019.2
## [1.1.0] - 2019-05-22
### Changed
- Update assembly definition formats to avoid testables in package manifest
## [1.0.9] - 2019-05-21
### Changed
- Update scripting runtime setting for 2019.3
## [1.0.8] - 2019-03-08
### Added
- Automation test deploy
## [1.0.7] - 2019-03-08
### Added
- Automation test deploy
## [1.0.6] - 2019-03-04
### Changed
- Updated changelog
## [1.0.5] - 2019-03-04
### Added
- Conditional support for 2019.1
## [1.0.4] - 2019-02-18
### Removed
- Unnecessary meta files
## [1.0.3] - 2019-02-18
### Changed
- package.json update
## [1.0.2] - 2019-02-18
### Changed
- package.json update
## [1.0.1] - 2019-02-18
### Changed
- Updated Documentation to reflect breaking changes
## [1.0.0] - 2019-02-15
### Changed
- Refactor attributes
## [0.1.50] - 2019-01-15
### Changed
- Results paths to persistent data
## [0.1.49] - 2018-12-04
### Changed
- Revert changes to profiler and GC
## [0.1.48] - 2018-11-22
### Changed
- Doc updates and ignore GC api in editor due to api issues
## [0.1.47] - 2018-11-14
### Removed
- Debug logs
## [0.1.46] - 2018-11-14
### Fixed
- Breaking changes introduced by testrunner API rename
## [0.1.45] - 2018-11-08
### Fixed
- Breaking changes to data submodule
## [0.1.44] - 2018-11-08
### Changed
- Disable GC and update API to work around warning
## [0.1.43] - 2018-10-30
### Fixed
- Method measurements setup and cleanup
## [0.1.42] - 2018-10-15
### Added
- Button on report window to open profiler output for test
- Save profiler output on perf tests
### Removed
- Unsupported features for legacy scripting runtime
- Unnecessary assembly definition
### Fixed
- Version attribute for test cases
## [0.1.41] - 2018-10-02
### Added
- Test report graph
## [0.1.40] - 2018-09-17
### Changed
- Update documentation
## [0.1.39] - 2018-09-14
### Removed
- Duplicate module from docs
## [0.1.38] - 2018-09-14
### Changed
- Updated documentation
## [0.1.36] - 2018-08-27
### Changed
- ProfilerMarkers now take params as arguments
## [0.1.35] - 2018-08-27
### Added
Measure.Method improvements:
- Add GC allocation to Measure.Method
- Add setup/cleanup for Measure.Method
- Move order of calls for Measure.Scope
## [0.1.34] - 2018-08-16
### Fixed
- Obsolete warnings
## [0.1.33] - 2018-08-03
### Fixed
- Obsolete warnings, doc update with modules and internals, ValueSource fix
## [0.1.32] - 2018-07-09
### Added
- Method and Frames measurements can now specify custom warmup, measurement and iteration counts
## [0.1.31] - 2018-07-04
### Changed
- Marked metadata tests with performance category
## [0.1.30] - 2018-06-27
### Fixed
- Method measurement
## [0.1.29] - 2018-06-12
### Changed
- Moving back to json in xml due to multiple instabilities
## [0.1.28] - 2018-06-01
### Removed
- json printing from output
## [0.1.27] - 2018-05-31
### Added
- Meta files to npm ignore
## [0.1.26] - 2018-05-31
### Changed
Preparing package for moving to public registry:
- Inversed changelog order
- Excluded CI files from published package
## [0.1.25] - 2018-05-31
### Removed
- Missing meta files
## [0.1.24] - 2018-05-31
### Changed
- Print out json to xml by default for backwards compatability
## [0.1.23] - 2018-05-30
### Fixed
- Issues with packman, bumping up version
## [0.1.22] - 2018-05-29
### Added
- Option to specify custom Measure.Method Execution and Warmup count
## [0.1.21] - 2018-05-25
### Fixed
- Issues introduced by .18 fix
## [0.1.19] - 2018-05-24
### Changed
- Package has been renamed to `com.unity.test-framework.performance` to match test framework
## [0.1.18] - 2018-05-24
### Fixed
- Fix SetUp and TearDown for 2018.1
## [0.1.17] - 2018-05-23
### Changed
- Refactor Method and Frames measurements
- Metadata collected using internal test runner API and player connection for 2018.3+
## [0.1.16] - 2018-05-09
### Fixed
- Bug fix regarding measureme methods being disposed twice
## [0.1.15] - 2018-05-02
### Fixed
- Metadata test, the test was failing if a json file was missing for playmode tests
## [0.1.14] - 2018-04-30
### Added
- Addition of measuring a method or frames for certain amount of times or for duration
- Introduced SampleGroupDefinition
### Changed
- Refactored measuring methods
### Removed
- Removes linq usage for due to issues with AOT platforms
## [0.1.13] - 2018-04-15
### Added
- Added total, std and sample count aggregations
- Added sample unit to multi sample groups
### Removed
- Removed totaltime from frametime measurements
### Fixed
- Fixed android metadata collecting
## [0.1.12] - 2018-04-11
### Changed
- Naming
### Fixed
- json serialization
## [0.1.11] - 2018-04-09
### Fixed
- 2018.1 internal namespaces
## [0.1.10] - 2018-04-09
### Added
- Added editmode and playmode tests that collect metadata
### Changed
- Change fields to UpperCamelCase
## [0.1.9] - 2018-04-06
### Added
- json output for 2018.1 which will be printed after test run
## [0.1.8] - 2018-04-03
### Fixed
- Fix an exception on 2018.1
## [0.1.7] - 2018-04-03
### Changed
- Changed some of the names to match new convention
- Addressed typos in docs
- Multiple overloads replaced by using default arguments
## [0.1.6] - 2018-03-28
### Added
- Measure.Custom got a new overload with SampleGroup
- Readme now includes installation and more examples
## [0.1.5] - 2018-03-20
### Added
- Checks for usage outside of Performance tests
## [0.1.4] - 2018-03-20
### Added
- System info to performance test output
- Preparing for reporting test data
## [0.1.3] - 2018-03-14
### Removed
- Temporarily removing tests from the package into separate repo
## [0.1.2] - 2018-03-14
### Fixed
- Update for a missing bracket
## [0.1.1] - 2018-03-14
### Added
- Test output now includes json that can be used to parse performance data from TestResults.xml
- Added defines to be compatible with 2018.1 and newer
- Measurement methods can now take in SampleGroup as argument
### Removed
- Removed unnecessary overloads for measurements due to introduction of SampleGroup
## [0.1.0] - 2018-02-27
### This is the first release of *Unity Package performancetesting*.
Initial version.
@@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 5cb0d418e2eda6f4a8ebfe6910c17dd4
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,13 @@
* [Performance Testing Package overview](./index.md)
* [Taking measurements](./taking-measurements.md)
* [Measure.Method](./measure-method.md)
* [Measure.Frames](./measure-frames.md)
* [Measure.Scope](./measure-scope.md)
* [Measure.ProfileMarkers](./measure-profile-markers.md)
* [Measure.Custom](./measure-custom.md)
* [Writing tests](./writing-tests.md)
* [Viewing results](./viewing-results.md)
* [Reference]
* [Classes](./classes.md)
* [Test attributes](./test-attributes.md)
* [Command-line arguments](./cmd-line-args.md)
@@ -0,0 +1,16 @@
# Classes
This section contains a reference for classes relevant when working with the Performance Testing Package.
## SampleGroup
**class SampleGroup** - represents a group of samples with the same purpose that share a name, sample unit and whether
an increase is better.
Optional parameters
- **Name** : Name of the measurement. If unspecified, "Time" is used as the default name.
- **Unit** : Unit of the measurement to report samples in. Possible values are:
Nanosecond, Microsecond, Millisecond, Second, Byte, Kilobyte, Megabyte, Gigabyte
- **IncreaseIsBetter** : If true, an increase in the measurement value is considered a performance improvement (
progression). If false, an increase is treated as a performance regression. False by default.
@@ -0,0 +1,29 @@
# Command-line arguments
This section lists Unity arguments supported by the package when running performance tests from the command line. For a
general overview of the topic, see
the [Unity Test Framework documentation](https://docs.unity3d.com/Packages/com.unity.test-framework@latest).
## Supported arguments
### perfTestResults
The path where Unity should save the JSON file with performance test run results. By default, Unity saves it in
the [Application.persistentDataPath](https://docs.unity3d.com/ScriptReference/Application-persistentDataPath.html)
folder, and, if you specify the `testResults` argument of the Unity Test Framework package, in the folder that holds the
NUnit test results file as well.
Please note that when this argument is not provided, Unity generates two report files at the default results location -
the JSON file with performance test run results as well as the NUnit test results XML file that these results are
extracted from. This behaviour currently exists for backwards compatibility reasons and will be deprecated over time, as
the destination path of the test results XML file is already controlled by the `testResults` command-line argument of
the Unity Test Framework package.
Unity updates the results file after each test finishes rather than only when the run completes, so a run that ends
unexpectedly still leaves the results that it had already measured on disk.
## Example of command-line usage
```bash
Unity.exe -runTests -batchmode -projectPath PATH_TO_YOUR_PROJECT -testResults C:\temp\results.xml -perfTestResults C:\temp\perfResults.json
```
@@ -0,0 +1,122 @@
# Performance Testing Package for Unity Test Framework
The Unity Performance Testing Package extends Unity Test Framework with performance testing capabilities. It provides an
API and test case decorators for taking measurements/samples of Unity profiler markers, and other custom metrics, in the
Unity Editor and built players. It also collects configuration metadata that is useful for comparing data across
different hardware and configurations, such as build and player settings.
The Performance Testing Package is intended for use with the Unity Test Framework. You should be familiar with how to
create and run tests as described in
the [Unity Test Framework documentation](https://docs.unity3d.com/Packages/com.unity.test-framework@latest).
> **Note:** When tests are run with Unity Test Framework, a development player is always built to support communication
> between the editor and player, effectively overriding the development build setting from the build settings UI or
> scripting API.
## Installing the Performance Testing Package
From Unity version 6000.6 the Performance Testing package is a [Core package](https://docs.unity3d.com/Manual/pack-core.html) which is automatically installed with the Unity Editor.
To install this package with an older supported Unity version, use the [Package Manager](https://docs.unity3d.com/Manual/upm-ui-actions.html).
When the package is installed, add a reference to `Unity.PerformanceTesting` in your assembly definition to access the
performance testing APIs.
![Performance test example01](images/example04.png)
## Unity version compatibility
Unity releases can often include changes that break compatibility with the Performance Testing Package, so we cannot
currently guarantee latest package version compatability with every Unity version. The table below shows which version
of the package is compatible with which Unity release streams.
| Unity stream | Package version |
|--------------|-----------------|
| 6000.7 | 6.7.0 |
| 6000.6 | 6.6.0 |
| 6000.3 | 3.5.0 |
| 6000.0 | 3.5.0 |
| 2022.3 | 3.5.0 |
| 2020.3 | 3.5.0 |
## Tips
### Project settings
- Remove all but one of the [Quality level settings](https://docs.unity3d.com/Manual/class-QualitySettings.html) in *
*Project Settings > Quality**. Otherwise, you may have different configurations when running on different platforms.
If you require different settings per platform then make sure they are being set as expected.
- Disable VSync under **Project Settings > Quality**. Some platforms like Android have a forced VSync and this will not
be possible.
- Disable HW Reporting **PlayerSettings -> Other -> Disable HW Reporting**.
- Remove camera and run in batchmode if you are not measuring rendering.
### Generating assets
Use [IPrebuildSetup](https://docs.unity3d.com/Packages/com.unity.test-framework@1.1/api/UnityEngine.TestTools.IPrebuildSetup.html)
attribute when you need to generate assets.
Place assets in Resources or [StreamingAssets](https://docs.unity3d.com/Manual/SpecialFolders.html) folders, scenes can
be placed anywhere in the project, but should be added to build settings.
#### Example 1: IPrebuildSetup implementation
``` csharp
public class TestsWithPrebuildStep : IPrebuildSetup
{
public void Setup()
{
// this code is executed before entering playmode or the player is executed
}
}
public class MyAmazingPerformanceTest
{
[Test, Performance]
[PrebuildSetup(typeof(TestsWithPrebuildStep))]
public void Test()
{
...
}
}
```
When loading scenes in IPrebuildSetup you have to use `LoadSceneMode.Additive`.
#### Example 2: Using EditorSceneManager to create new scenes additively, save and add them to build settings.
``` csharp
private static string m_ArtifactsPath = "Assets/Artifacts/";
public static Scene NewScene(NewSceneSetup setup)
{
Scene scene = EditorSceneManager.NewScene(setup, NewSceneMode.Additive);
EditorSceneManager.SetActiveScene(scene);
return scene;
}
public static void SaveScene(Scene scene, string name, bool closeScene = true)
{
EditorSceneManager.SaveScene(scene, GetScenePath(name));
if (closeScene)
{
foreach (var sceneSetting in EditorBuildSettings.scenes)
if (sceneSetting.path == GetScenePath((name)))
return;
EditorSceneManager.CloseScene(scene, true);
EditorSceneManager.SetActiveScene(EditorSceneManager.GetSceneAt(0));
var newListOfScenes = new List<EditorBuildSettingsScene>();
newListOfScenes.Add(new EditorBuildSettingsScene(GetScenePath(name), true));
newListOfScenes.AddRange(EditorBuildSettings.scenes);
EditorBuildSettings.scenes = newListOfScenes.ToArray();
}
}
public static string GetScenePath(string name)
{
return m_ArtifactsPath + name + ".unity";
}
```
@@ -0,0 +1,16 @@
# Measure.Custom()
When you want to record samples outside of frame time, method time, or profiler markers, use a custom measurement. It
can be any value.
#### Example: Use a custom measurement to capture total allocated memory
``` csharp
[Test, Performance]
public void Test()
{
SampleGroup sampleGroup = new SampleGroup("TotalAllocatedMemory", SampleUnit.Megabyte, false);
var allocatedMemory = UnityEngine.Profiling.Profiler.GetTotalAllocatedMemoryLong() / 1048576f;
Measure.Custom(sampleGroup, allocatedMemory);
}
```
@@ -0,0 +1,120 @@
# Measure.Frames()
Records time per frame by default and provides additional properties/methods to control how the measurements are taken:
* **WarmupCount(int n)** - number of times to execute before measurements are collected. If unspecified, a default
warmup is executed for 80 ms or until at least 3 full frames have rendered, whichever is longest.
* **MeasurementCount(int n)** - number of frames to capture measurements for. If this value is not specified, as many
frames as possible are captured until approximately 500 ms has elapsed.
* **DynamicMeasurementCount(OutlierMode outlierMode)** - dynamically find a suitable measurement count based on the
margin of error of the samples. The measurements will stop once a certain amount of samples (specified by a confidence
interval) falls within an acceptable error range from the result (defined by a relative error of the mean). A default
margin of error range of 2% and a default confidence interval of 99% will be used. Statistical outliers will not be
taken into account unless different behaviour is specified through the outlierMode argument.
* **DynamicMeasurementCount(double maxRelativeError, ConfidenceLevel confidenceLevel, OutlierMode outlierMode)** -
dynamically find a suitable measurement count based on the margin of error of the samples and using the provided
confidence interval and error range.
* **DontRecordFrametime()** - disables frametime measurement.
* **ProfilerMarkers(...)** - sample profile markers per frame. Does not work for deep profiling and
`Profiler.BeginSample()`
* **SampleGroup(string name)** - name of the measurement, defaults to "Time" if unspecified.
* **Scope()** - measures frame times in a given coroutine scope. By default it uses a SampleGroup named "Time" with
Milliseconds as measurement unit. You can also create your own SampleGroup, specifying a custom name and the
measurement unit you want your results in, see [example 5](#example-5-specify-custom-samplegroup-in-the-scope).
## Limitations
* Not supported in Unity Test
Framework [Edit Mode tests](https://docs.unity3d.com/Packages/com.unity.test-framework@latest?subfolder=/manual/edit-mode-vs-play-mode-tests.html#edit-mode-tests).
#### Example 1: Simple frame time measurement using default values of at least 7 frames and default WarmupCount (see description above).
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
...
yield return Measure.Frames().Run();
}
```
#### Example 2: Sample profile markers per frame, disable frametime measurement
If you'd like to sample profiler markers across multiple frames and don't need to record frametime, it is possible to
disable the frame time measurement.
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
...
yield return Measure.Frames()
.ProfilerMarkers(...)
.DontRecordFrametime()
.Run();
}
```
#### Example 3: Sample frame times in a scope
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
using (Measure.Frames().Scope())
{
yield return ...;
}
}
```
#### Example 4: Specify custom WarmupCount and MeasurementCount per frame
If you want more control, you can specify how many frames you want to measure.
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
...
yield return Measure.Frames()
.WarmupCount(5)
.MeasurementCount(10)
.Run();
}
```
#### Example 5: Specify Custom SampleGroup in the Scope
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
var sg = new SampleGroup("MarkerName", SampleUnit.Second);
using (Measure.Frames().Scope(sg))
{
yield return ...;
}
}
```
#### Example 6: Sample profile markers per frame with custom SampleGroups that change sample unit
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
var sampleGroup = new SampleGroup("Name", SampleUnit.Milliseconds);
var profileMarkersSampleGroups = new []{
new SampleGroup("MarkerName", SampleUnit.Second),
new SampleGroup("MarkerName1", SampleUnit.Nanosecond)
};
yield return Measure.Frames().SampleGroup(sg).ProfilerMarkers(profileMarkersSampleGroups).Run();
...
}
```
@@ -0,0 +1,63 @@
# Measure.Method()
Executes the provided method, sampling performance using the following additional properties/methods to control how the
measurements are taken:
* **WarmupCount(int n)** - number of times to execute before measurements are collected. If unspecified, a default
warmup is executed for 100 ms or until at least 3 method executions have completed, whichever is longer.
* **MeasurementCount(int n)** - number of measurements to capture, defaults to 9 if not specified.
* **DynamicMeasurementCount(OutlierMode outlierMode)** - dynamically find a suitable measurement count based on the
margin of error of the samples. The measurements will stop once a certain amount of samples (specified by a confidence
interval) falls within an acceptable error range from the result (defined by a relative error of the mean). A default
margin of error range of 2% and a default confidence interval of 99% will be used. Statistical outliers will not be
taken into account unless different behaviour is specified through the outlierMode argument.
* **DynamicMeasurementCount(double maxRelativeError, ConfidenceLevel confidenceLevel, OutlierMode outlierMode)** -
dynamically find a suitable measurement count based on the margin of error of the samples and using the provided
confidence interval and error range.
* **IterationsPerMeasurement(int n)** - number of method executions per measurement to use. If this value is not
specified, the method is executed as many times as possible until approximately 100 ms has elapsed.
* **SampleGroup(string name)** - name of the measurement, defaults to "Time" if unspecified.
* **SampleGroup(SampleGroup sampleGroup)** - a sample group with a custom name and measurement unit. This will override
the otherwise default value of "Time".
* **GC()** - if specified, measures the total number of Garbage Collection allocation calls (known as `GC.Alloc` in Unity Profiler) as an additional sample group with ".GC()" name postfix. Also performs a full blocking garbage collection immediately before each timed region, so that a collection triggered by the measured method's own allocations does not land inside it and show up as periodic spikes across the samples. Normally that is one collection per measurement, but `SetUp` and `CleanUp` cause each of the `IterationsPerMeasurement` inner executions to be timed separately, so each of those gets its own collection - a 10-measurement run with 5 iterations per measurement and a setup action performs 50 collections, which is considerably slower. Because of all this, the times reported for a method that allocates are not directly comparable between a run with `GC()` and one without.
* **SetUp(Action action)** - is called every iteration before method execution. Setup time is not measured.
* **CleanUp(Action action)** - is called every iteration after method execution. Cleanup time is not measured.
#### Example 1: Simple method measurement using default values
``` csharp
[Test, Performance]
public void Test()
{
Measure.Method(() => { ... }).Run();
}
```
#### Example 2: Customize Measure.Method properties
``` csharp
[Test, Performance]
public void Test()
{
Measure.Method(() => { ... })
.WarmupCount(10)
.MeasurementCount(10)
.IterationsPerMeasurement(5)
.GC()
.SetUp(()=> {/*setup code*/})
.CleanUp(()=> {/*cleanup code*/})
.Run();
}
```
#### Example 3: Measure.Method, SampleGroup and ProfilerMarkers
``` csharp
[Test, Performance]
public void Test()
{
var sg = new SampleGroup("Name", SampleUnit.Microsecond);
var sgMarker = new SampleGroup("MarkerName", SampleUnit.Seconds);
Measure.Method(() => { ... }).SampleGroup(sg).ProfilerMarkers(sgMarker).Run();
}
```
@@ -0,0 +1,57 @@
# Measure.ProfilerMarkers()
Used to record profiler markers. Profiler marker timings are recorded automatically and sampled within the scope of the
`using` statement. Names should match profiler marker labels. Profiler markers are sampled once per frame. Sampling the
same profiler marker per frame will result in the sum of all invocations.
You can also create your own SampleGroups, specifying a custom name and the measurement units you want your results in,
see [example 2](#example-2-measuring-profiler-markers-in-a-scope-with-custom-samplegroups).
## Limitations
* Not supported in Unity Test
Framework [Edit Mode tests](https://docs.unity3d.com/Packages/com.unity.test-framework@latest?subfolder=/manual/edit-mode-vs-play-mode-tests.html#edit-mode-tests).
* Not supported in the Unity
Profiler's [Deep Profiling](https://docs.unity3d.com/Manual/ProfilerWindow.html#deep-profiling) mode.
* Profiler markers created using `Profiler.BeginSample()` are not supported, switch to `ProfilerMarker` if possible.
#### Example: Measuring profiler markers in a scope
``` csharp
[Test, Performance]
public void Test()
{
string[] markers =
{
"Instantiate",
"Instantiate.Copy",
"Instantiate.Produce",
"Instantiate.Awake"
};
using(Measure.ProfilerMarkers(markers))
{
...
}
}
```
#### Example 2: Measuring profiler markers in a scope with custom SampleGroups
``` csharp
[UnityTest, Performance]
public IEnumerator Test()
{
var sampleGroups = new []{
new SampleGroup("Instantiate", SampleUnit.Second),
new SampleGroup("Instantiate.Copy", SampleUnit.Nanosecond),
new SampleGroup("Instantiate.Awake", SampleUnit.Microsecond)
};
using (Measure.ProfilerMarkers(sampleGroups))
{
...
}
}
```
@@ -0,0 +1,34 @@
# Measure.Scope(string name = "Time")
Measures execution time for the scope as a single time, for both synchronous and coroutine methods. Passing the name
argument overrides the name of the created SampleGroup.
The defualt SampleGroup is named "Time" and with Milliseconds as measurement unit. You can also create your own
SampleGroup, specifying a custom name and the measurement unit you want your results in,
see [example 2](#example-2-specify-custom-samplegroup).
#### Example 1: Measuring a scope; execution time is measured for everything in the using statement
``` csharp
[Test, Performance]
public void Test()
{
using(Measure.Scope())
{
...
}
}
```
#### Example 2: Specify Custom SampleGroup
``` csharp
[Test, Performance]
public void Test()
{
var sampleGroup = new SampleGroup("Scope", SampleUnit.Microsecond);
using (Measure.Scope(sg))
{
...
}
}
```
@@ -0,0 +1,14 @@
# Taking measurements
The Performance Testing Package provides several API methods for taking measurements in your performance test, depending
on what you need to measure and how you want to do it. Include `using Unity.PerformanceTesting` at the top of your
script to access the methods.
The pages in this section detail the specifics of each measurement method with examples:
* [Measure.Method](./measure-method.md)
* [Measure.Frames](./measure-frames.md)
* [Measure.Scope](./measure-scope.md)
* [Measure.ProfilerMarkers](./measure-profile-markers.md)
* [Measure.Custom](./measure-custom.md)
@@ -0,0 +1,15 @@
# Test Attributes
This section contains a reference for attributes supported by the Performance Testing Package.
**[Performance]** - Use this with `Test` and `UnityTest` attributes. It will initialize necessary test setup for
performance tests.
**[Test]** - Non-yielding test. This type of test starts and ends within the same frame.
**[UnityTest]** - Yielding test. This is a good choice if you want to sample measurements across multiple frames. For
more on the difference between `[Test]` and `[UnityTest]`, see the Unity Test
Framework [documentation](https://docs.unity3d.com/Packages/com.unity.test-framework@1.1/manual/reference-attribute-unitytest.html).
**[Version(string version)]** - Performance tests should be versioned with every change. If not specified, the default
used is 1. This is essential when comparing results as results will vary any time the test changes.
@@ -0,0 +1,36 @@
## Output
When a test is selected in the **Test Runner** window in the Unity Editor, each performance test will have a performance
test summary. This summary includes every sample group's aggregated samples such as median, min, max, average, standard
deviation, sample count and sum of all samples.
#### Example: Performance Test Summary from Test Runner window
```
Time Microsecond Median:2018.60 Min:1903.00 Max:2819.80 Avg:2186.35 Std:368.42 SampleCount: 4 Sum: 8745.40
```
## The Performance Test Report
The Performance Test Report (**Window > General > Performance Test Report**) shows a detailed breakdown of individual
test runs. This can be used to assess the stability of each test. It provides a visualization of each individual sample
recorded within a sample group along with summary statistics for the selected test.
The Performance Test Report is split into two views: the *test view* and the *sample group view*.
**Test View:** Provides a list of all tests. Tests are separated by class names. Each of the columns can be clicked to
sort the view. Column values show the sample group with highest deviation.
* *Name* - Name of the test.
* *Deviation* - The deviation is calculated by dividing the standard deviation by the median for a sample group. It
shows the sample group with the highest 'deviation' value. Useful for defining stability of the test.
* *Standard Deviation* - Standard deviation of the samples in a sample group. It shows the sample group with the highest
standard deviation.
**Sample Group View:** Visualizes sample groups for the test selected in the Test View. Provides:
* Sample group summary displaying the min, max, and median values for a given sample group.
* Samples displayed in a bar chart, ordered by time, with a blue line indicating the median.
* Box plot showing upper (75%) and lower (25%) quartiles, min, max and median of the samples for a given sample group.
![Performance test report](images/graphtool.png)
@@ -0,0 +1,264 @@
# Writing a simple test
This example uses `Measure.Method` to measure the performance of Vector2 operations. This executes the provided method
and samples performance. Increasing `MeasurementCount` from the default of 7 to 20 improves performance test stability.
You can execute a `Vector2.MoveTowards` test in Edit Mode or Play Mode.
``` csharp
[Test, Performance]
public void Vector2_operations()
{
var a = Vector2.one;
var b = Vector2.zero;
Measure.Method(() =>
{
Vector2.MoveTowards(a, b, 0.5f);
Vector2.ClampMagnitude(a, 0.5f);
Vector2.Reflect(a, b);
Vector2.SignedAngle(a, b);
})
.MeasurementCount(20)
.Run();
}
```
To view the results, go to **Window > General > Performance Test Report**.
![Performance test example01](images/example01.png)
In this example, the results show that the first method execution took five times longer than the subsequent methods,
and those subsequent method executions were unstable. Also, you can't tell from these test results how long it took for
each Vector2 operation to execute.
## Improving test stability
Now we'll look at how to improve the stability of our test. Instability can occur at the beginning of a test for several
reasons, such as entering Play Mode or method initialization, because the tested method is quite fast and more sensitive
to other running background processes.
To improve stability, use `WarmupCount(n)`. This allows you to execute methods before recording data, so Unity doesn't
record method initialization. The simplest way to increase test execution time is to repeat the method in a loop.
Avoid having measurements that are less than 1ms because they are usually more sensitive to unstable environments.
To help you track the execution time for each operation, split the Vector2 operations into several tests. Often, when
writing tests, we use setup and clean up methods to isolate the test environment. However, in this case, methods are
isolated and do not affect other methods, so we don't need a cleanup or setup.
The following code example shows a performance test for the `Vector2.MoveTowards` operation. Other Vector2 performance
tests are identical.
``` csharp
[Test, Performance]
public void Vector2_MoveTowards()
{
Measure.Method(() =>
{
Vector2.MoveTowards(Vector2.one, Vector2.zero, 0.5f);
})
.WarmupCount(5)
.IterationsPerMeasurement(10000)
.MeasurementCount(20)
.Run();
}
```
With 100000 iterations in this test, we see a small fluctuation in method execution time but the standard deviation is
low, which means the test is reasonably stable.
![Performance test example02](images/example02.png)
## Measure a Play Mode only method
To measure a method that only runs in Play Mode (for example `Physics.Raycast`), you can use `Measure.Frames()`, which
records time per frame by default. To only measure `Physics.Raycast` time, you can disable frame time measurements with
`DontRecordFrametime` and just measure the `Physics.Raycast` profiler marker.
This test creates objects that you need to dispose of at the end of each test, because multiple unnecessary objects can
affect the next test results. Use the SetUp method to create GameObjects, and the TearDown method to destroy the created
GameObjects after each test.
``` csharp
[SetUp]
public void SetUp()
{
for (int i = 0; i < 100; ++i)
{
GameObject raycaster = new GameObject("raycast", typeof(Raycast));
raycaster.transform.SetParent(_parent.transform);
}
}
[UnityTest, Performance]
public IEnumerator Physics_RaycastTests()
{
string profilierMarker = "Physics.Raycast";
yield return Measure.Frames()
.WarmupCount(3)
.DontRecordFrametime()
.MeasurementCount(10)
.ProfilerMarkers(profilierMarker)
.Run();
}
public class Raycast : MonoBehaviour
{
private void Update()
{
RaycastHit hit;
bool ray = Physics.Raycast(
transform.position,
Vector3.forward,
out hit,
Mathf.Infinity);
}
}
[TearDown]
public void TearDown()
{
GameObject.DestroyImmediate(_parent);
}
```
To record your own measurements, create a new sample group and record a custom metric. The following example measures
`Allocated` and `Reserved` memory.
``` csharp
[Test, Performance]
public void Empty()
{
var allocated = new SampleGroup("TotalAllocatedMemory", SampleUnit.Megabyte);
var reserved = new SampleGroup("TotalReservedMemory", SampleUnit.Megabyte);
using (Measure.Scope())
{
Measure.Custom(allocated, UnityEngine.Profiling.Profiler.GetTotalAllocatedMemoryLong() / 1048576f);
Measure.Custom(reserved, UnityEngine.Profiling.Profiler.GetTotalReservedMemoryLong() / 1048576f);
}
}
```
Before you start to collect package performance data, make sure the tests you run locally are stable (the data set
deviation is <5%). In the **Performance Test Report** window, ensure the test isn't fluctuating and that the results
between runs are similar.
Results of performance tests run on a local machine can be significantly different to previous test runs because of
other applications running in the background, CPU overheating, or CPU boosting. Make sure that CPU intensive
applications are turned off where possible. You can disable CPU boost in the BIOS or with third-party software such as
Real Temp.
For comparing performance data between runs, use
the [Unity Performance Benchmark Reporter](https://github.com/Unity-Technologies/PerformanceBenchmarkReporter/wiki),
which provides a graphical HTML report that enables you to compare performance metric baselines and subsequent
performance metrics.
## Further examples
#### Example 1: Measure Frame Time For Scene
``` csharp
[UnityTest, Performance, Version("4")]
public IEnumerator MainSceneFrameTime_StartPosition()
{
SceneManager.LoadScene("Demo", LoadSceneMode.Single);
// Measure initial time of first 25 frames after loading the scene
using(Measure.Frames().Scope("FrameTime.FirstFramesAfterLoadingScene"))
{
for (var i = 0; i < 25; i++)
{
yield return null;
}
}
// Measure frame times for ten seconds during rest of the "Demo" scene
using (Measure.Frames().Scope("FrameTime.Main"))
{
yield return new WaitForSeconds(10);
}
}
```
#### Example 2: Measure execution time to serialize simple object to JSON
``` csharp
[Test, Performance, Version("2")]
public void Serialize_SimpleObject()
{
var obj = new SimpleObject();
obj.Init();
Measure.Method(() => JsonUtility.ToJson(obj)).Run();
}
[Serializable]
public class SimpleObject
{
public int IntField;
public string StringField;
public float FloatField;
public bool BoolField;
[Serializable]
public struct NestedStruct
{
public int A, B;
}
public NestedStruct Str;
public Vector3 Position;
public void Init()
{
IntField = 1;
StringField = "Test";
FloatField = 2.0f;
BoolField = false;
Str.A = 15;
Str.B = 20;
}
}
```
#### Example 3: Measure execution time to create 5000 simple cubes
``` csharp
string[] markers =
{
"Instantiate",
"Instantiate.Copy",
"Instantiate.Produce",
"Instantiate.Awake"
};
[Test, Performance]
public void Instantiate_CreateCubes()
{
using (Measure.ProfilerMarkers(markers))
{
using(Measure.Scope())
{
var cube = GameObject.CreatePrimitive(PrimitiveType.Cube);
for (var i = 0; i < 5000; i++)
{
UnityEngine.Object.Instantiate(cube);
}
}
}
}
```
#### Example 4: Custom measurement to capture total allocated and reserved memory
``` csharp
[Test, Performance, Version("1")]
public void Measure_Empty()
{
var allocated = new SampleGroup("TotalAllocatedMemory", SampleUnit.Megabyte);
var reserved = new SampleGroup("TotalReservedMemory", SampleUnit.Megabyte);
Measure.Custom(allocated, UnityEngine.Profiling.Profiler.GetTotalAllocatedMemoryLong() / 1048576f);
Measure.Custom(reserved, UnityEngine.Profiling.Profiler.GetTotalReservedMemoryLong() / 1048576f);
}
```
@@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 0855336f89ed65c45a15c56d2269fffd
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,6 @@
using System.Reflection;
using System.Runtime.CompilerServices;
[assembly: InternalsVisibleTo("Unity.PerformanceTesting.Tests.Editor")]
[assembly: AssemblyVersion("3.4.2")]
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: fd4abc6933ef4db296c17e31d92d800b
timeCreated: 1617350583
@@ -0,0 +1,68 @@
using System;
using UnityEditor.TestTools.TestRunner.Api;
using UnityEngine;
namespace Unity.PerformanceTesting.Editor
{
[Serializable]
class CmdLineResultsSavingCallbacks : ScriptableObject, ICallbacks
{
[SerializeField]
string resultsLocation;
IncrementalRunResultsWriter m_IncrementalWriter;
IncrementalRunResultsWriter IncrementalWriter
{
get
{
if (m_IncrementalWriter == null)
{
m_IncrementalWriter = new IncrementalRunResultsWriter();
}
return m_IncrementalWriter;
}
}
void ICallbacks.RunStarted(ITestAdaptor testsToRun)
{
PerformanceTest.Active = null;
IncrementalWriter.BeginRun();
}
void ICallbacks.RunFinished(ITestResultAdaptor result)
{
PlayerCallbacks.Saved = false;
try
{
var performanceTestRun = TestResultsParser.GetPerformanceTestRunData(result);
if (performanceTestRun == null)
{
return;
}
Debug.LogFormat(LogType.Log, LogOption.NoStacktrace, null, "Saving performance results to: {0}", resultsLocation);
IncrementalRunResultsWriter.WriteResultsFile(resultsLocation, performanceTestRun);
}
catch (Exception e)
{
Debug.LogError("Saving performance results file failed.");
Debug.LogException(e);
}
}
void ICallbacks.TestStarted(ITestAdaptor test) { }
void ICallbacks.TestFinished(ITestResultAdaptor result)
{
IncrementalWriter.AppendTestResults(resultsLocation, result.Output);
}
public void SetResultsLocation(string perfTestResultsPath)
{
resultsLocation = perfTestResultsPath;
}
}
}
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: 65a435b219724f1c9e25b4f8d4564bb3
timeCreated: 1742825901
@@ -0,0 +1,134 @@
using System;
using System.IO;
using Unity.PerformanceTesting.Data;
using UnityEditor;
using UnityEngine;
namespace Unity.PerformanceTesting.Editor
{
// Keeps a complete performance results file on disk while a run is still in progress, so that
// results already collected survive a crash, a test timeout or any other premature shutdown.
class IncrementalRunResultsWriter
{
// The results file is replaced by moving this file over it. A shutdown between removing the
// previous file and the move landing therefore leaves the complete results here and nothing
// at the results path, so whoever recovers the results of a run that went down has to look
// here when the results file itself is missing.
internal const string TempFileExtension = ".tmp";
const string k_RunTokenKey = "Unity.PerformanceTesting.IncrementalResults.RunToken";
const string k_WrittenRunTokenKey = "Unity.PerformanceTesting.IncrementalResults.WrittenRunToken";
Run m_Run;
bool m_RunStateInitialized;
bool m_WriteFailureLogged;
internal void BeginRun()
{
m_Run = null;
m_RunStateInitialized = true;
m_WriteFailureLogged = false;
// The results file of an earlier run is left in place until this run has something to
// replace it with, and is identified as not belonging to this run by the token.
SessionState.SetString(k_RunTokenKey, Guid.NewGuid().ToString("N"));
SessionState.EraseString(k_WrittenRunTokenKey);
}
internal void AppendTestResults(string resultsPath, string testOutput)
{
if (string.IsNullOrEmpty(resultsPath) || string.IsNullOrEmpty(testOutput))
return;
try
{
RestoreRunStateFromDisk(resultsPath);
if (!TestResultsParser.TryAppendPerformanceRunData(testOutput, ref m_Run))
return;
WriteResultsFile(resultsPath, m_Run);
SessionState.SetString(k_WrittenRunTokenKey, SessionState.GetString(k_RunTokenKey, string.Empty));
}
catch (Exception e)
{
// Only reported once because the run is still able to write its results when it finishes.
if (m_WriteFailureLogged)
return;
m_WriteFailureLogged = true;
Debug.LogWarning($"Failed to save in-progress performance results to '{resultsPath}'.\n{e}");
}
}
internal static void WriteResultsFile(string resultsPath, Run run)
{
var json = JsonUtility.ToJson(run, true);
CreateDirectoryIfNecessary(resultsPath);
// Staged through a temporary file so that a shutdown part way through the write cannot
// leave a truncated, unparsable results file behind. File.Replace is not dependable on
// every file system the results are written to, so the previous file is removed first.
// The temporary file is complete by the time the removal happens, which is what lets a
// reader treat it as the results of the run when the results file is not there.
var tempPath = GetTempFilePath(resultsPath);
File.WriteAllText(tempPath, json);
if (File.Exists(resultsPath))
{
File.Delete(resultsPath);
}
File.Move(tempPath, resultsPath);
}
internal static string GetTempFilePath(string resultsPath)
{
return resultsPath + TempFileExtension;
}
// A domain reload discards the accumulated run, so continue from the file on disk rather
// than replacing it with only the results gathered since the reload.
void RestoreRunStateFromDisk(string resultsPath)
{
if (m_RunStateInitialized)
return;
m_RunStateInitialized = true;
if (!WrittenResultsBelongToCurrentRun())
return;
// A replacement that never landed leaves the results of this run in the temporary file
// alone, so that file stands in for the results file when it is the only one present.
var pathToRestoreFrom = File.Exists(resultsPath) ? resultsPath : GetTempFilePath(resultsPath);
if (!File.Exists(pathToRestoreFrom))
return;
try
{
m_Run = JsonUtility.FromJson<Run>(File.ReadAllText(pathToRestoreFrom));
}
catch (Exception e)
{
Debug.LogWarning($"Failed to read the in-progress performance results file '{pathToRestoreFrom}'.\n{e}");
}
}
static bool WrittenResultsBelongToCurrentRun()
{
var runToken = SessionState.GetString(k_RunTokenKey, string.Empty);
return !string.IsNullOrEmpty(runToken)
&& runToken == SessionState.GetString(k_WrittenRunTokenKey, string.Empty);
}
static void CreateDirectoryIfNecessary(string filePath)
{
var directoryPath = Path.GetDirectoryName(filePath);
if (!string.IsNullOrEmpty(directoryPath) && !Directory.Exists(directoryPath))
{
Directory.CreateDirectory(directoryPath);
}
}
}
}
@@ -0,0 +1,2 @@
fileFormatVersion: 2
guid: 8ec0cb4cdd3a9ae48b3a962d84c0dbe8
@@ -0,0 +1,24 @@
using System.Linq;
using UnityEditor;
using UnityEditor.Build;
namespace Unity.PerformanceTesting.Editor
{
internal class PerformanceTestBuildAssemblyFilter : IFilterBuildAssemblies
{
private const string unityTestRunnerAssemblyName = "Unity.PerformanceTesting";
public int callbackOrder { get; }
public string[] OnFilterAssemblies(BuildOptions buildOptions, string[] assemblies)
{
if ((buildOptions & BuildOptions.IncludeTestAssemblies) == BuildOptions.IncludeTestAssemblies)
{
return assemblies;
}
return assemblies.Where(x => !x.Contains(unityTestRunnerAssemblyName))
.ToArray();
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 6b7e8a2b3c6739c44a35905b9331e9b4
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,94 @@
using System;
using System.IO;
using Unity.PerformanceTesting.Runtime;
using UnityEngine;
using UnityEditor.TestTools.TestRunner.Api;
using UnityEditor.TestTools.TestRunner.CommandLineTest;
namespace Unity.PerformanceTesting.Editor
{
[Serializable]
internal class PerformanceTestRunSaver : ScriptableObject, ICallbacks
{
internal const string ResultsFileName = "PerformanceTestResults.json";
// Directory of the NUnit results file requested with -testResults. Results are saved there
// as well, because that is the directory a command line run collects as its artifacts.
[SerializeField]
string testResultsDirectory;
IncrementalRunResultsWriter m_IncrementalWriter;
internal static string DefaultResultsPath
{
get { return Path.Combine(Application.persistentDataPath, ResultsFileName); }
}
internal string ResultsPath
{
get
{
return string.IsNullOrEmpty(testResultsDirectory)
? DefaultResultsPath
: Path.Combine(testResultsDirectory, ResultsFileName);
}
}
IncrementalRunResultsWriter IncrementalWriter
{
get
{
if (m_IncrementalWriter == null)
{
m_IncrementalWriter = new IncrementalRunResultsWriter();
}
return m_IncrementalWriter;
}
}
internal void SetTestResultsDirectory(string directoryPath)
{
testResultsDirectory = directoryPath;
}
void ICallbacks.RunStarted(ITestAdaptor testsToRun)
{
PerformanceTest.Active = null;
IncrementalWriter.BeginRun();
}
void ICallbacks.RunFinished(ITestResultAdaptor result)
{
PlayerCallbacks.Saved = false;
try
{
var resultWriter = new ResultsWriter();
var xmlPath = Path.Combine(Application.persistentDataPath, "TestResults.xml");
resultWriter.WriteResultToFile(result, xmlPath);
var xmlParser = new TestResultXmlParser();
var run = xmlParser.GetPerformanceTestRunFromXml(xmlPath);
if (run == null) return;
var resultsPath = ResultsPath;
IncrementalRunResultsWriter.WriteResultsFile(resultsPath, run);
if (resultsPath != DefaultResultsPath)
{
IncrementalRunResultsWriter.WriteResultsFile(DefaultResultsPath, run);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message + "\n" + e.InnerException);
}
}
void ICallbacks.TestStarted(ITestAdaptor test) { }
void ICallbacks.TestFinished(ITestResultAdaptor result)
{
IncrementalWriter.AppendTestResults(ResultsPath, result.Output);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 62eec8a012ecc7c4a8b5a38b5dc130e4
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 30bb01dc09cb89b4faf344490fee76a7
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,11 @@
namespace Unity.PerformanceTesting.Editor
{
internal struct SampleGroupAdditionalData
{
public float min;
public float lowerQuartile;
public float median;
public float upperQuartile;
public float max;
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: c6efe3fc0e07c46468863dfd0a15978d
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,21 @@
using System;
namespace Unity.PerformanceTesting.Editor
{
internal struct SamplePoint : IComparable<SamplePoint>
{
public double sample;
public int index;
public SamplePoint(double _sample, int _index)
{
sample = _sample;
index = _index;
}
public int CompareTo(SamplePoint other)
{
return sample.CompareTo(other.sample);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: dd6eec692697e8f4bbd715f4d546a329
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,457 @@
using System;
using System.Collections.Generic;
using System.Linq;
using UnityEditor;
using UnityEditor.IMGUI.Controls;
using UnityEngine;
using UnityEngine.Assertions;
using Unity.PerformanceTesting.Data;
#if UNITY_6000_2_OR_NEWER
using TreeView = UnityEditor.IMGUI.Controls.TreeView<int>;
using TreeViewItem = UnityEditor.IMGUI.Controls.TreeViewItem<int>;
using TreeViewState = UnityEditor.IMGUI.Controls.TreeViewState<int>;
#endif
namespace Unity.PerformanceTesting.Editor
{
internal class TestListTableItem : TreeViewItem
{
public PerformanceTestResult performanceTest;
public double deviation;
public double standardDeviation;
public double median;
public double min;
public double max;
public TestListTableItem(int id, int depth, string displayName, PerformanceTestResult performanceTest)
: base(id, depth,
displayName)
{
this.performanceTest = performanceTest;
deviation = 0f;
if (performanceTest != null)
{
foreach (var sampleGroup in performanceTest.SampleGroups)
{
var thisDeviation =
sampleGroup.Median == 0 ? 0 : sampleGroup.StandardDeviation / sampleGroup.Median;
if (sampleGroup.Name == "Time" || sampleGroup.Samples.Count <= 1 || thisDeviation > deviation)
{
standardDeviation = sampleGroup.StandardDeviation;
median = sampleGroup.Median;
min = sampleGroup.Min;
max = sampleGroup.Max;
deviation = thisDeviation;
break;
}
}
}
}
}
internal class TestListTable : TreeView
{
TestReportWindow m_testReportWindow;
const float kRowHeights = 20f;
readonly List<TreeViewItem> m_Rows = new List<TreeViewItem>(100);
// All columns
public enum MyColumns
{
Name,
SampleCount,
StandardDeviation,
Deviation,
Median,
Min,
Max,
}
public enum SortOption
{
Name,
SampleCount,
StandardDeviation,
Deviation,
Median,
Min,
Max,
}
// Sort options per column
SortOption[] m_SortOptions =
{
SortOption.Name,
SortOption.SampleCount,
SortOption.StandardDeviation,
SortOption.Deviation,
SortOption.Median,
SortOption.Min,
SortOption.Max,
};
public TestListTable(TreeViewState state, MultiColumnHeader multicolumnHeader,
TestReportWindow testReportWindow)
: base(state, multicolumnHeader)
{
m_testReportWindow = testReportWindow;
Assert.AreEqual(m_SortOptions.Length, Enum.GetValues(typeof(MyColumns)).Length,
"Ensure number of sort options are in sync with number of MyColumns enum values");
// Custom setup
rowHeight = kRowHeights;
showAlternatingRowBackgrounds = true;
showBorder = true;
customFoldoutYOffset =
(kRowHeights - EditorGUIUtility.singleLineHeight) *
0.5f; // center foldout in the row since we also center content. See RowGUI
// extraSpaceBeforeIconAndLabel = 0;
multicolumnHeader.sortingChanged += OnSortingChanged;
Reload();
}
protected override TreeViewItem BuildRoot()
{
var root = new TestListTableItem(-1, -1, "root", null);
var results = m_testReportWindow.GetResults();
var searchText = m_testReportWindow.searchString?.ToLower();
var methodItemId = 0;
if (results == null) return root;
var classItemId = results.Results.Count(); // Class ID will always be bigger than the count of results (from methods) we get
var classGroups = results.Results.GroupBy(r => r.ClassName);
foreach (var classGroup in classGroups)
{
var classItem = new TestListTableItem(classItemId++, 0, classGroup.Key, null);
root.AddChild(classItem);
// Create method items and add them to the class item if they match the search text
// Currently Name provides both MethodName and Test parameter - example would be - ValueSource(Cube)
// We are using Name and simply removing the ClassPart as we need to keep that parameter
var methodItems = classGroup
.Select(r =>
{
// Calculate the starting index for the substring to extract methodname from full name
int startIndex = r.Name.IndexOf(classGroup.Key, StringComparison.Ordinal) + classGroup.Key.Length + 1;
// Check if the calculated startIndex is within the bounds of the string
// If it's within bounds, extract the substring; otherwise, return full name
string methodName = startIndex < r.Name.Length ? r.Name.Substring(startIndex) : r.Name;
// Create a TestListTableItem using the extracted methodName and other parameters
return new TestListTableItem(methodItemId++, 1, methodName, r);
});
foreach (var methodItem in methodItems)
{
var matchesSearchText = string.IsNullOrEmpty(searchText) ||
methodItem.displayName.ToLower().Contains(searchText);
if (matchesSearchText) classItem.AddChild(methodItem);
}
}
// Remove class items with no children
root.children?.RemoveAll(c => !c.hasChildren);
return root;
}
protected override IList<TreeViewItem> BuildRows(TreeViewItem root)
{
m_Rows.Clear();
if (rootItem != null && rootItem.children != null)
{
m_Rows.AddRange(base.BuildRows(root));
}
SortIfNeeded(m_Rows);
return m_Rows;
}
void OnSortingChanged(MultiColumnHeader _multiColumnHeader)
{
SortIfNeeded(GetRows());
}
void SortIfNeeded(IList<TreeViewItem> rows)
{
if (rows.Count <= 1)
{
return;
}
if (multiColumnHeader.sortedColumnIndex == -1)
{
return; // No column to sort for (just use the order the data are in)
}
// Sort the roots of the existing tree items
SortByMultipleColumns();
// Update the data with the sorted content
rows.Clear();
AddExpandedRows(rootItem, rows);
Repaint();
}
void SortByMultipleColumns()
{
var sortedColumns = multiColumnHeader.state.sortedColumns;
if (sortedColumns.Length == 0)
{
return;
}
var classItems = rootItem.children;
foreach (var classItem in classItems)
{
var myTypes = classItem.children.Cast<TestListTableItem>();
var orderedQuery = InitialOrder(myTypes, sortedColumns);
for (int i = 1; i < sortedColumns.Length; i++)
{
SortOption sortOption = m_SortOptions[sortedColumns[i]];
bool ascending = multiColumnHeader.IsSortedAscending(sortedColumns[i]);
switch (sortOption)
{
case SortOption.Name:
orderedQuery = orderedQuery.ThenBy(l => l.displayName, ascending);
break;
case SortOption.SampleCount:
orderedQuery = orderedQuery.ThenBy(l => l.performanceTest.SampleGroups.Count, ascending);
break;
case SortOption.Deviation:
orderedQuery = orderedQuery.ThenBy(l => l.deviation, ascending);
break;
case SortOption.StandardDeviation:
orderedQuery = orderedQuery.ThenBy(l => l.standardDeviation, ascending);
break;
case SortOption.Median:
orderedQuery = orderedQuery.ThenBy(l => l.median, ascending);
break;
case SortOption.Min:
orderedQuery = orderedQuery.ThenBy(l => l.min, ascending);
break;
case SortOption.Max:
orderedQuery = orderedQuery.ThenBy(l => l.max, ascending);
break;
}
}
classItem.children = orderedQuery.Cast<TreeViewItem>().ToList();
}
}
IOrderedEnumerable<TestListTableItem> InitialOrder(IEnumerable<TestListTableItem> myTypes, int[] history)
{
SortOption sortOption = m_SortOptions[history[0]];
bool ascending = multiColumnHeader.IsSortedAscending(history[0]);
switch (sortOption)
{
case SortOption.Name:
return myTypes.Order(l => l.displayName, ascending);
case SortOption.SampleCount:
return myTypes.Order(l => l.performanceTest.SampleGroups.Count, ascending);
case SortOption.Deviation:
return myTypes.Order(l => l.deviation, ascending);
case SortOption.StandardDeviation:
return myTypes.Order(l => l.standardDeviation, ascending);
case SortOption.Median:
return myTypes.Order(l => l.median, ascending);
case SortOption.Min:
return myTypes.Order(l => l.min, ascending);
case SortOption.Max:
return myTypes.Order(l => l.max, ascending);
default:
Assert.IsTrue(false, "Unhandled enum");
break;
}
// default
return myTypes.Order(l => l.displayName, ascending);
}
protected override void RowGUI(RowGUIArgs args)
{
var item = (TestListTableItem)args.item;
var nameRect = args.GetCellRect(0);
nameRect.x += GetContentIndent(item);
nameRect.xMax -= GetContentIndent(item);
if (item.depth == 0)
{
EditorGUI.LabelField(nameRect, item.displayName);
}
if (item.depth == 1)
{
for (var i = 0; i < args.GetNumVisibleColumns(); ++i)
{
var column = (MyColumns)args.GetColumn(i);
if (column == MyColumns.Name)
{
EditorGUI.LabelField(nameRect, item.displayName);
}
else
{
CellGUI(args.GetCellRect(i), item, column);
}
}
}
}
void CellGUI(Rect cellRect, TestListTableItem item, MyColumns column)
{
// Center cell rect vertically (makes it easier to place controls, icons etc in the cells)
CenterRectUsingSingleLineHeight(ref cellRect);
switch (column)
{
case MyColumns.SampleCount:
EditorGUI.LabelField(cellRect, $"{item.performanceTest.SampleGroups.Count}");
break;
case MyColumns.Deviation:
EditorGUI.LabelField(cellRect, $"{item.deviation:f2}");
break;
case MyColumns.StandardDeviation:
EditorGUI.LabelField(cellRect, $"{item.standardDeviation:f2}");
break;
case MyColumns.Median:
EditorGUI.LabelField(cellRect, $"{item.median:f2}");
break;
case MyColumns.Min:
EditorGUI.LabelField(cellRect, $"{item.min:f2}");
break;
case MyColumns.Max:
EditorGUI.LabelField(cellRect, $"{item.max:f2}");
break;
}
}
// Misc
//--------
protected override bool CanMultiSelect(TreeViewItem item)
{
return false;
}
struct HeaderData
{
public readonly GUIContent content;
public readonly float width;
public readonly float minWidth;
public readonly bool autoResize;
public readonly bool allowToggleVisibility;
public readonly bool ascending;
public HeaderData(string name, string tooltip = "", float width = 100, float minWidth = 50, bool autoResize = true, bool allowToggleVisibility = true, bool ascending = false)
{
content = new GUIContent(name, tooltip);
this.width = width;
this.minWidth = minWidth;
this.autoResize = autoResize;
this.allowToggleVisibility = allowToggleVisibility;
this.ascending = ascending;
}
}
public static MultiColumnHeaderState CreateDefaultMultiColumnHeaderState(float treeViewWidth)
{
var columnList = new List<MultiColumnHeaderState.Column>();
HeaderData[] headerData = new HeaderData[]
{
new HeaderData("Name", "Name of test", width : 500, minWidth : 100, autoResize : false, allowToggleVisibility : false, ascending : true),
new HeaderData("Groups", "Number of Sample Groups", width : 60, minWidth : 50),
new HeaderData("SD", "Standard Deviation"), // (of sample group with largest deviation)
new HeaderData("Deviation", "Standard Deviation / Median"),
new HeaderData("Median", "Median value"),
new HeaderData("Min", "Min value"),
new HeaderData("Max", "Max value"),
};
foreach (var header in headerData)
{
columnList.Add(new MultiColumnHeaderState.Column
{
headerContent = header.content,
headerTextAlignment = TextAlignment.Left,
sortedAscending = header.ascending,
sortingArrowAlignment = TextAlignment.Left,
width = header.width,
minWidth = header.minWidth,
autoResize = header.autoResize,
allowToggleVisibility = header.allowToggleVisibility
});
};
var columns = columnList.ToArray();
Assert.AreEqual(columns.Length, Enum.GetValues(typeof(MyColumns)).Length,
"Number of columns should match number of enum values: You probably forgot to update one of them.");
var state = new MultiColumnHeaderState(columns);
state.visibleColumns = new int[]
{
(int)MyColumns.Name,
(int)MyColumns.SampleCount,
(int)MyColumns.Deviation,
(int)MyColumns.StandardDeviation,
(int)MyColumns.Median,
(int)MyColumns.Min,
(int)MyColumns.Max,
};
return state;
}
protected override void SelectionChanged(IList<int> selectedIds)
{
base.SelectionChanged(selectedIds);
if (selectedIds.Count > 0)
m_testReportWindow.SelectTest(selectedIds[0]);
}
}
internal static class MyExtensionMethods
{
public static IOrderedEnumerable<T> Order<T, TKey>(this IEnumerable<T> source, Func<T, TKey> selector,
bool ascending)
{
if (ascending)
{
return source.OrderBy(selector);
}
else
{
return source.OrderByDescending(selector);
}
}
public static IOrderedEnumerable<T> ThenBy<T, TKey>(this IOrderedEnumerable<T> source, Func<T, TKey> selector,
bool ascending)
{
if (ascending)
{
return source.ThenBy(selector);
}
else
{
return source.ThenByDescending(selector);
}
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 242480b4ab9580c44b35d75dfa073192
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,47 @@
Shader "Unlit/TestReportShader"
{
Properties
{
}
SubShader
{
Tags { "RenderType"="Transparent" }
LOD 100
ZWrite Off
Blend SrcAlpha OneMinusSrcAlpha
Pass
{
CGPROGRAM
#pragma target 2.0
#pragma vertex vert
#pragma fragment frag
#include "UnityCG.cginc"
struct appdata
{
float4 vertex : POSITION;
fixed4 color : COLOR;
};
struct v2f
{
float4 vertex : SV_POSITION;
fixed4 color : COLOR;
};
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
o.color.rgba = v.color;
return o;
}
fixed4 frag (v2f i) : SV_Target { return i.color; }
ENDCG
}
}
}
@@ -0,0 +1,9 @@
fileFormatVersion: 2
guid: c7715b638c6f5c849a6ec12234ae264c
ShaderImporter:
externalObjects: {}
defaultTextures: []
nonModifiableTextures: []
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,913 @@
using System.Collections.Generic;
using UnityEngine;
using UnityEditor;
using UnityEditorInternal;
using System;
using UnityEditor.IMGUI.Controls;
using System.IO;
using System.Linq;
using System.Text;
using Unity.PerformanceTesting.Data;
using Unity.PerformanceTesting.Editor.UIElements;
using Unity.PerformanceTesting.Runtime;
#if UNITY_6000_2_OR_NEWER
using TreeViewState = UnityEditor.IMGUI.Controls.TreeViewState<int>;
#endif
namespace Unity.PerformanceTesting.Editor
{
internal class TestReportWindow : EditorWindow
{
static int s_windowWidth = 800;
static int s_windowHeight = 600;
private GUIStyle m_glStyle = null;
private GUIStyle m_boldStyle = null;
private Material m_material;
public Color m_colorWhite = new Color(1.0f, 1.0f, 1.0f);
public Color m_colorBarBackground = new Color(0.5f, 0.5f, 0.5f);
public Color m_colorBoxAndWhiskerBackground = new Color(0.4f, 0.4f, 0.4f);
public Color m_colorBar = new Color(0.95f, 0.95f, 0.95f);
public Color m_colorStandardLine = new Color(1.0f, 1.0f, 1.0f);
public Color m_colorMedianLine = new Color(0.2f, 0.5f, 1.0f, 0.5f);
public Color m_colorMedianText = new Color(0.4f, 0.7f, 1.0f, 1.0f);
public Color m_colorWarningText = Color.red;
public Color m_toolbarSeparator = new Color (0.15f, 0.15f, 0.15f, 1.0f);
private Run m_resultsData = null;
private string m_selectedTest;
private DateTime m_lastResultsDateTime = new DateTime(0);
private bool m_NewerFileExists = false;
private double m_LastFileCheckTime = 0f;
private float m_FileCheckFrequencySeconds = 3f;
private bool m_AutoRefresh;
private List<string> m_sampleGroups = new List<string>();
//private int m_selectedSampleGroupIndex;
private bool m_showTests = true;
private bool m_showSamples = true;
private int[] m_columnWidth = new int[4];
private bool m_isResizing;
private float m_testListHeight = (float)s_windowHeight / 4;
private Rect m_splitterRect;
private float m_windowHeight;
[SerializeField]
TreeViewState m_testListTreeViewState;
[SerializeField]
MultiColumnHeaderState m_testListMulticolumnHeaderState;
TestListTable m_testListTable;
Vector2 m_sampleGroupScroll = new Vector2(0, 0);
private readonly Dictionary<Tuple<string, string>, SampleGroupAdditionalData> m_sampleGroupAdditionalData = new Dictionary<Tuple<string, string>, SampleGroupAdditionalData>();
private bool m_isResultsCountPositive;
private ToolbarWithSearch m_toolbarWithSearch;
public string searchString;
private const int m_chartLimit = 1000;
private void CreateTestListTable()
{
if (m_testListTreeViewState == null)
m_testListTreeViewState = new TreeViewState();
//if (m_profileMulticolumnHeaderState==null)
m_testListMulticolumnHeaderState = TestListTable.CreateDefaultMultiColumnHeaderState(700);
var multiColumnHeader = new MultiColumnHeader(m_testListMulticolumnHeaderState);
multiColumnHeader.SetSorting((int)TestListTable.MyColumns.Name, true);
multiColumnHeader.ResizeToFit();
m_testListTable = new TestListTable(m_testListTreeViewState, multiColumnHeader, this);
m_testListTable.ExpandAll();
}
public Run GetResults()
{
return m_resultsData;
}
public void SelectTest(int index)
{
if (index < 0 || index >= m_resultsData.Results.Count)
{
m_sampleGroups.Clear();
return;
}
var result = m_resultsData.Results[index];
SelectTest(result);
}
public void SelectTest(string name)
{
foreach (var result in m_resultsData.Results)
{
if (result.Name == name)
{
SelectTest(result);
return;
}
}
}
public void SelectTest(PerformanceTestResult result)
{
m_selectedTest = result.Name;
m_sampleGroups.Clear();
foreach (var sampleGroup in result.SampleGroups)
{
m_sampleGroups.Add(sampleGroup.Name);
}
}
[MenuItem("Window/General/Performance Test Report", false, priority = 202)]
private static void Init()
{
var window = GetWindow<TestReportWindow>("Test Report");
window.minSize = new Vector2(640, 480);
window.position.size.Set(s_windowWidth, s_windowHeight);
window.Show();
}
public void SetupMaterial()
{
// Create a new material
m_material = new Material(Shader.Find("Unlit/TestReportShader"));
m_material.SetPass(0);
}
private string GetResultsPath()
{
return Path.Combine(Application.persistentDataPath, "PerformanceTestResults.json");
}
private bool NewerFileExists()
{
m_LastFileCheckTime = EditorApplication.timeSinceStartup;
string filePath = GetResultsPath();
if (!File.Exists(filePath))
return false;
if (m_resultsData == null)
return true;
DateTime dateTime = File.GetLastWriteTime(GetResultsPath());
if ((dateTime - m_lastResultsDateTime).TotalMilliseconds > 0)
return true;
return false;
}
private void ResetFileCheck()
{
m_lastResultsDateTime = File.GetLastWriteTime(GetResultsPath());
m_NewerFileExists = false;
m_LastFileCheckTime = EditorApplication.timeSinceStartup;
}
private bool CheckIfNewerFileExists()
{
if (EditorApplication.timeSinceStartup - m_LastFileCheckTime < m_FileCheckFrequencySeconds)
return m_NewerFileExists;
m_NewerFileExists = NewerFileExists();
return m_NewerFileExists;
}
private void LoadData()
{
m_toolbarWithSearch?.ClearSearchString();
string filePath = GetResultsPath();
if (!File.Exists(filePath)) return;
string json = File.ReadAllText(filePath);
m_resultsData = JsonUtility.FromJson<Run>(json);
ResetFileCheck();
List<SamplePoint> samplePoints = new List<SamplePoint>();
m_sampleGroupAdditionalData.Clear();
foreach (var result in m_resultsData.Results)
{
foreach (var sampleGroup in result.SampleGroups)
{
samplePoints.Clear();
for (int index = 0; index < sampleGroup.Samples.Count; index++)
{
var sample = sampleGroup.Samples[index];
samplePoints.Add(new SamplePoint(sample, index));
}
samplePoints.Sort();
int discard;
SampleGroupAdditionalData data = new SampleGroupAdditionalData();
data.min = (float)GetPercentageOffset(samplePoints, 0, out discard);
data.lowerQuartile = (float)GetPercentageOffset(samplePoints, 25, out discard);
data.median = (float)sampleGroup.Median;
data.upperQuartile = (float)GetPercentageOffset(samplePoints, 75, out discard);
data.max = (float)GetPercentageOffset(samplePoints, 100, out discard);
var key = Tuple.Create(result.Name, sampleGroup.Name);
m_sampleGroupAdditionalData[key] = data;
}
}
CreateTestListTable();
}
private void OnEnable()
{
SetupMaterial();
LoadData();
}
private void OnFocus()
{
CheckIfNewerFileExists();
}
private void OnGUI()
{
if (m_glStyle == null)
{
m_glStyle = new GUIStyle(GUI.skin.box);
m_glStyle.padding = new RectOffset(0, 0, 0, 0);
m_glStyle.margin = new RectOffset(0, 0, 0, 0);
}
if (m_boldStyle == null)
{
m_boldStyle = new GUIStyle(GUI.skin.label);
m_boldStyle.fontStyle = FontStyle.Bold;
}
if (m_toolbarWithSearch == null)
{
m_toolbarWithSearch = new ToolbarWithSearch();
m_toolbarWithSearch.SearchTextChanged += s =>
{
searchString = s;
CreateTestListTable();
};
}
m_isResultsCountPositive = m_resultsData?.Results.Count > 0;
DrawToolbar();
if (!DrawPerformanceDataStatusLabel()) return;
m_toolbarWithSearch.Draw();
DrawTestView();
DrawSampleView();
}
private void Update()
{
CheckIfNewerFileExists();
if (m_NewerFileExists && m_AutoRefresh)
Refresh();
}
private double GetPercentageOffset(List<SamplePoint> samplePoint, float percent, out int outputFrameIndex)
{
// A sample group can be reported without any samples, so there is no frame to point at.
if (samplePoint.Count == 0)
{
outputFrameIndex = -1;
return 0;
}
int index = (int)((samplePoint.Count - 1) * percent / 100);
outputFrameIndex = samplePoint[index].index;
// True median is half of the sum of the middle 2 frames for an even count. However this would be a value never recorded so we avoid that.
return samplePoint[index].sample;
}
private bool BoldFoldout(bool toggle, string text)
{
GUIStyle foldoutStyle = new GUIStyle(EditorStyles.foldout);
foldoutStyle.fontStyle = FontStyle.Bold;
return EditorGUILayout.Foldout(toggle, text, foldoutStyle);
/*
EditorGUILayout.LabelField(text, EditorStyles.boldLabel, GUILayout.Width(100));
return true;
*/
}
private void SetColumnSizes(int a, int b, int c, int d)
{
m_columnWidth[0] = a;
m_columnWidth[1] = b;
m_columnWidth[2] = c;
m_columnWidth[3] = d;
}
private void DrawColumn(int n, string col)
{
if (m_columnWidth[n] > 0)
EditorGUILayout.LabelField(col, GUILayout.Width(m_columnWidth[n]));
else
EditorGUILayout.LabelField(col);
}
private void DrawColumn(int n, float value)
{
DrawColumn(n, string.Format("{0:f2}", value));
}
private void Draw2Column(string label, float value)
{
EditorGUILayout.BeginHorizontal();
DrawColumn(0, label);
DrawColumn(1, value);
EditorGUILayout.EndHorizontal();
}
private void Refresh()
{
LoadData();
CreateTestListTable();
if (m_resultsData == null) return;
if (m_resultsData.Results == null) return;
if (m_resultsData.Results.Any(result => result.Name == m_selectedTest))
SelectTest(m_selectedTest);
else
SelectTest(0);
Repaint();
}
private void Export()
{
if (m_resultsData == null || m_resultsData.Results.Count <= 0) return;
var path = EditorUtility.SaveFilePanel("Save CSV data", "", "PerformanceTestResults.csv", "csv");
if (string.IsNullOrEmpty(path)) return;
var sb = new StringBuilder();
const string header =
"Index,Test Name,Version,Sample Group Name,Unit,Increase Is Better,Min,Max,Median,Average,Standard Deviation,Sum,Values\n";
sb.Append(header);
var index = 0;
foreach (var result in m_resultsData.Results)
foreach (var sampleGroup in result.SampleGroups)
{
var sampleValues = string.Join(", ", sampleGroup.Samples);
var increaseIsBetter = sampleGroup.IncreaseIsBetter ? "Yes" : "No";
var line =
$"{index},\"{result.Name}\",{result.Version},\"{sampleGroup.Name}\",\"{sampleGroup.Unit}\",{increaseIsBetter},{sampleGroup.Min},{sampleGroup.Max},{sampleGroup.Median},{sampleGroup.Average},{sampleGroup.StandardDeviation},{sampleGroup.Sum},{sampleValues}\n";
sb.Append(line);
index++;
}
File.WriteAllText(path, sb.ToString());
}
private void ClearResults()
{
File.Delete(GetResultsPath());
m_resultsData = null;
GUILayout.Label(string.Empty);
}
private void DrawToolbar()
{
GUILayout.BeginHorizontal();
m_AutoRefresh = EditorPrefs.GetBool("ToggleState", false);
m_AutoRefresh = GUILayout.Toggle(m_AutoRefresh, "Auto Refresh");
EditorPrefs.SetBool("ToggleState", m_AutoRefresh);
if (GUILayout.Button("Refresh", EditorStyles.toolbarButton)) Refresh();
GUILayout.FlexibleSpace();
if (m_isResultsCountPositive) GUILayout.Label($"Last results {m_lastResultsDateTime}");
GUI.enabled = m_isResultsCountPositive;
if (GUILayout.Button("Clear Results", EditorStyles.toolbarButton))
{
if (EditorUtility.DisplayDialog("Clear Results",
"Are you sure you want to clear all Performance test results?", "Clear", "Cancel"))
{
ClearResults();
}
}
if (GUILayout.Button("Export", EditorStyles.toolbarButton)) Export();
GUI.enabled = true;
DrawHelpIcon();
GUILayout.EndHorizontal();
EditorGUI.DrawRect(new Rect(0, GUILayoutUtility.GetLastRect().yMax, position.width, 1), m_toolbarSeparator);
EditorGUILayout.Space();
}
private static void DrawHelpIcon()
{
if (!GUILayout.Button(EditorGUIUtility.TrIconContent("_Help", "Open Documentation"),
EditorStyles.toolbarButton)) return;
var packageInfo = UnityEditor.PackageManager.PackageInfo.FindForAssetPath("Packages/com.unity.test-framework.performance");
var shortVersion = packageInfo.version.Substring(0,
packageInfo.version.IndexOf('.', packageInfo.version.IndexOf('.') + 1));
var documentationUrl = $"https://docs.unity3d.com/Packages/{packageInfo.name}@{shortVersion}";
Help.ShowHelpPage(documentationUrl);
}
private bool DrawPerformanceDataStatusLabel()
{
var isDataMissing = m_resultsData == null || !m_isResultsCountPositive;
var shouldRefreshData = m_NewerFileExists && !m_AutoRefresh;
if (isDataMissing || shouldRefreshData)
{
var content = new GUIContent
{
image = EditorGUIUtility.IconContent("console.infoicon").image,
text = shouldRefreshData
? "New performance <b>data is available</b>. Click <b>\"Refresh\"</b> or <b>\"Auto Refresh\"</b> to view it"
: "No performance data found"
};
var style = new GUIStyle(GUI.skin.GetStyle("HelpBox"))
{
richText = true,
alignment = TextAnchor.MiddleLeft
};
GUILayout.Label(content, style);
if (isDataMissing) return false;
}
return true;
}
private void DrawTestView()
{
m_showTests = BoldFoldout(m_showTests, "Test View");
if (m_showTests)
{
if (m_testListTable != null)
{
var r = GUILayoutUtility.GetRect(position.width, m_testListHeight, GUI.skin.box,
GUILayout.ExpandWidth(true));
m_testListTable.OnGUI(r);
Resize(r.y);
}
}
if (!string.IsNullOrEmpty(m_selectedTest))
{
var profileDataFile = Path.Combine(Application.persistentDataPath,
Utils.RemoveIllegalCharacters(m_selectedTest) + ".raw");
if (File.Exists(profileDataFile))
{
if (GUILayout.Button($"Load profiler data for test: {m_selectedTest}"))
{
ProfilerDriver.LoadProfile(profileDataFile, false);
}
}
}
}
private void DrawSampleView()
{
m_showSamples = BoldFoldout(m_showSamples, "Sample Group View");
if (m_showSamples)
{
if (m_sampleGroups?.Count <= 0)
{
EditorGUILayout.LabelField("This Selection Has No Sample Groups", m_boldStyle);
return;
}
SetColumnSizes(50, 50, 50, 50);
var boxStyle = GUI.skin.box;
var boxWidth = position.width - GUI.skin.verticalScrollbar.fixedWidth -
(boxStyle.padding.horizontal + boxStyle.margin.horizontal);
var graphWidth = position.width - 200;
EditorGUILayout.BeginVertical();
m_sampleGroupScroll = EditorGUILayout.BeginScrollView(m_sampleGroupScroll, false, true);
foreach (var result in m_resultsData.Results)
{
if (result.Name != m_selectedTest) continue;
EditorGUILayout.LabelField(result.Name, m_boldStyle);
foreach (var sampleGroup in result.SampleGroups)
{
var key = Tuple.Create(result.Name, sampleGroup.Name);
if (m_sampleGroupAdditionalData.TryGetValue(key, out var data))
{
var samplesToDraw = sampleGroup.Samples;
var min = data.min;
var lowerQuartile = data.lowerQuartile;
var median = data.median;
var upperQuartile = data.upperQuartile;
var max = data.max;
var graphMin = min > 0.0f ? 0.0f : min;
EditorGUILayout.BeginVertical(boxStyle, GUILayout.Width(boxWidth),
GUILayout.ExpandHeight(false));
EditorGUILayout.LabelField(sampleGroup.Name, m_boldStyle);
EditorGUILayout.LabelField($"Sample Unit: {sampleGroup.Unit}");
if (samplesToDraw.Count > m_chartLimit)
{
EditorGUILayout.HelpBox(
$"Sample Group has more than {m_chartLimit} Samples. The first {m_chartLimit} Samples will be displayed. However, calculations are done according to all samples received from the test run.",
MessageType.Warning, true);
samplesToDraw = samplesToDraw.Take(m_chartLimit).ToList();
}
EditorGUILayout.BeginHorizontal(GUILayout.Height(100), GUILayout.ExpandHeight(false));
EditorGUILayout.BeginVertical(GUILayout.Width(100), GUILayout.ExpandHeight(true));
Draw2Column("Max", max);
GUILayout.FlexibleSpace();
var oldColor = GUI.contentColor;
GUI.contentColor = median < 0.01f ? m_colorWarningText : m_colorMedianText;
Draw2Column("Median", median);
GUI.contentColor = oldColor;
GUILayout.FlexibleSpace();
Draw2Column("Min", min);
EditorGUILayout.EndVertical();
DrawBarGraph(graphWidth, 100, samplesToDraw, graphMin, max, median);
DrawBoxAndWhiskerPlot(50, 100, min, lowerQuartile, median, upperQuartile, max, min, max,
(float)sampleGroup.StandardDeviation, m_colorWhite, m_colorBoxAndWhiskerBackground);
EditorGUILayout.EndHorizontal();
EditorGUILayout.EndVertical();
}
}
}
EditorGUILayout.EndScrollView();
EditorGUILayout.EndVertical();
}
}
private void Resize(float headerHeight)
{
m_splitterRect = new Rect(0, headerHeight + m_testListHeight, position.width, 5f);
EditorGUIUtility.AddCursorRect(m_splitterRect, MouseCursor.ResizeVertical);
if (Event.current.type == EventType.MouseDown)
m_isResizing = m_splitterRect.Contains(Event.current.mousePosition);
const float minListHeight = 80f;
var maxListHeight = position.height - 240f;
var needMoveSplitterAndRepaint = false;
if (m_isResizing && Event.current.type == EventType.MouseDrag)
{
m_testListHeight = Mathf.Clamp(Event.current.mousePosition.y - headerHeight, minListHeight, maxListHeight);
needMoveSplitterAndRepaint = true;
}
else if (Math.Abs(m_windowHeight - position.height) > float.Epsilon)
{
m_windowHeight = position.height;
m_testListHeight = Mathf.Clamp(m_testListHeight, minListHeight, maxListHeight);
needMoveSplitterAndRepaint = true;
}
if (needMoveSplitterAndRepaint)
{
m_splitterRect.Set(m_splitterRect.x, m_testListHeight, m_splitterRect.width, m_splitterRect.height);
Repaint();
}
if (Event.current.type == EventType.MouseUp)
m_isResizing = false;
}
private void DrawBarGraph(float width, float height, List<double> samples, float min, float max, float median)
{
if (DrawStart(width, height))
{
Rect rect = GUILayoutUtility.GetLastRect();
float clipH = Math.Max(m_sampleGroupScroll.y - rect.y, 0);
float maxH = Math.Max(height - clipH, 0);
int xAxisDivisions = samples.Count;
float xAxisInc = width / xAxisDivisions;
float yRange = max - min;
float x = 0;
float y = 0;
float spacing = 2;
float w = xAxisInc - spacing;
if (w < 1) spacing = 0;
float h = 0;
for (int i = 0; i < samples.Count; i++)
{
float sample = (float)samples[i];
w = xAxisInc - spacing;
h = ((sample - min) * height) / yRange;
if (h > maxH)
h = maxH;
DrawBar(x, y + (height - h), w, h, m_colorBar);
x += xAxisInc;
}
h = ((median - min) * height) / yRange;
if (h <= maxH)
{
//DrawLine(0, (height - h), width, (height - h), m_colorMedianLine);
DrawBar(0, (height - h), width, 3, m_colorMedianLine);
}
x = 0;
for (int i = 0; i < samples.Count; i++)
{
float sample = (float)samples[i];
string tooltip = string.Format("{0} (at sample {1} of {2})", sample, i+1, samples.Count);
GUI.Label(new Rect(rect.x + x, rect.y + y, xAxisInc, height), new GUIContent("", tooltip));
x += xAxisInc;
}
DrawEnd();
}
}
public bool DrawStart(Rect r)
{
if (Event.current.type != EventType.Repaint)
return false;
GL.PushMatrix();
SetupMaterial();
Matrix4x4 matrix = new Matrix4x4();
matrix.SetTRS(new Vector3(r.x, r.y, 0), Quaternion.identity, Vector3.one);
GL.MultMatrix(matrix);
return true;
}
public bool DrawStart(float w, float h, GUIStyle style = null)
{
Rect r = GUILayoutUtility.GetRect(w, h, style == null ? m_glStyle : style);
return DrawStart(r);
}
public void DrawEnd()
{
GL.PopMatrix();
}
public void DrawBar(float x, float y, float w, float h, Color col)
{
GL.Begin(GL.TRIANGLE_STRIP);
GL.Color(col);
GL.Vertex3(x, y, 0);
GL.Vertex3(x + w, y, 0);
GL.Vertex3(x, y + h, 0);
GL.Vertex3(x + w, y + h, 0);
GL.End();
}
void DrawBar(float x, float y, float w, float h, float r, float g, float b)
{
DrawBar(x, y, w, h, new Color(r, g, b));
}
void DrawLine(float x, float y, float x2, float y2, Color col)
{
GL.Begin(GL.LINES);
GL.Color(col);
GL.Vertex3(x, y, 0);
GL.Vertex3(x2, y2, 0);
GL.End();
}
void DrawLine(float x, float y, float x2, float y2, float r, float g, float b)
{
DrawLine(x, y, x2, y2, new Color(r, g, b));
}
void DrawBox(float x, float y, float w, float h, Color col)
{
GL.Begin(GL.LINE_STRIP);
GL.Color(col);
GL.Vertex3(x, y, 0);
GL.Vertex3(x + w, y, 0);
GL.Vertex3(x + w, y + h, 0);
GL.Vertex3(x, y + h, 0);
GL.Vertex3(x, y, 0);
GL.End();
}
void DrawBox(float x, float y, float w, float h, float r, float g, float b)
{
DrawBox(x, y, w, h, new Color(r, g, b));
}
private float ClampToRange(float value, float min, float max)
{
return Math.Max(min, Math.Min(value, max));
}
private void DrawHistogramStart(float width)
{
EditorGUILayout.BeginHorizontal(GUILayout.Width(width + 10));
//GUILayoutUtility.GetRect(GUI.skin.box.margin.left, 1);
EditorGUILayout.BeginVertical();
}
private void DrawHistogramEnd(float width, float min, float max, float spacing)
{
EditorGUILayout.BeginHorizontal();
float lastBar = width - 50;
GUIStyle rightAlignStyle = new GUIStyle(GUI.skin.label);
rightAlignStyle.alignment = TextAnchor.MiddleCenter;
EditorGUILayout.LabelField(string.Format("{0:f2}", min), GUILayout.Width(lastBar));
EditorGUILayout.LabelField(string.Format("{0:f2}", max), rightAlignStyle, GUILayout.Width(50));
EditorGUILayout.EndHorizontal();
EditorGUILayout.EndVertical();
EditorGUILayout.EndHorizontal();
}
private void DrawHistogramBackground(float width, float height, int bucketCount, float spacing)
{
float x = (spacing / 2);
float y = 0;
float w = ((width + spacing) / bucketCount) - spacing;
float h = height;
for (int i = 0; i < bucketCount; i++)
{
DrawBar(x, y, w, h, m_colorBarBackground);
x += w;
x += spacing;
}
}
private void DrawHistogramData(float width, float height, int[] buckets, int totalFrameCount, float min,
float max, Color barColor, float spacing)
{
float x = (spacing / 2);
float y = 0;
float w = ((width + spacing) / buckets.Length) - spacing;
float h = height;
Rect rect = GUILayoutUtility.GetLastRect();
int bucketCount = buckets.Length;
float bucketWidth = ((max - min) / bucketCount);
for (int bucketAt = 0; bucketAt < bucketCount; bucketAt++)
{
var count = buckets[bucketAt];
float barHeight = (h * count) / totalFrameCount;
if (barHeight > rect.height)
barHeight = rect.height;
DrawBar(x, y + (h - barHeight), w, barHeight, barColor);
float bucketStart = min + (bucketAt * bucketWidth);
float bucketEnd = bucketStart + bucketWidth;
GUI.Label(new Rect(rect.x + x, rect.y + y, w, h),
new GUIContent("", string.Format("{0:f2}-{1:f2}ms\n{2} frames", bucketStart, bucketEnd, count)));
x += w;
x += spacing;
}
}
private void DrawHistogram(float width, float height, int[] buckets, int totalFrameCount, float min, float max,
Color barColor)
{
DrawHistogramStart(width);
float spacing = 2;
if (DrawStart(width, height))
{
Rect rect = GUILayoutUtility.GetLastRect();
float clipH = Math.Max(m_sampleGroupScroll.y - rect.y, 0);
float maxH = Math.Max(height - clipH, 0);
DrawHistogramBackground(width, maxH, buckets.Length, spacing);
DrawHistogramData(width, height, buckets, totalFrameCount, min, max, barColor, spacing);
DrawEnd();
}
DrawHistogramEnd(width, min, max, spacing);
}
private void DrawBoxAndWhiskerPlot(float width, float height, float min, float lowerQuartile, float median,
float upperQuartile, float max, float yAxisStart, float yAxisEnd, float standardDeviation, Color color,
Color colorBackground)
{
if (DrawStart(width, height))
{
Rect rect = GUILayoutUtility.GetLastRect();
float clipH = Math.Max(m_sampleGroupScroll.y - rect.y, 0);
float maxH = Math.Max(height - clipH, 0);
if (maxH > 0)
{
float x = 0;
float y = 0;
float w = width;
float h = height;
DrawBoxAndWhiskerPlot(rect, x, y, w, h, min, lowerQuartile, median, upperQuartile, max, yAxisStart,
yAxisEnd, standardDeviation, color, colorBackground);
}
DrawEnd();
}
}
private void DrawBoxAndWhiskerPlot(Rect rect, float x, float y, float w, float h, float min,
float lowerQuartile, float median, float upperQuartile, float max, float yAxisStart, float yAxisEnd,
float standardDeviation, Color color, Color colorBackground)
{
string tooltip = string.Format(
"Max :\t\t{0:f2}\n\nUpper Quartile :\t{1:f2}\nMedian :\t\t{2:f2}\nLower Quartile :\t{3:f2}\n\nMin :\t\t{4:f2}\n\nStandard Deviation:\t{5:f2}",
max, upperQuartile, median, lowerQuartile, min,
standardDeviation);
GUI.Label(rect, new GUIContent("", tooltip));
float clipH = Math.Max(m_sampleGroupScroll.y - rect.y, 0);
float maxH = Math.Max(h - clipH, 0);
DrawBar(x, y + (h - maxH), w, maxH, m_colorBoxAndWhiskerBackground);
float first = yAxisStart;
float last = yAxisEnd;
float range = last - first;
bool startCap = (min >= first) ? true : false;
bool endCap = (max <= last) ? true : false;
// Range clamping
min = ClampToRange(min, first, last);
lowerQuartile = ClampToRange(lowerQuartile, first, last);
median = ClampToRange(median, first, last);
upperQuartile = ClampToRange(upperQuartile, first, last);
max = ClampToRange(max, first, last);
float yMin = h * (min - first) / range;
float yLowerQuartile = h * (lowerQuartile - first) / range;
float yMedian = h * (median - first) / range;
float yUpperQuartile = h * (upperQuartile - first) / range;
float yMax = h * (max - first) / range;
// Clamp to scroll area
yMin = Math.Min(yMin, maxH);
yLowerQuartile = Math.Min(yLowerQuartile, maxH);
yMedian = Math.Min(yMedian, maxH);
yUpperQuartile = Math.Min(yUpperQuartile, maxH);
yMax = Math.Min(yMax, maxH);
// Min to max line
//DrawLine(x + (w / 2), y + (h - yMin), x + (w / 2), y + (h - yMax), color);
DrawLine(x + (w / 2), y + (h - yMin), x + (w / 2), y + (h - yLowerQuartile), color);
DrawLine(x + (w / 2), y + (h - yUpperQuartile), x + (w / 2), y + (h - yMax), color);
// Quartile boxes
float xMargin = (2 * w / 8);
if (colorBackground != color)
DrawBar(x + xMargin, y + (h - yMedian), w - (2 * xMargin), (yMedian - yLowerQuartile), colorBackground);
DrawBox(x + xMargin, y + (h - yMedian), w - (2 * xMargin), (yMedian - yLowerQuartile), color);
if (colorBackground != color)
DrawBar(x + xMargin, y + (h - yUpperQuartile), w - (2 * xMargin), (yUpperQuartile - yMedian),
colorBackground);
DrawBox(x + xMargin, y + (h - yUpperQuartile), w - (2 * xMargin), (yUpperQuartile - yMedian), color);
// Median line
DrawLine(x + xMargin, y + (h - yMedian), x + (w - xMargin), y + (h - yMedian), color);
// Line caps
xMargin = (3 * w / 8);
if (startCap)
DrawLine(x + xMargin, y + (h - yMin), x + (w - xMargin), y + (h - yMin), color);
if (endCap)
DrawLine(x + xMargin, y + (h - yMax), x + (w - xMargin), y + (h - yMax), color);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 4272062d6dbd273438736c85848ea974
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,22 @@
using System;
using Unity.PerformanceTesting.Data;
using UnityEngine;
namespace Unity.PerformanceTesting.Editor
{
/// <summary>
/// Helper class to parse test runs into performance test runs.
/// </summary>
public class TestResultXmlParser
{
/// <summary>
/// Parses performance test run from test run result xml.
/// </summary>
/// <param name="resultXmlFileName">Path to test results xml file.</param>
/// <returns>Performance test run data extracted from the NUnit xml results file.</returns>
public Run GetPerformanceTestRunFromXml(string resultXmlFileName)
{
return TestResultsParser.GetPerformanceTestRunDataFromXmlFile(resultXmlFileName);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: acd1f5cc793f24d4ba37a3c34952bde0
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,249 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using System.Xml.Linq;
using Unity.PerformanceTesting.Data;
using UnityEditor.TestTools.TestRunner.Api;
using UnityEngine;
namespace Unity.PerformanceTesting.Editor
{
static class TestResultsParser
{
internal static Run GetPerformanceTestRunData(ITestResultAdaptor testResults)
{
var testOutputs = GetTestOutputsRecursively(testResults);
return ExtractPerformanceRunData(testOutputs);
}
internal static Run GetPerformanceTestRunDataFromXmlFile(string xmlResultsPath)
{
try
{
var xmlDocument = XDocument.Load(xmlResultsPath);
return GetPerformanceTestRunData(xmlDocument);
}
catch (Exception e)
{
Debug.LogWarning("Failed to load performance test results from XML.\n" +
$"{e.GetType()}: {e.Message}\n" +
$"{e.StackTrace}");
}
return null;
}
static Run GetPerformanceTestRunData(XDocument testResults)
{
var testOutputs = testResults.Descendants("output")
.Select(outputElement => outputElement.Value)
.ToArray();
return ExtractPerformanceRunData(testOutputs);
}
static string[] GetTestOutputsRecursively(ITestResultAdaptor testResults)
{
var res = new List<string>();
if (testResults != null)
{
AccumulateTestRunOutputRecursively(testResults, res);
}
return res.ToArray();
}
static void AccumulateTestRunOutputRecursively(ITestResultAdaptor parentResult, List<string> outputs)
{
var children = parentResult.Children;
foreach (var child in children)
{
AccumulateTestRunOutputRecursively(child, outputs);
}
var currentTestOutput = parentResult.Output;
if (!string.IsNullOrEmpty(currentTestOutput))
{
outputs.Add(currentTestOutput);
}
}
internal static Run ExtractPerformanceRunData(string[] testOutputs)
{
if (testOutputs == null || testOutputs.Length == 0)
{
return null;
}
try
{
var run = ExtractPerformanceTestRunInfo(testOutputs);
if (run != null)
{
DeserializeTestResults(testOutputs, run);
}
return run;
}
catch (FormatException fe)
{
Debug.LogError($"Invalid performance test results format: {fe.Message}");
}
catch (Exception e)
{
Debug.LogError($"Unexpected exception while reading performance test results: {e.Message}");
}
return null;
}
// Appends the results found in the output of a single finished test to an accumulating run,
// creating the run from the execution metadata when it has not been encountered yet.
internal static bool TryAppendPerformanceRunData(string testOutput, ref Run run)
{
if (string.IsNullOrEmpty(testOutput))
{
return false;
}
var testOutputs = new[] { testOutput };
try
{
if (run == null)
{
run = ExtractPerformanceTestRunInfo(testOutputs);
if (run == null)
{
return false;
}
}
var resultCountBeforeAppend = run.Results.Count;
DeserializeTestResults(testOutputs, run);
return run.Results.Count > resultCountBeforeAppend;
}
catch (FormatException fe)
{
Debug.LogError($"Invalid performance test results format: {fe.Message}");
}
catch (Exception e)
{
Debug.LogError($"Unexpected exception while reading performance test results: {e.Message}");
}
return false;
}
static Run ExtractPerformanceTestRunInfo(string[] testOutputs)
{
foreach (var output in testOutputs)
{
const string pattern = @"##performancetestruninfo2:(.+)\n";
var regex = new Regex(pattern);
var matches = regex.Match(output);
if (matches.Groups.Count == 0 || matches.Captures.Count == 0)
{
continue;
}
if (matches.Groups[1].Captures.Count > 1)
{
throw new FormatException("Multiple execution metadata instances found.");
}
var json = matches.Groups[1].Value;
if (string.IsNullOrEmpty(json))
{
throw new FormatException("No execution metadata found.");
}
return ReadPerformanceTestRunJsonObject(json);
}
return null;
}
static Run ReadPerformanceTestRunJsonObject(string json)
{
try
{
return JsonUtility.FromJson<Run>(json);
}
catch (Exception e)
{
throw new FormatException($"Failed to read performance execution metadata from json string: '{json}'.\n" +
$"Exception: {e.Message}\n" +
$"{e.StackTrace}");
}
}
static void DeserializeTestResults(string[] testOutputs, Run run)
{
foreach (var output in testOutputs)
{
foreach (var line in output.Split('\n'))
{
var json = GetJsonFromHashtag("performancetestresult2", line);
if (json == null)
{
continue;
}
var result = ReadPerformanceTestResultJsonObject(json);
if (result != null)
{
run.Results.Add(result);
}
}
}
}
static PerformanceTestResult ReadPerformanceTestResultJsonObject(string json)
{
try
{
return JsonUtility.FromJson<PerformanceTestResult>(json);
}
catch (Exception e)
{
Debug.LogWarning($"Failed to read performance results from json string: '{json}'.\n" +
$"Exception: {e.Message}\n" +
$"{e.StackTrace}");
}
return null;
}
static string GetJsonFromHashtag(string tag, string line)
{
if (!line.Contains($"##{tag}:"))
{
return null;
}
var jsonStart = line.IndexOf('{');
var openBrackets = 0;
var stringIndex = jsonStart;
while (openBrackets > 0 || stringIndex == jsonStart)
{
var character = line[stringIndex];
switch (character)
{
case '{':
openBrackets++;
break;
case '}':
openBrackets--;
break;
}
stringIndex++;
}
var jsonEnd = stringIndex;
return line.Substring(jsonStart, jsonEnd - jsonStart);
}
}
}
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: 6ad2e365b2be44c3aedc5892edc0ae8d
timeCreated: 1742825212
@@ -0,0 +1,375 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using Unity.PerformanceTesting.Data;
using Unity.PerformanceTesting.Editor;
using Unity.PerformanceTesting.Runtime;
using UnityEditor;
using UnityEngine;
using UnityEngine.TestTools;
using UnityEditor.Build;
using UnityEditor.Build.Reporting;
[assembly: PrebuildSetup(typeof(TestRunBuilder))]
[assembly: PostBuildCleanup(typeof(TestRunBuilder))]
namespace Unity.PerformanceTesting.Editor
{
internal class TestRunBuilder : IPrebuildSetup, IPostBuildCleanup, IPreprocessBuildWithReport, IPostprocessBuildWithReport
{
private const string cleanResources = "PT_ResourcesCleanup";
private const bool EnableDetailedDiffLogging = true;
// UUM-132359: Library/ cache for the Resources/PerformanceTestRun*.json .meta files, so their GUID survives across builds.
private const string MetaCacheDir = "Library/PerformanceTesting/MetaCache";
public int callbackOrder
{
get { return 0; }
}
public void OnPreprocessBuild(BuildReport report)
{
if (!ShouldWriteBuildFiles(report.summary.options))
{
// Sweep artifacts a previously aborted test build may have left
// behind, so they don't get baked into a non-test player.
Cleanup();
return;
}
CreateResourcesFolder();
// UUM-132359: restore the cached .meta before writing, so the asset keeps its GUID instead of getting a new one.
RestoreCachedMeta(Utils.TestRunPath);
RestoreCachedMeta(Utils.RunSettingsPath);
var run = CreateBuildInfo();
SaveToStorage(run, Utils.TestRunPath);
var settings = new RunSettings(Environment.GetCommandLineArgs());
SaveToStorage(settings, Utils.RunSettingsPath);
}
public void OnPostprocessBuild(BuildReport report)
{
if (ShouldWriteBuildFiles(report.summary.options))
{
// UUM-132359: stash the .meta before Cleanup() deletes it, so the next test build can reuse the same GUID.
StashMeta(Utils.TestRunPath);
StashMeta(Utils.RunSettingsPath);
}
// Runs unconditionally so a non-test build also sweeps up files
// left behind by a previously aborted test build.
Cleanup();
}
// The run info and settings files are only ever read by performance tests
// executing inside the player, so builds without test assemblies never need them.
internal static bool ShouldWriteBuildFiles(BuildOptions options)
{
return (options & BuildOptions.IncludeTestAssemblies) != 0;
}
public void Setup()
{
SessionState.SetBool(cleanResources, false);
var run = CreateRunInfo();
SaveToPrefs(run, Utils.PlayerPrefKeyRunJSON);
var settings = new RunSettings(Environment.GetCommandLineArgs());
SaveToPrefs(settings, Utils.PlayerPrefKeySettingsJSON);
}
#if !UNITY_2021_1_OR_NEWER
private static List<string> cachedDependencies;
#endif
static List<string> GetPackageDependencies()
{
#if !UNITY_2021_1_OR_NEWER
if (cachedDependencies != null)
return cachedDependencies;
#endif
IEnumerable<UnityEditor.PackageManager.PackageInfo> packages;
#if !UNITY_2021_1_OR_NEWER
var listRequest = UnityEditor.PackageManager.Client.List(true);
while (!listRequest.IsCompleted)
System.Threading.Thread.Sleep(10);
if (listRequest.Status == UnityEditor.PackageManager.StatusCode.Failure)
Debug.LogError("Failed to list local packages");
packages = new List<UnityEditor.PackageManager.PackageInfo>(listRequest.Result);
#else
packages = UnityEditor.PackageManager.PackageInfo.GetAllRegisteredPackages();
#endif
var reformated = packages.Select(p => $"{p.name}@{p.version}").OrderBy(p => p).ToList();
#if !UNITY_2021_1_OR_NEWER
cachedDependencies = reformated;
#endif
return reformated;
}
public void Cleanup()
{
bool modifiedAssets = false;
if (DeleteFileAndMeta(Utils.TestRunPath))
{
modifiedAssets = true;
}
if (DeleteFileAndMeta(Utils.RunSettingsPath))
{
modifiedAssets = true;
}
// Only delete the Resources folder if we created it and it is empty by
// now - anything still inside was put there by the user, e.g. while a
// stale cleanup flag was left behind by an aborted build.
if (SessionState.GetBool(cleanResources, false) && Directory.Exists(Utils.ResourcesPath)
&& Directory.GetFileSystemEntries(Utils.ResourcesPath).Length == 0)
{
try
{
// Non-recursive on purpose: if an external process created a file
// in the window since the emptiness check, fail instead of
// deleting it. Cleanup also runs during build preprocessing,
// where an unhandled exception would fail the build, so
// deletion failures only warn - a leftover empty folder is acceptable.
Directory.Delete(Utils.ResourcesPath, false);
modifiedAssets = true;
}
catch (Exception e) when (e is IOException || e is UnauthorizedAccessException)
{
Debug.LogWarning($"[TestRunBuilder] Could not delete '{Utils.ResourcesPath}': {e.Message}");
}
// Only remove the folder's meta once the folder is really gone -
// deleting the meta of an existing folder would make Unity
// regenerate it with a new GUID.
if (!Directory.Exists(Utils.ResourcesPath))
{
modifiedAssets |= TryDeleteFile(Utils.ResourcesPath + ".meta");
}
}
// The flag describes at most one build cycle; reset it so it cannot
// leak into a later, unrelated build.
SessionState.SetBool(cleanResources, false);
// Scrub the legacy EditorPrefs flag older package versions may have
// left behind; it is no longer read.
EditorPrefs.DeleteKey(cleanResources);
// Only refresh the AssetDatabase if we actually deleted performance test files
if (modifiedAssets)
{
AssetDatabase.Refresh();
}
}
private bool DeleteFileAndMeta(string path)
{
// Non-short-circuiting so the meta file is attempted even if the
// main file could not be deleted.
return TryDeleteFile(path) | TryDeleteFile(path + ".meta");
}
private static bool TryDeleteFile(string path)
{
if (!File.Exists(path))
{
return false;
}
try
{
File.Delete(path);
return true;
}
catch (Exception e) when (e is IOException || e is UnauthorizedAccessException)
{
// Cleanup runs during build preprocessing, where an unhandled
// exception (e.g. from a transiently locked file) would fail the
// build; the next cleanup pass will retry.
Debug.LogWarning($"[TestRunBuilder] Could not delete '{path}': {e.Message}");
return false;
}
}
// UUM-132359: copy the asset's .meta into MetaCacheDir before we delete it, preserving its GUID.
private static void StashMeta(string assetPath)
{
var srcMeta = assetPath + ".meta";
if (!File.Exists(srcMeta)) return;
try
{
Directory.CreateDirectory(MetaCacheDir);
var destMeta = Path.Combine(MetaCacheDir, Path.GetFileName(srcMeta));
File.Copy(srcMeta, destMeta, overwrite: true);
}
catch (Exception e) when (e is IOException || e is UnauthorizedAccessException)
{
Debug.LogWarning($"[TestRunBuilder] Could not cache '{srcMeta}': {e.Message}");
}
}
// UUM-132359: counterpart to StashMeta - restores the cached .meta before the asset is rewritten. No-op if the
// cache is empty or a .meta already exists at the destination.
private static void RestoreCachedMeta(string assetPath)
{
var destMeta = assetPath + ".meta";
if (File.Exists(destMeta)) return;
var cachedMeta = Path.Combine(MetaCacheDir, Path.GetFileName(destMeta));
if (!File.Exists(cachedMeta)) return;
try
{
var destDir = Path.GetDirectoryName(destMeta);
if (!string.IsNullOrEmpty(destDir) && !Directory.Exists(destDir))
{
Directory.CreateDirectory(destDir);
}
File.Copy(cachedMeta, destMeta, overwrite: false);
}
catch (Exception e) when (e is IOException || e is UnauthorizedAccessException)
{
Debug.LogWarning($"[TestRunBuilder] Could not restore cached meta for '{assetPath}': {e.Message}");
}
}
private static Data.Editor GetEditorInfo()
{
var fullVersion = UnityEditorInternal.InternalEditorUtility.GetFullUnityVersion();
const string pattern = @"(.+\.+.+\.\w+)|((?<=\().+(?=\)))";
var matches = Regex.Matches(fullVersion, pattern);
return new Data.Editor
{
Branch = GetEditorBranch(),
Version = matches[0].Value,
Changeset = matches[1].Value,
Date = UnityEditorInternal.InternalEditorUtility.GetUnityVersionDate(),
};
}
private static string GetEditorBranch()
{
foreach (var method in typeof(UnityEditorInternal.InternalEditorUtility).GetMethods())
{
if (method.Name.Contains("GetUnityBuildBranch"))
{
return (string) method.Invoke(null, null);
}
}
return "null";
}
private static void SetBuildSettings(Run run)
{
if (run.Player == null) run.Player = new Player();
run.Player.GpuSkinning = PlayerSettings.gpuSkinning;
#if UNITY_2021_2_OR_NEWER
run.Player.ScriptingBackend = PlayerSettings
.GetScriptingBackend(NamedBuildTarget.FromBuildTargetGroup(EditorUserBuildSettings.selectedBuildTargetGroup)).ToString();
#else
run.Player.ScriptingBackend = PlayerSettings
.GetScriptingBackend(EditorUserBuildSettings.selectedBuildTargetGroup).ToString();
#endif
run.Player.RenderThreadingMode = PlayerSettings.graphicsJobs ? PlayerSettings.graphicsJobMode.ToString() :
PlayerSettings.MTRendering ? "MultiThreaded" : "SingleThreaded";
run.Player.AndroidTargetSdkVersion = PlayerSettings.Android.targetSdkVersion.ToString();
run.Player.AndroidBuildSystem = EditorUserBuildSettings.androidBuildSystem.ToString();
run.Player.BuildTarget = EditorUserBuildSettings.activeBuildTarget.ToString();
run.Player.StereoRenderingPath = PlayerSettings.stereoRenderingPath.ToString();
}
public Run CreateRunInfo()
{
var run = new Run();
run.Editor = GetEditorInfo();
run.Dependencies = GetPackageDependencies();
SetBuildSettings(run);
run.Date = Utils.ConvertToUnixTimestamp(DateTime.Now);
return run;
}
public Run CreateBuildInfo()
{
var run = new Run();
run.Editor = GetEditorInfo();
run.Dependencies = GetPackageDependencies();
SetBuildSettings(run);
return run;
}
public Run GetPerformanceTestRun()
{
var run = CreateRunInfo();
Metadata.SetRuntimeSettings(run);
return run;
}
private void CreateResourcesFolder()
{
if (Directory.Exists(Utils.ResourcesPath))
{
SessionState.SetBool(cleanResources, false);
return;
}
SessionState.SetBool(cleanResources, true);
AssetDatabase.CreateFolder("Assets", "Resources");
}
private string SaveToStorage(object obj, string path)
{
var json = JsonUtility.ToJson(obj);
if (File.Exists(path))
{
var existing = File.ReadAllText(path);
if (existing == json)
{
return json;
}
Debug.LogWarning($"[TestRunBuilder] Content changed for '{path}' - rewriting file\nOld length: {existing.Length} bytes\nNew length: {json.Length} bytes");
if (EnableDetailedDiffLogging)
{
LogContentDifferences(existing, json);
}
}
else
{
Debug.Log($"[TestRunBuilder] Creating new file '{path}' ({json.Length} bytes)");
}
File.WriteAllText(path, json);
return json;
}
private void LogContentDifferences(string existing, string json)
{
Debug.Log($"[TestRunBuilder] Old content:\n{existing}");
Debug.Log($"[TestRunBuilder] New content:\n{json}");
}
private string SaveToPrefs(object obj, string key)
{
var json = JsonUtility.ToJson(obj, true);
PlayerPrefs.SetString(key, json);
return json;
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 3e68133cd13ef104088b87bbf4f00320
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,87 @@
using System;
using System.IO;
using UnityEditor;
using UnityEditor.TestRunner.CommandLineParser;
using UnityEditor.TestTools.TestRunner.Api;
using UnityEngine;
namespace Unity.PerformanceTesting.Editor
{
[InitializeOnLoad]
class TestRunnerInitializer
{
static TestRunnerInitializer()
{
var args = GetCmdLineArguments();
var resultsHandler = args.IsCmdLineRun && !string.IsNullOrEmpty(args.PerfTestResults)
? CreateCmdLineResultsHandler(args)
: CreateDefaultResultsHandler(args);
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RegisterCallbacks(resultsHandler);
}
static ICallbacks CreateCmdLineResultsHandler(CmdLineArguments cmdLineArguments)
{
var callbacks = ScriptableObject.CreateInstance<CmdLineResultsSavingCallbacks>();
callbacks.SetResultsLocation(cmdLineArguments.PerfTestResults);
return callbacks;
}
static ICallbacks CreateDefaultResultsHandler(CmdLineArguments cmdLineArguments)
{
var callbacks = ScriptableObject.CreateInstance<PerformanceTestRunSaver>();
callbacks.SetTestResultsDirectory(GetDirectoryOrNull(cmdLineArguments.TestResults));
return callbacks;
}
static string GetDirectoryOrNull(string filePath)
{
if (string.IsNullOrEmpty(filePath))
{
return null;
}
try
{
return Path.GetDirectoryName(Path.GetFullPath(filePath));
}
catch (Exception e)
{
Debug.LogWarning($"Could not determine the test results directory from '{filePath}'.\n{e}");
return null;
}
}
static CmdLineArguments GetCmdLineArguments()
{
var isCmdLineTestRun = false;
string resultFilePath = null;
string testResultsFilePath = null;
var optionSet = new CommandLineOptionSet(
new CommandLineOption("runTests", () => { isCmdLineTestRun = true; }),
new CommandLineOption("runEditorTests", () => { isCmdLineTestRun = true; }),
new CommandLineOption("perfTestResults", filePath => { resultFilePath = filePath; }),
new CommandLineOption("testResults", filePath => { testResultsFilePath = filePath; })
);
optionSet.Parse(Environment.GetCommandLineArgs());
return new CmdLineArguments(isCmdLineTestRun, resultFilePath, testResultsFilePath);
}
class CmdLineArguments
{
public bool IsCmdLineRun;
public string PerfTestResults;
public string TestResults;
public CmdLineArguments(bool isCmdLineRun, string perfTestResults, string testResults)
{
IsCmdLineRun = isCmdLineRun;
PerfTestResults = perfTestResults;
TestResults = testResults;
}
}
}
}
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: 4ea07ee0847c4e359272dd9b57e1cb2b
timeCreated: 1742825056
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: b6a3ada20a8c4030907eb96c0cb331bc
timeCreated: 1683027438
@@ -0,0 +1,39 @@
using System;
using UnityEditor;
using UnityEditor.UIElements;
using UnityEngine.UIElements;
namespace Unity.PerformanceTesting.Editor.UIElements
{
internal class ToolbarWithSearch : VisualElement
{
private string m_searchString;
public Action<string> SearchTextChanged;
public ToolbarWithSearch()
{
// Create the toolbar and search field elements
var toolbar = new Toolbar();
var searchField = new ToolbarSearchField();
// Add the toolbar and search field elements to this element
Add(toolbar);
Add(searchField);
}
public void Draw()
{
EditorGUILayout.BeginHorizontal(EditorStyles.toolbar);
EditorGUI.BeginChangeCheck();
m_searchString = EditorGUILayout.TextField(m_searchString, EditorStyles.toolbarSearchField);
if (EditorGUI.EndChangeCheck()) SearchTextChanged?.Invoke(m_searchString);
EditorGUILayout.EndHorizontal();
}
public void ClearSearchString()
{
m_searchString = string.Empty;
SearchTextChanged?.Invoke(m_searchString);
}
}
}
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: f660b03f07d34d62990e579338856f5f
timeCreated: 1683033109
@@ -0,0 +1,30 @@
{
"name": "Unity.PerformanceTesting.Editor",
"rootNamespace": "",
"references": [
"Unity.PerformanceTesting",
"UnityEngine.TestRunner",
"UnityEditor.TestRunner"
],
"includePlatforms": [
"Editor"
],
"excludePlatforms": [],
"allowUnsafeCode": false,
"overrideReferences": true,
"precompiledReferences": [
"nunit.framework.dll"
],
"autoReferenced": false,
"defineConstraints": [
"UNITY_TESTS_FRAMEWORK"
],
"versionDefines": [
{
"name": "com.unity.test-framework",
"expression": "",
"define": "UNITY_TESTS_FRAMEWORK"
}
],
"noEngineReferences": false
}
@@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 9f843d4b38153f04b9cee0502d568525
AssemblyDefinitionImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,5 @@
com.unity.test-framework.performance copyright © 2020 Unity Technologies ApS
Licensed under the Unity Companion License for Unity-dependent projects--see [Unity Companion License](http://www.unity3d.com/legal/licenses/Unity_Companion_License).
Unless expressly provided otherwise, the Software under this license is made available strictly on an “AS IS” BASIS WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. Please review the license for details on these and other terms and conditions.
@@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 9c8887056eaaa7e4888ca0131e2892da
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 6edbafb3b8c712640ae66bb69eabb0a8
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,10 @@
using System.Reflection;
using System.Runtime.CompilerServices;
[assembly: AssemblyTitle("Unity.PerformanceTesting")]
[assembly: InternalsVisibleTo("Unity.PerformanceTesting.Editor")]
[assembly: InternalsVisibleTo("Unity.PerformanceTesting.Tests.Runtime")]
[assembly: InternalsVisibleTo("Unity.GraphTools.Foundation.Testing.Editor")]
[assembly: AssemblyVersion("3.4.2")]
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: ac464bc14bd17cc4ca2e8f909ca4f061
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: d2bd35c14da30b94c81ec15c7db30136
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,85 @@
using System;
using System.Collections;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using NUnit.Framework.Internal;
using Unity.PerformanceTesting.Data;
using Unity.PerformanceTesting.Runtime;
using UnityEngine.TestTools;
namespace Unity.PerformanceTesting
{
/// <summary>
/// Test attribute to specify a performance test. It will add category "Performance" to test properties.
/// Optionally, a performance regression threshold can be specified, to be used at the test result endpoint.
/// </summary>
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class PerformanceAttribute : CategoryAttribute, IOuterUnityTestAction, IApplyToTest
{
/// <summary>
/// Adds performance attribute to a test method.
/// </summary>
public PerformanceAttribute()
: base("Performance") { }
/// <summary>
/// Performance regression threshold.
/// </summary>
public double Threshold { get; } = PerformanceTest.DefaultThresholdValue;
/// <summary>
/// Adds performance attribute to a test method with a performance regression threshold.
/// </summary>
/// <param name="threshold">A performance regression threshold to associate for the given test, taken into account at the result endpoint. The value represents a multiple of the result. Only positive values are considered valid. If the value is invalid, the property will not be serialized/sent to the result endpoint.</param>
public PerformanceAttribute(double threshold) : base("Performance")
{
Threshold = threshold;
}
/// <summary>
/// Executed before a test execution.
/// </summary>
/// <param name="test">Test to execute.</param>
/// <returns>Enumerable collection of actions to perform before test setup.</returns>
public IEnumerator BeforeTest(ITest test)
{
if (RunSettings.Instance == null)
{
RunSettings.Instance = ResourcesLoader.Load<RunSettings>(Utils.RunSettings, Utils.PlayerPrefKeySettingsJSON);
}
// domain reload will cause this method to be hit multiple times
// active performance test is serialized and survives reloads
if (PerformanceTest.Active == null)
{
PerformanceTest.StartTest(test);
yield return null;
}
}
/// <summary>
/// Executed after a test execution.
/// </summary>
/// <param name="test">Executed test.</param>
/// <returns>Enumerable collection of actions to perform after test teardown.</returns>
public IEnumerator AfterTest(ITest test)
{
PerformanceTest.EndTest(test);
yield return null;
}
/// <summary>
/// Used to apply the Threshold property to the Test's Properties
/// </summary>
/// <param name="test">An NUnit test</param>
public new void ApplyToTest(Test test)
{
if (Threshold > PerformanceTest.ThresholdSerializationCutOff)
{
test.Properties.Add("Threshold", Threshold);
}
base.ApplyToTest(test);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: f97b05006f7e8f34bbde1823b11b5308
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,11 @@
namespace Unity.PerformanceTesting
{
internal class RequiredMemberAttribute
#if UNITY_2021_1_OR_NEWER
: UnityEngine.Scripting.RequiredMemberAttribute
#else
: System.Attribute
#endif
{
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 0d9f444a464203c478024791eff61732
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,37 @@
using System;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using NUnit.Framework.Internal;
namespace Unity.PerformanceTesting
{
/// <summary>
/// Test attribute to specify test version.
/// </summary>
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
public class VersionAttribute : NUnitAttribute, IApplyToTest
{
/// <summary>
/// Test version.
/// </summary>
public string Version { get; }
/// <summary>
/// Adds attribute to specify test version.
/// </summary>
/// <param name="version">Version of the test.</param>
public VersionAttribute(string version)
{
Version = version;
}
/// <summary>
/// Used by NUnit to apply version to properties.
/// </summary>
/// <param name="test">An NUnit test to apply the version property to.</param>
public void ApplyToTest(Test test)
{
test.Properties.Add("Version", this);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 3ea2d7f3aee148a48bf80eafd7113fe0
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 0369bc643b4a11246a869e582c0f275a
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,31 @@
using System;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents information about Editor version.
/// </summary>
[Serializable]
public class Editor
{
/// <summary>
/// Full editor version including changeset.
/// </summary>
[RequiredMember] public string Version;
/// <summary>
/// Branch name from which the editor was built.
/// </summary>
[RequiredMember] public string Branch;
/// <summary>
/// Changeset 12 characters long.
/// </summary>
[RequiredMember] public string Changeset;
/// <summary>
/// Editor version datetime in Unix Epoch milliseconds format.
/// </summary>
[RequiredMember] public int Date;
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 634ca08416be7e446b17074b7a8d85ef
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,46 @@
using System;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents information about Hardware.
/// </summary>
[Serializable]
public class Hardware
{
/// <summary>
/// Operating system.
/// </summary>
[RequiredMember] public string OperatingSystem;
/// <summary>
/// Device model.
/// </summary>
[RequiredMember] public string DeviceModel;
/// <summary>
/// Device name.
/// </summary>
[RequiredMember] public string DeviceName;
/// <summary>
/// Processor type name.
/// </summary>
[RequiredMember] public string ProcessorType;
/// <summary>
/// Processor thread count.
/// </summary>
[RequiredMember] public int ProcessorCount;
/// <summary>
/// Graphics device name.
/// </summary>
[RequiredMember] public string GraphicsDeviceName;
/// <summary>
/// System Memory size in MegaBytes.
/// </summary>
[RequiredMember] public int SystemMemorySizeMB;
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: eecaaae1ce1cb6d4191f80484e152f46
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,47 @@
using System;
using System.Collections.Generic;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents a performance test result.
/// </summary>
[Serializable]
public class PerformanceTestResult
{
/// <summary>
/// Full test name.
/// </summary>
[RequiredMember] public string Name;
/// <summary>
/// Test script name.
/// </summary>
[RequiredMember] public string ClassName;
/// <summary>
/// Test method name.
/// </summary>
[RequiredMember] public string MethodName;
/// <summary>
/// Test version.
/// </summary>
[RequiredMember] public string Version;
/// <summary>
/// Categories.
/// </summary>
[RequiredMember] public List<string> Categories = new List<string>();
/// <summary>
/// Groups of performance samples.
/// </summary>
[RequiredMember] public List<SampleGroup> SampleGroups = new List<SampleGroup>();
/// <summary>
/// Performance regression threshold. A negative value indicates it is not to be used/transmitted.
/// </summary>
public double Threshold = PerformanceTest.DefaultThresholdValue;
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: eee38b155cc303447b8c478d95185d54
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,123 @@
using System;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents player settings of a performance test, sampled at the beginning of a test run.
/// </summary>
[Serializable]
public class Player
{
/// <summary>
/// True if the player is a Development build. False if the player is a Release build.
/// </summary>
[RequiredMember] public bool Development;
/// <summary>
/// Screen resolution width.
/// </summary>
[RequiredMember] public int ScreenWidth;
/// <summary>
/// Screen resolution height.
/// </summary>
[RequiredMember] public int ScreenHeight;
/// <summary>
/// Screen refresh rate.
/// </summary>
[RequiredMember] public int ScreenRefreshRate;
/// <summary>
/// Whether the player is in fullscreen or windowed modes.
/// </summary>
[RequiredMember] public bool Fullscreen;
/// <summary>
/// The number of VSyncs that should pass between each frame. Use 'Don't Sync' (0) to not wait for VSync. Value must be 0, 1, 2, 3, or 4.
/// </summary>
[RequiredMember] public int Vsync;
/// <summary>
/// Anti aliasing.
/// </summary>
[RequiredMember] public int AntiAliasing;
/// <summary>
/// Whether player is in batchmode or not.
/// </summary>
[RequiredMember] public bool Batchmode;
/// <summary>
/// Render threading mode.
/// </summary>
[RequiredMember] public string RenderThreadingMode;
/// <summary>
/// Whether the render threading mode is multithreaded or not.
/// </summary>
[RequiredMember] public bool MtRendering;
/// <summary>
/// Whether the render threading mode uses Graphics Jobs or not.
/// </summary>
[RequiredMember] public bool GraphicsJobs;
/// <summary>
/// Whether the Unity Graphics Kernel is enabled or not.
/// </summary>
[RequiredMember] public bool UnityGraphicsKernel;
/// <summary>
/// Whether GPU skinning is enabled or not.
/// </summary>
[RequiredMember] public bool GpuSkinning;
/// <summary>
/// String representation of players RuntimePlatform.
/// </summary>
[RequiredMember] public string Platform;
/// <summary>
/// Color space.
/// </summary>
[RequiredMember] public string ColorSpace;
/// <summary>
/// Anisotropic filtering.
/// </summary>
[RequiredMember] public string AnisotropicFiltering;
/// <summary>
/// Blend weights.
/// </summary>
[RequiredMember] public string BlendWeights;
/// <summary>
/// Graphics API.
/// </summary>
[RequiredMember] public string GraphicsApi;
// strings because their enums are editor only.
/// <summary>
/// Scripting backend.
/// </summary>
[RequiredMember] public string ScriptingBackend;
/// <summary>
/// Android target SDK version.
/// </summary>
[RequiredMember] public string AndroidTargetSdkVersion;
/// <summary>
/// Android build system.
/// </summary>
[RequiredMember] public string AndroidBuildSystem;
/// <summary>
/// Build target.
/// </summary>
[RequiredMember] public string BuildTarget;
/// <summary>
/// Stereo rendering path.
/// </summary>
[RequiredMember] public string StereoRenderingPath;
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 451a2d412ec85674b870cfe148524e5c
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,33 @@
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents Project version.
/// </summary>
public class Project
{
/// <summary>
/// Name of the project.
/// </summary>
[RequiredMember] public string Name;
/// <summary>
/// Version of the project unrelated to changeset e.g. package version.
/// </summary>
[RequiredMember] public string Version;
/// <summary>
/// Commit branch.
/// </summary>
[RequiredMember] public string Branch;
/// <summary>
/// Commit changeset.
/// </summary>
[RequiredMember] public string Changeset;
/// <summary>
/// Commit datetime in Unix Epoch milliseconds format.
/// </summary>
[RequiredMember] public int Date;
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 601d7eee390860c4e80380ed9ff5e296
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,41 @@
using System;
using System.Collections.Generic;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents a performance test run.
/// </summary>
[Serializable]
public class Run
{
/// <summary>
/// Test Suite.
/// </summary>
[RequiredMember] public string TestSuite;
/// <summary>
/// Test run start datetime in Unix Epoch milliseconds format.
/// </summary>
[RequiredMember] public long Date;
/// <summary>
/// Player settings.
/// </summary>
[RequiredMember] public Player Player;
/// <summary>
/// Hardware information.
/// </summary>
[RequiredMember] public Hardware Hardware;
/// <summary>
/// Editor version.
/// </summary>
[RequiredMember] public Editor Editor;
/// <summary>
/// Package dependencies.
/// </summary>
[RequiredMember] public List<string> Dependencies = new List<string>();
/// <summary>
/// List of performance test results.
/// </summary>
[RequiredMember] public List<PerformanceTestResult> Results = new List<PerformanceTestResult>();
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 2cea93e2f05193940bf0b8deca0c3898
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,68 @@
using System;
using System.Runtime.Serialization;
using Unity.PerformanceTesting.Runtime;
using UnityEngine;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Settings store
/// @TODO make it public along with guidelines of how and when to use
/// @TODO add a property bag
/// </summary>
[Serializable]
internal class RunSettings : IDeserializationCallback
{
public RunSettings(params string[] args)
{
if (int.TryParse(Utils.GetArg(args, "-performance-measurement-count"), out var measurementMultiplier))
{
MeasurementCount = measurementMultiplier;
}
}
private static RunSettings m_Instance;
#if UNITY_EDITOR
[RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.SubsystemRegistration)]
static void ResetStaticsOnLoad()
{
m_Instance = null;
}
#endif
/// <summary>
/// Singleton instance of settings.
/// </summary>
public static RunSettings Instance
{
get
{
if (m_Instance == null)
{
m_Instance = ResourcesLoader.Load<RunSettings>(Utils.RunSettings, Utils.PlayerPrefKeySettingsJSON);
}
return m_Instance;
}
set { m_Instance = value; }
}
/// <summary>
/// Measurement counts will be overriden by specified value when using Measure.Method and Measure.Frames.
/// </summary>
public int MeasurementCount = -1;
/// <summary>
/// Validates the deserialized object.
/// </summary>
/// <param name="sender">The object that initiated the deserialization process.</param>
public void OnDeserialization(object sender)
{
if (MeasurementCount < -1)
{
throw new SerializationException("MeasurementCount cannot be negative, except for the initial value of -1.");
}
}
}
}
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: 60f3b660212a40e7afb5bf15edcd64f2
timeCreated: 1617790852
@@ -0,0 +1,91 @@
using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
using Unity.PerformanceTesting.Exceptions;
namespace Unity.PerformanceTesting.Data
{
/// <summary>
/// Represents a group of samples for a performance test that share a common name and unit.
/// </summary>
[Serializable]
public class SampleGroup : IDeserializationCallback
{
/// <summary>
/// Name of the sample group.
/// </summary>
public string Name;
/// <summary>
/// Measurement unit.
/// </summary>
public SampleUnit Unit;
/// <summary>
/// Whether the measurement is inverted and increase is positive.
/// </summary>
public bool IncreaseIsBetter;
/// <summary>
/// List of samples.
/// </summary>
public List<double> Samples = new List<double>();
/// <summary>
/// Minimum value of samples.
/// </summary>
public double Min;
/// <summary>
/// Maximum value of samples.
/// </summary>
public double Max;
/// <summary>
/// Median value of samples.
/// </summary>
public double Median;
/// <summary>
/// Average value of samples.
/// </summary>
public double Average;
/// <summary>
/// Standard deviation of samples.
/// </summary>
public double StandardDeviation;
/// <summary>
/// Sum of samples.
/// </summary>
public double Sum;
/// <summary>
/// Creates a sample group without initializing values.
/// </summary>
public SampleGroup(){}
/// <summary>
/// Creates a sample group with provided arguments.
/// </summary>
/// <param name="name">Sample group name.</param>
/// <param name="unit">Measurement unit.</param>
/// <param name="increaseIsBetter">Whether the measurement is inverted and increase is positive.</param>
/// <exception cref="PerformanceTestException">Exception thrown when invalid name is used.</exception>
public SampleGroup(string name, SampleUnit unit, bool increaseIsBetter)
{
Name = name;
Unit = unit;
IncreaseIsBetter = increaseIsBetter;
if (string.IsNullOrEmpty(name))
{
throw new PerformanceTestException("Sample group name is empty. Please assign a valid name.");
}
}
/// <summary>
/// Validates the deserialized object.
/// </summary>
/// <param name="sender">The object that initiated the deserialization process.</param>
public void OnDeserialization(object sender)
{
if (string.IsNullOrEmpty(Name))
{
throw new PerformanceTestException("Sample group name is empty. Please assign a valid name.");
}
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: ea30bf37a2f167f4aa4086531e8d5b03
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,85 @@
using System;
namespace Unity.PerformanceTesting
{
/// <summary>
/// Measurement unit used for sample groups.
/// </summary>
public enum SampleUnit
{
/// <summary>
/// Nanoseconds.
/// </summary>
Nanosecond,
/// <summary>
/// Microseconds.
/// </summary>
Microsecond,
/// <summary>
/// Milliseconds.
/// </summary>
Millisecond,
/// <summary>
/// Seconds.
/// </summary>
Second,
/// <summary>
/// Bytes.
/// </summary>
Byte,
/// <summary>
/// Kilobytes.
/// </summary>
Kilobyte,
/// <summary>
/// Megabytes.
/// </summary>
Megabyte,
/// <summary>
/// Gigabytes.
/// </summary>
Gigabyte,
/// <summary>
/// Undefined, represents any other unit we don't have by default. When using it make sure your sample group name represents the measurement.
/// </summary>
Undefined
}
static class SampleUnitExtensions
{
public static string ShortName(this SampleUnit s)
{
switch (s)
{
case SampleUnit.Nanosecond:
return "ns";
case SampleUnit.Microsecond:
return "μs";
case SampleUnit.Millisecond:
return "ms";
case SampleUnit.Second:
return "s";
case SampleUnit.Byte:
return "b";
case SampleUnit.Kilobyte:
return "kb";
case SampleUnit.Megabyte:
return "mb";
case SampleUnit.Gigabyte:
return "gb";
case SampleUnit.Undefined:
return "";
default:
throw new ArgumentOutOfRangeException(nameof(s), s, null);
}
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 8a61e4b297fe5f94cb16e6f77a71892c
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,120 @@
using System;
using System.Diagnostics;
using Unity.PerformanceTesting.Exceptions;
using Unity.PerformanceTesting.Measurements;
using Unity.PerformanceTesting.Runtime;
using UnityEngine;
using Object = UnityEngine.Object;
namespace Unity.PerformanceTesting
{
/// <summary>
/// Enables measuring of performance metrics during a performance test.
/// </summary>
public static class Measure
{
/// <summary>
/// Saves provided value as a performance measurement.
/// </summary>
/// <param name="sampleGroup">The sample group to save the value to.</param>
/// <param name="value">Value to be saved.</param>
public static void Custom(SampleGroup sampleGroup, double value)
{
VerifyValue(sampleGroup.Name, value);
var activeSampleGroup = PerformanceTest.GetSampleGroup(sampleGroup.Name);
if (activeSampleGroup == null)
{
PerformanceTest.AddSampleGroup(sampleGroup);
activeSampleGroup = sampleGroup;
}
activeSampleGroup.Samples.Add(value);
}
/// <summary>
/// Saves provided value as a performance measurement.
/// </summary>
/// <param name="name">The name of the sample group to save the value to.</param>
/// <param name="value">Value to be saved.</param>
public static void Custom(string name, double value)
{
VerifyValue(name, value);
var activeSampleGroup = PerformanceTest.GetSampleGroup(name);
if (activeSampleGroup == null)
{
activeSampleGroup = new SampleGroup(name);
PerformanceTest.AddSampleGroup(activeSampleGroup);
}
activeSampleGroup.Samples.Add(value);
}
static void VerifyValue(string name, double value)
{
if (double.IsNaN(value))
throw new PerformanceTestException(
$"Trying to record value which is not a number for sample group: {name}");
}
/// <summary>
/// Measures execution time for the given scope as a single time.
/// </summary>
/// <param name="name">Name to use for the sample group.</param>
/// <returns>IDisposable <see cref="ScopeMeasurement"/> on which you should call the <see cref="ScopeMeasurement.Dispose"/> method to stop measurement.</returns>
public static ScopeMeasurement Scope(string name = "Time")
{
return new ScopeMeasurement(name);
}
/// <summary>
/// Measures execution time for the given scope as a single time.
/// </summary>
/// <param name="sampleGroup">Sample group to use to save samples to.</param>
/// <returns>IDisposable <see cref="ScopeMeasurement"/> on which you should call the <see cref="ScopeMeasurement.Dispose"/> method to stop measurement.</returns>
public static ScopeMeasurement Scope(SampleGroup sampleGroup)
{
return new ScopeMeasurement(sampleGroup);
}
/// <summary>
/// Measures profiler markers for the given scope.
/// </summary>
/// <param name="profilerMarkerLabels">List of profiler marker names.</param>
/// <returns>A ProfilerMeasurement instance configured according to the given arguments.</returns>
public static ProfilerMeasurement ProfilerMarkers(params string[] profilerMarkerLabels)
{
return new ProfilerMeasurement(profilerMarkerLabels);
}
/// <summary>
/// Measures profiler markers for the given scope.
/// </summary>
/// <param name="sampleGroups">List of SampleGroups where the name matches the profiler marker to measure.</param>
/// <returns>A ProfilerMeasurement instance configured according to the given arguments.</returns>
public static ProfilerMeasurement ProfilerMarkers(params SampleGroup[] sampleGroups)
{
return new ProfilerMeasurement(sampleGroups);
}
/// <summary>
/// Measures execution time for a method with given parameters.
/// </summary>
/// <param name="action">The action to be measured.</param>
/// <returns><see cref="MethodMeasurement"/>using a builder pattern to provide parameters. Call <see cref="ScopeMeasurement.Run"/> to start measurement.</returns>
public static MethodMeasurement Method(Action action)
{
return new MethodMeasurement(action);
}
/// <summary>
/// Measures frame times with given parameters.
/// </summary>
/// <returns><see cref="FramesMeasurement"/> using a builder pattern to provide parameters. Call <see cref="FramesMeasurement.Run"/> method to start measurement.</returns>
public static FramesMeasurement Frames()
{
return new FramesMeasurement();
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 788d1fc54ab24f14d80cb45b26012c82
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: c0c170ab616b34944bcd5e75cfcb9d06
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,372 @@
using System;
using System.Collections;
using System.Diagnostics;
using Unity.PerformanceTesting.Data;
using Unity.PerformanceTesting.Runtime;
using Unity.PerformanceTesting.Statistics;
using UnityEngine;
using Debug = UnityEngine.Debug;
using Object = UnityEngine.Object;
namespace Unity.PerformanceTesting.Measurements
{
/// <summary>
/// Allows measuring of frame times.
/// </summary>
public class FramesMeasurement
{
private const int k_MinTestTimeMs = 500;
private const int k_MinWarmupTimeMs = 80;
private const int k_ProbingMultiplier = 4;
internal const int k_MinIterations = 7;
internal const int k_MaxDynamicMeasurements = 1000;
private const double k_DefaultMaxRelativeError = 0.02;
private const ConfidenceLevel k_DefaultConfidenceLevel = ConfidenceLevel.L99;
private const OutlierMode k_DefaultOutlierMode = OutlierMode.Remove;
private SampleGroup[] m_ProfilerSampleGroups;
private SampleGroup m_SampleGroup = new SampleGroup("FrameTime");
private int m_DesiredFrameCount;
internal bool m_DynamicMeasurementCount;
private double m_MaxRelativeError = k_DefaultMaxRelativeError;
private ConfidenceLevel m_ConfidenceLevel = k_DefaultConfidenceLevel;
private OutlierMode m_OutlierMode = k_DefaultOutlierMode;
private int m_Executions;
private int m_Warmup = -1;
private bool m_RecordFrametime = true;
/// <summary>
/// Records provided profiler markers once per frame.
/// </summary>
/// <param name="profilerMarkerNames">Profiler marker names as in profiler window.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement ProfilerMarkers(params string[] profilerMarkerNames)
{
m_ProfilerSampleGroups = Utils.CreateSampleGroupsFromMarkerNames(profilerMarkerNames);
return this;
}
/// <summary>
/// Records provided profiler markers once per frame.
/// </summary>
/// <param name="sampleGroups">List of SampleGroups where a name matches the profiler marker and desired SampleUnit</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement ProfilerMarkers(params SampleGroup[] sampleGroups)
{
m_ProfilerSampleGroups = sampleGroups;
return this;
}
/// <summary>
/// Overrides the name of default sample group "Time".
/// </summary>
/// <param name="name">Name of the sample group.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement SampleGroup(string name)
{
m_SampleGroup.Name = name;
return this;
}
/// <summary>
/// Overrides the default sample group "Time"
/// </summary>
/// <param name="sampleGroup">Sample group to use.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement SampleGroup(SampleGroup sampleGroup)
{
m_SampleGroup = sampleGroup;
return this;
}
/// <summary>
/// Count of measurements to take.
/// </summary>
/// <param name="count">Count of measurements.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement MeasurementCount(int count)
{
m_Executions = count;
return this;
}
/// <summary>
/// Dynamically find a suitable measurement count based on the margin of error of the samples.
/// The measurements will stop once a certain amount of samples (specified by a confidence interval)
/// falls within an acceptable error range from the result (defined by a relative error of the mean).
/// A default margin of error range of 2% and a default confidence interval of 99% will be used.
/// </summary>
/// <param name="outlierMode">Outlier mode allows to include or exclude outliers when evaluating the stop criterion.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement DynamicMeasurementCount(OutlierMode outlierMode = k_DefaultOutlierMode)
{
m_DynamicMeasurementCount = true;
m_OutlierMode = outlierMode;
return this;
}
/// <summary>
/// Dynamically find a suitable measurement count based on the margin of error of the samples.
/// The measurements will stop once a certain amount of samples (specified by a confidence interval)
/// falls within an acceptable error range from the result (defined by a relative error of the mean).
/// </summary>
/// <param name="maxRelativeError">The maximum relative error of the mean that the margin of error must fall into.</param>
/// <param name="confidenceLevel">The confidence interval which will be used to calculate the margin of error.</param>
/// <param name="outlierMode">Outlier mode allows to include or exclude outliers when evaluating the stop criterion.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement DynamicMeasurementCount(double maxRelativeError, ConfidenceLevel confidenceLevel = k_DefaultConfidenceLevel,
OutlierMode outlierMode = k_DefaultOutlierMode)
{
m_MaxRelativeError = maxRelativeError;
m_ConfidenceLevel = confidenceLevel;
m_DynamicMeasurementCount = true;
m_OutlierMode = outlierMode;
return this;
}
/// <summary>
/// Count of warmup executions.
/// </summary>
/// <param name="count">Count of warmup executions.</param>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement WarmupCount(int count)
{
m_Warmup = count;
return this;
}
/// <summary>
/// Specifies frame times should not be recorded.
/// </summary>
/// <returns>An updated instance of the FramesMeasurement to be used in fluent syntax.</returns>
public FramesMeasurement DontRecordFrametime()
{
m_RecordFrametime = false;
return this;
}
/// <summary>
/// Switches frame time measurement to asynchronous scope measurement.
/// </summary>
/// <param name="name">Sample group name.</param>
/// <returns>A ScopedFrameTimeMeasurement instance configured according to the given arguments.</returns>
public ScopedFrameTimeMeasurement Scope(string name = "Time")
{
return new ScopedFrameTimeMeasurement(name);
}
/// <summary>
/// Switches frame time measurement to asynchronous scope measurement.
/// </summary>
/// <param name="sampleGroup">Sample group to save measurements.</param>
/// <returns>A ScopedFrameTimeMeasurement instance configured according to the given arguments.</returns>
public ScopedFrameTimeMeasurement Scope(SampleGroup sampleGroup)
{
return new ScopedFrameTimeMeasurement(sampleGroup);
}
/// <summary>
/// Executes the frame time measurement with given parameters. When MeasurementCount is not provided, a probing method will run to determine desired measurement counts.
/// </summary>
/// <returns>IEnumerator to yield until finish.</returns>
public IEnumerator Run()
{
ValidateCorrectDynamicMeasurementCountUsage();
if (!ValidateMeasurementAndWarmupCount()) yield break;
SettingsOverride();
yield return m_Warmup > -1 ? WaitFor(m_Warmup) : GetDesiredIterationCount();
using (Measure.ProfilerMarkers(m_ProfilerSampleGroups))
{
if (m_DynamicMeasurementCount)
{
yield return RunDynamicMeasurementCount();
}
else
{
yield return RunFixedMeasurementCount();
}
// WaitForEndOfFrame coroutine is not invoked on the editor in batch mode
// This may lead to unexpected behavior and is better to avoid
// https://docs.unity3d.com/ScriptReference/WaitForEndOfFrame.html
if (!Application.isBatchMode && Application.isPlaying)
{
yield return new WaitForEndOfFrame();
}
}
}
private IEnumerator RunDynamicMeasurementCount()
{
while (true)
{
using (Measure.Scope(m_SampleGroup))
{
yield return null;
}
if (SampleCountFulfillsRequirements())
break;
}
}
private IEnumerator RunFixedMeasurementCount()
{
m_DesiredFrameCount = m_Executions > 0 ? m_Executions : m_DesiredFrameCount;
for (var i = 0; i < m_DesiredFrameCount; i++)
{
if (m_RecordFrametime)
{
using (Measure.Scope(m_SampleGroup))
{
yield return null;
}
}
else
{
yield return null;
}
}
}
private bool ValidateMeasurementAndWarmupCount()
{
if (m_DynamicMeasurementCount || m_Executions != 0 || m_Warmup < 0) return true;
Debug.LogError("Provide execution count or remove warmup count from frames measurement.");
return false;
}
private void ValidateCorrectDynamicMeasurementCountUsage()
{
if (!m_DynamicMeasurementCount)
return;
if (m_Executions > 0)
{
m_DynamicMeasurementCount = false;
Debug.LogWarning("DynamicMeasurementCount will be ignored because MeasurementCount was specified.");
return;
}
if (!m_RecordFrametime)
{
m_DynamicMeasurementCount = false;
Debug.LogWarning("DynamicMeasurementCount will be ignored because FrameTime measurement was disabled.");
}
}
private bool SampleCountFulfillsRequirements()
{
var samples = m_SampleGroup.Samples;
var sampleCount = samples.Count;
var statistics = MeasurementsStatistics.Calculate(samples, m_OutlierMode, m_ConfidenceLevel);
var actualError = statistics.MarginOfError;
var maxError = m_MaxRelativeError * statistics.Mean;
if (sampleCount >= k_MinIterations && actualError < maxError)
return true;
if (sampleCount >= k_MaxDynamicMeasurements)
return true;
return false;
}
/// <summary>
/// Overrides measurement count based on performance run settings
/// </summary>
private void SettingsOverride()
{
var count = RunSettings.Instance.MeasurementCount;
if (count < 0) { return; }
m_Executions = count;
m_Warmup = m_Warmup < 1 ? 0 : count;
m_DynamicMeasurementCount = false;
}
private IEnumerator GetDesiredIterationCount()
{
var executionTime = 0.0D;
var iterations = 1;
while (executionTime < k_MinWarmupTimeMs)
{
var sw = Stopwatch.GetTimestamp();
yield return WaitFor(iterations);
executionTime = TimeSpan.FromTicks(Stopwatch.GetTimestamp() - sw).TotalMilliseconds;
if (iterations == 1 && executionTime > 40)
{
m_DesiredFrameCount = k_MinIterations;
yield break;
}
if (iterations == 64)
{
m_DesiredFrameCount = 120;
yield break;
}
if (executionTime < k_MinWarmupTimeMs)
{
iterations *= k_ProbingMultiplier;
}
}
m_DesiredFrameCount = (int)(k_MinTestTimeMs * iterations / executionTime);
}
private IEnumerator WaitFor(int iterations)
{
for (var i = 0; i < iterations; i++)
{
yield return null;
}
}
/// <summary>
/// Provides a way to measure frame time within a scope.
/// </summary>
public struct ScopedFrameTimeMeasurement : IDisposable
{
private readonly FrameTimeMeasurement m_Test;
/// <summary>
/// Initializes a scoped frame time measurement.
/// </summary>
/// <param name="sampleGroup">Sample group used to store measurements.</param>
public ScopedFrameTimeMeasurement(SampleGroup sampleGroup)
{
var go = new GameObject("Recorder");
if (Application.isPlaying) Object.DontDestroyOnLoad(go);
m_Test = go.AddComponent<FrameTimeMeasurement>();
m_Test.SampleGroup = sampleGroup;
PerformanceTest.Disposables.Add(this);
}
/// <summary>
/// Initializes a scoped frame time measurement.
/// </summary>
/// <param name="name">Sample group name used to store measurements.</param>
public ScopedFrameTimeMeasurement(string name): this(new SampleGroup(name))
{
}
/// <summary>
/// Stops scoped frame time measurement and adds it to provided sample group.
/// </summary>
public void Dispose()
{
PerformanceTest.Disposables.Remove(this);
Object.DestroyImmediate(m_Test.gameObject);
}
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 9f2b7554f46f00c488dabcaaf44ed09e
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,14 @@
using UnityEngine;
namespace Unity.PerformanceTesting.Measurements
{
internal class FrameTimeMeasurement : MonoBehaviour
{
public SampleGroup SampleGroup;
void Update()
{
Measure.Custom(SampleGroup, Time.unscaledDeltaTime * 1000);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 2ba8bef1806543b47a718ff4d3cb0934
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,516 @@
using System;
using Unity.PerformanceTesting.Data;
using Unity.PerformanceTesting.Exceptions;
using Unity.PerformanceTesting.Meters;
using Unity.PerformanceTesting.Runtime;
using Unity.PerformanceTesting.Statistics;
using Unity.Profiling;
using UnityEngine;
namespace Unity.PerformanceTesting.Measurements
{
/// <summary>
/// Used as a helper class to sample execution time of methods. Uses fluent pattern to build and needs to be executed with Run method.
/// </summary>
public class MethodMeasurement
{
internal const int k_MeasurementCount = 9;
private const int k_MinMeasurementTimeMs = 100;
private const int k_MinWarmupTimeMs = 100;
private const int k_ProbingMultiplier = 4;
private const int k_MaxIterations = 10000;
internal const int k_MaxDynamicMeasurements = 1000;
private const double k_DefaultMaxRelativeError = 0.02;
private const ConfidenceLevel k_DefaultConfidenceLevel = ConfidenceLevel.L99;
private const OutlierMode k_DefaultOutlierMode = OutlierMode.Remove;
private readonly Action m_Action;
private ProfilerMarkerMeasurement m_ProfilerMarkerMeasurement = new ProfilerMarkerMeasurement(averageSampleMeasurement: true);
private ProfilerRecorder m_GCRecorder;
private Action m_Setup;
private Action m_Cleanup;
private SampleGroup m_SampleGroup = new SampleGroup("Time", SampleUnit.Millisecond, false);
private SampleGroup m_SampleGroupGC = new SampleGroup("Time.GC()", SampleUnit.Undefined, false);
private int m_WarmupCount;
private int m_MeasurementCount;
internal bool m_DynamicMeasurementCount;
private double m_MaxRelativeError = k_DefaultMaxRelativeError;
private ConfidenceLevel m_ConfidenceLevel = k_DefaultConfidenceLevel;
private OutlierMode m_OutlierMode = k_DefaultOutlierMode;
private int m_IterationCount = 1;
private bool m_GC;
private IStopWatch m_Watch;
/// <summary>
/// Initializes a method measurement.
/// </summary>
/// <param name="action">Method to be measured.</param>
public MethodMeasurement(Action action)
{
m_Action = action;
if (m_Watch == null) m_Watch = new StopWatch();
}
internal MethodMeasurement StopWatch(IStopWatch watch)
{
m_Watch = watch;
return this;
}
/// <summary>
/// Will record provided profiler markers once per frame.
/// </summary>
/// <param name="profilerMarkerNames">Profiler marker names as in profiler window.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement ProfilerMarkers(params string[] profilerMarkerNames)
{
if (profilerMarkerNames == null)
return this;
foreach (var marker in profilerMarkerNames)
{
var sampleGroup = new SampleGroup(marker, SampleUnit.Nanosecond, false);
m_ProfilerMarkerMeasurement.AddProfilerSample(sampleGroup);
}
return this;
}
/// <summary>
/// Will record provided profiler markers once per frame with additional control over the SampleUnit.
/// </summary>
/// <param name="sampleGroups">List of SampleGroups where a name matches the profiler marker and desired SampleUnit.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement ProfilerMarkers(params SampleGroup[] sampleGroups)
{
if (sampleGroups == null)
return this;
m_ProfilerMarkerMeasurement.AddProfilerSampleGroup(sampleGroups);
return this;
}
/// <summary>
/// Overrides the default SampleGroup of "Time".
/// </summary>
/// <param name="name">Desired name for measurement SampleGroup.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement SampleGroup(string name)
{
m_SampleGroup = new SampleGroup(name, SampleUnit.Millisecond, false);
m_SampleGroupGC = new SampleGroup(name + ".GC()", SampleUnit.Undefined, false);
return this;
}
/// <summary>
/// Overrides the default SampleGroup.
/// </summary>
/// <param name="sampleGroup">SampleGroup with your desired name and unit.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement SampleGroup(SampleGroup sampleGroup)
{
m_SampleGroup = sampleGroup;
m_SampleGroupGC = new SampleGroup(sampleGroup.Name + ".GC()", SampleUnit.Undefined, false);
return this;
}
/// <summary>
/// Count of times to execute before measurements are collected. If unspecified, a default warmup will be assigned.
/// </summary>
/// <param name="count">Count of warmup iterations to execute.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement WarmupCount(int count)
{
m_WarmupCount = count;
return this;
}
/// <summary>
/// Specifies the amount of method executions for a single measurement.
/// </summary>
/// <param name="count">Count of method executions.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement IterationsPerMeasurement(int count)
{
m_IterationCount = count;
return this;
}
/// <summary>
/// Specifies the number of measurements to take.
/// </summary>
/// <param name="count">Count of measurements to take.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement MeasurementCount(int count)
{
m_MeasurementCount = count;
return this;
}
/// <summary>
/// Dynamically find a suitable measurement count based on the margin of error of the samples.
/// The measurements will stop once a certain amount of samples (specified by a confidence interval)
/// falls within an acceptable error range from the result (defined by a relative error of the mean).
/// A default margin of error range of 2% and a default confidence interval of 99% will be used.
/// </summary>
/// <param name="outlierMode">Outlier mode allows to include or exclude outliers when evaluating the stop criterion.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement DynamicMeasurementCount(OutlierMode outlierMode = k_DefaultOutlierMode)
{
m_DynamicMeasurementCount = true;
m_OutlierMode = outlierMode;
return this;
}
/// <summary>
/// Dynamically find a suitable measurement count based on the margin of error of the samples.
/// The measurements will stop once a certain amount of samples (specified by a confidence interval)
/// falls within an acceptable error range from the result (defined by a relative error of the mean).
/// </summary>
/// <param name="maxRelativeError">The maximum relative error of the mean that the margin of error must fall into.</param>
/// <param name="confidenceLevel">The confidence interval which will be used to calculate the margin of error.</param>
/// <param name="outlierMode">Outlier mode allows to include or exclude outliers when evaluating the stop criterion.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement DynamicMeasurementCount(double maxRelativeError, ConfidenceLevel confidenceLevel = k_DefaultConfidenceLevel,
OutlierMode outlierMode = k_DefaultOutlierMode)
{
m_MaxRelativeError = maxRelativeError;
m_ConfidenceLevel = confidenceLevel;
m_DynamicMeasurementCount = true;
m_OutlierMode = outlierMode;
return this;
}
/// <summary>
/// Used to provide a cleanup method which will not be measured.
/// </summary>
/// <param name="action">Cleanup method to execute.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement CleanUp(Action action)
{
m_Cleanup = action;
return this;
}
/// <summary>
/// Used to provide a setup method which will run before the measurement.
/// </summary>
/// <param name="action">Setup method to execute.</param>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement SetUp(Action action)
{
m_Setup = action;
return this;
}
/// <summary>
/// Enables recording of the number of managed allocations made by the measured method - the "GC.Alloc" counter
/// in the Unity Profiler - as an additional sample group with a ".GC()" name postfix.
/// </summary>
/// <remarks>
/// <para>
/// The recorded value is a count of allocations, not an amount of bytes, and only allocations made on the
/// thread running the measurement are counted. It is reported per iteration when IterationsPerMeasurement is
/// used, and warmup, probing, SetUp and CleanUp are excluded from it.
/// Enabling this also forces a full blocking garbage collection immediately before each timed region, so that a
/// collection triggered by the measured method's own allocations does not land inside the timed region and show
/// up as periodic spikes across the samples. Because of that, the times reported for a method that allocates
/// are not directly comparable between a run with GC() and one without.
/// </para>
/// </remarks>
/// <returns>An updated instance of the MethodMeasurement to be used in fluent syntax.</returns>
public MethodMeasurement GC()
{
if (!m_GC)
{
// Create the GC Alloc recorder
m_GCRecorder = new ProfilerRecorder(ProfilerCategory.Memory, "GC.Alloc", 1,
ProfilerRecorderOptions.WrapAroundWhenCapacityReached | ProfilerRecorderOptions.SumAllSamplesInFrame | ProfilerRecorderOptions.CollectOnlyOnCurrentThread);
Debug.Assert(m_GCRecorder.Valid, "GC.Alloc ProfilerRecorder must be valid for the GC measurements");
m_GC = true;
}
return this;
}
/// <summary>
/// Executes the measurement with given parameters. When MeasurementCount is not provided, a probing method will run to determine desired measurement counts.
/// </summary>
public void Run()
{
ValidateCorrectDynamicMeasurementCountUsage();
SettingsOverride();
var settingsCount = RunSettings.Instance.MeasurementCount;
if (m_MeasurementCount > 0 || settingsCount > -1)
{
// Allocate sample groups up front to minimize GC allocations during/between tests
if (m_GC)
{
m_SampleGroupGC.Samples.Capacity = m_MeasurementCount;
var sampleGroupGC = PerformanceTest.GetSampleGroup(m_SampleGroupGC.Name);
if (sampleGroupGC == null)
{
PerformanceTest.AddSampleGroup(m_SampleGroupGC);
}
}
m_SampleGroup.Samples.Capacity = m_MeasurementCount;
var sampleGroup = PerformanceTest.GetSampleGroup(m_SampleGroup.Name);
if (sampleGroup == null)
{
PerformanceTest.AddSampleGroup(m_SampleGroup);
}
Warmup(m_WarmupCount);
RunForIterations(m_IterationCount, m_MeasurementCount, useAverage: false);
return;
}
if (m_DynamicMeasurementCount)
{
Warmup(m_WarmupCount);
RunForIterations(m_IterationCount);
return;
}
var iterations = Probing();
RunForIterations(iterations, k_MeasurementCount, useAverage: true);
}
private void ValidateCorrectDynamicMeasurementCountUsage()
{
if (!m_DynamicMeasurementCount)
return;
if (m_MeasurementCount > 0)
{
m_DynamicMeasurementCount = false;
Debug.LogWarning("DynamicMeasurementCount will be ignored because MeasurementCount was specified.");
}
}
/// <summary>
/// Overrides measurement count based on performance run settings
/// </summary>
private void SettingsOverride()
{
var count = RunSettings.Instance.MeasurementCount;
if (count < 0) { return; }
m_MeasurementCount = count;
m_WarmupCount = m_WarmupCount > 0 ? count : 0;
m_DynamicMeasurementCount = false;
}
private void RunForIterations(int iterations, int measurements, bool useAverage)
{
// Start marker recorders only for the measured iterations so warmup and probing samples
// do not contribute to the reported marker values
m_ProfilerMarkerMeasurement.StartRecording();
for (var j = 0; j < measurements; j++)
{
var executionTime = iterations == 1 ? ExecuteSingleIteration() : ExecuteForIterations(iterations);
if (useAverage) executionTime /= iterations;
var delta = Utils.ConvertSample(SampleUnit.Millisecond, m_SampleGroup.Unit, executionTime);
Measure.Custom(m_SampleGroup, delta);
}
DisableAndMeasureMarkers();
}
private void RunForIterations(int iterations)
{
// Start marker recorders only for the measured iterations so warmup and probing samples
// do not contribute to the reported marker values
m_ProfilerMarkerMeasurement.StartRecording();
while(true)
{
var executionTime = iterations == 1 ? ExecuteSingleIteration() : ExecuteForIterations(iterations);
var delta = Utils.ConvertSample(SampleUnit.Millisecond, m_SampleGroup.Unit, executionTime);
Measure.Custom(m_SampleGroup, delta);
if (SampleCountFulfillsRequirements())
break;
}
DisableAndMeasureMarkers();
}
private void DisableAndMeasureMarkers()
{
m_ProfilerMarkerMeasurement.SampleProfilerSamples(stopRecorders: true);
m_ProfilerMarkerMeasurement.Dispose();
m_ProfilerMarkerMeasurement = null;
m_GCRecorder.Dispose();
}
private bool SampleCountFulfillsRequirements()
{
var samples = m_SampleGroup.Samples;
var sampleCount = samples.Count;
var statistics = MeasurementsStatistics.Calculate(samples, m_OutlierMode, m_ConfidenceLevel);
var actualError = statistics.MarginOfError;
var maxError = m_MaxRelativeError * statistics.Mean;
if (sampleCount >= k_MeasurementCount && actualError < maxError)
return true;
if (sampleCount >= k_MaxDynamicMeasurements)
return true;
return false;
}
private int Probing()
{
var priorGC = m_GC;
m_GC = false;
var executionTime = 0.0D;
var iterations = 1;
if (m_WarmupCount > 0)
throw new PerformanceTestException(
"Please provide MeasurementCount or remove WarmupCount in your usage of Measure.Method");
while (executionTime < k_MinWarmupTimeMs)
{
executionTime = m_Watch.Split();
Warmup(iterations);
executionTime = m_Watch.Split() - executionTime;
if (executionTime < k_MinWarmupTimeMs)
{
iterations *= k_ProbingMultiplier;
}
}
if (iterations == 1)
{
ExecuteActionWithCleanupSetup();
ExecuteActionWithCleanupSetup();
m_GC = priorGC;
return 1;
}
var deisredIterationsCount =
Mathf.Clamp((int) (k_MinMeasurementTimeMs * iterations / executionTime), 1, k_MaxIterations);
m_GC = priorGC;
return deisredIterationsCount;
}
private void Warmup(int iterations)
{
var priorGC = m_GC;
m_GC = false;
for (var i = 0; i < iterations; i++)
{
ExecuteForIterations(m_IterationCount);
}
m_GC = priorGC;
}
private double ExecuteActionWithCleanupSetup()
{
m_Setup?.Invoke();
if (m_GC)
StartGCRecorder();
var executionTime = m_Watch.Split();
m_Action.Invoke();
executionTime = m_Watch.Split() - executionTime;
if (m_GC)
StopGCRecorder();
m_Cleanup?.Invoke();
return executionTime;
}
private double ExecuteSingleIteration()
{
m_Setup?.Invoke();
if (m_GC)
StartGCRecorder();
var executionTime = m_Watch.Split();
m_Action.Invoke();
executionTime = m_Watch.Split() - executionTime;
if (m_GC)
EndGCRecorderAndMeasure(1);
m_Cleanup?.Invoke();
return executionTime;
}
private double ExecuteForIterations(int iterations)
{
var executionTime = 0.0D;
if (m_Cleanup != null || m_Setup != null)
{
for (var i = 0; i < iterations; i++)
{
executionTime += ExecuteActionWithCleanupSetup();
}
if (m_GC)
EndGCRecorderAndMeasure(iterations);
}
else
{
if (m_GC)
StartGCRecorder();
executionTime = m_Watch.Split();
for (var i = 0; i < iterations; i++)
{
m_Action.Invoke();
}
executionTime = m_Watch.Split() - executionTime;
if (m_GC)
EndGCRecorderAndMeasure(iterations);
}
return executionTime;
}
private void StartGCRecorder()
{
// Collecting here, before the timed region opens, keeps the collection that the measured code's own
// allocations would otherwise trigger from landing inside it. Every caller opens a timed region right
// after this, so with SetUp or CleanUp - where each of the IterationsPerMeasurement inner executions is
// timed separately - this deliberately runs once per inner execution rather than once per sample.
// Fully qualified because this class has its own GC() method.
System.GC.Collect();
m_GCRecorder.Start();
}
private void StopGCRecorder()
{
m_GCRecorder.Stop();
}
private void EndGCRecorderAndMeasure(int iterations)
{
m_GCRecorder.Stop();
var gcAllocCount = m_GCRecorder.Count > 0 ? m_GCRecorder.GetSample(0).Count : 0;
Measure.Custom(m_SampleGroupGC, gcAllocCount / iterations);
m_GCRecorder.Reset();
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 573489ea329e0b246a993a3c9bcd9b5c
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,122 @@
using System;
using System.Collections.Generic;
using Unity.PerformanceTesting.Runtime;
using Unity.Profiling;
using UnityEngine;
namespace Unity.PerformanceTesting.Measurements
{
class ProfilerMarkerMeasurement : IDisposable
{
private readonly bool m_AverageSampleMeasurement;
bool m_DisposedValue;
bool m_RecordingStarted;
protected struct RecordedSampleGroup
{
public SampleGroup SampleGroup;
public ProfilerRecorder ProfilerRecorder;
}
protected readonly List<RecordedSampleGroup> m_SampleGroups = new List<RecordedSampleGroup>();
public ProfilerMarkerMeasurement(bool averageSampleMeasurement)
{
m_AverageSampleMeasurement = averageSampleMeasurement;
}
public void AddProfilerSampleGroup(IEnumerable<SampleGroup> sampleGroups)
{
foreach (var sampleGroup in sampleGroups)
{
AddProfilerSample(sampleGroup);
}
}
public void AddProfilerSample(SampleGroup sampleGroup)
{
m_SampleGroups.Add(new RecordedSampleGroup { SampleGroup = sampleGroup });
}
/// <summary>
/// Creates and starts the recorders for all registered sample groups. Samples produced by the
/// measured markers before this call are not part of the measurement, which allows warmup
/// iterations to be excluded from the reported values.
/// </summary>
public void StartRecording()
{
if (m_RecordingStarted)
return;
m_RecordingStarted = true;
for (var i = 0; i < m_SampleGroups.Count; i++)
{
var sampleGroup = m_SampleGroups[i];
sampleGroup.ProfilerRecorder = new ProfilerRecorder(sampleGroup.SampleGroup.Name, 1,
ProfilerRecorderOptions.WrapAroundWhenCapacityReached | ProfilerRecorderOptions.SumAllSamplesInFrame | ProfilerRecorderOptions.StartImmediately);
m_SampleGroups[i] = sampleGroup;
}
}
public void SampleProfilerSamples(bool stopRecorders = false)
{
foreach (var sampleGroup in m_SampleGroups)
{
// Validate that the recorder is attached to a valid marker. This check must stay ahead of
// any Stop/GetSample call: it also skips recorders that were never started (default handle),
// on which Stop would throw InvalidOperationException
if (!sampleGroup.ProfilerRecorder.Valid)
{
Debug.LogError($"ProfilerMarker measurement is attached to invalid marker \"{sampleGroup.SampleGroup.Name}\"! Ensure the marker is created at the time of the measurement");
continue;
}
if (stopRecorders)
sampleGroup.ProfilerRecorder.Stop();
var delta = 0.0;
// Record the last recorded value if present, otherwise use 0.
if (sampleGroup.ProfilerRecorder.Count > 0 && sampleGroup.ProfilerRecorder.GetSample(0).Count > 0)
{
var sampleCount = m_AverageSampleMeasurement ? sampleGroup.ProfilerRecorder.GetSample(0).Count : 1;
if (sampleCount > 0)
{
var totalTimeNs = sampleGroup.ProfilerRecorder.GetSample(0).Value / sampleCount;
delta = Utils.ConvertSample(SampleUnit.Nanosecond, sampleGroup.SampleGroup.Unit, totalTimeNs);
}
}
Measure.Custom(sampleGroup.SampleGroup, delta);
}
}
public void StopAndSampleRecorders() => SampleProfilerSamples(true);
protected virtual void Dispose(bool disposing)
{
if (m_DisposedValue)
return;
foreach (var sampleGroup in m_SampleGroups)
{
sampleGroup.ProfilerRecorder.Dispose();
}
m_DisposedValue = true;
}
~ProfilerMarkerMeasurement()
{
// Do not change this code. Put cleanup code in 'Dispose(bool disposing)' method
Dispose(disposing: false);
}
public void Dispose()
{
// Do not change this code. Put cleanup code in 'Dispose(bool disposing)' method
Dispose(disposing: true);
GC.SuppressFinalize(this);
}
}
}
@@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: aa2734fdf41ccae4d8e95492e9e75620
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:
@@ -0,0 +1,60 @@
using System;
using Unity.PerformanceTesting.Runtime;
using UnityEngine;
using Object = UnityEngine.Object;
namespace Unity.PerformanceTesting.Measurements
{
/// <summary>
/// Provides a way to measure profiler markers.
/// </summary>
public struct ProfilerMeasurement : IDisposable
{
private readonly ProfilerMeasurementBehaviour m_Test;
/// <summary>
/// Initializes a profiler marker measurement.
/// </summary>
/// <param name="sampleGroups">List of sample groups with name set to match profiler markers to be measured.</param>
public ProfilerMeasurement(SampleGroup[] sampleGroups)
{
if (sampleGroups == null)
{
m_Test = null;
return;
}
if (sampleGroups.Length == 0)
{
m_Test = null;
return;
}
var go = new GameObject("Recorder");
if (Application.isPlaying) Object.DontDestroyOnLoad(go);
go.hideFlags = HideFlags.HideAndDontSave;
m_Test = go.AddComponent<ProfilerMeasurementBehaviour>();
m_Test.AddProfilerSampleGroup(sampleGroups);
PerformanceTest.Disposables.Add(this);
}
/// <summary>
/// Initializes a profiler marker measurement.
/// </summary>
/// <param name="profilerMarkers">List of profiler markers to be measured.</param>
public ProfilerMeasurement(string[] profilerMarkers): this(Utils.CreateSampleGroupsFromMarkerNames(profilerMarkers))
{
}
/// <summary>
/// Stops profiler marker measurement and adds them to the provided sample group.
/// </summary>
public void Dispose()
{
PerformanceTest.Disposables.Remove(this);
if (m_Test == null) return;
m_Test.StopAndSampleRecorders();
Object.DestroyImmediate(m_Test.gameObject);
}
}
}
@@ -0,0 +1,3 @@
fileFormatVersion: 2
guid: 848e89034d174c77aae5f9f6a46b904b
timeCreated: 1617691772

Some files were not shown because too many files have changed in this diff Show More