57 lines
2.0 KiB
Markdown
57 lines
2.0 KiB
Markdown
Benchmark tests are to use `PHPUnit` as integration platform and do not always
|
|
represent the best tool for a performance comparison (as it depends on environmental
|
|
factors such as hardware and software constraints which might not be under the
|
|
control of the tester) but it can help to identify performance regressions among
|
|
newly introduced features that run with the same environmental specification.
|
|
|
|
Benchmarks are not performed in isolation and therefore run in concert with the
|
|
`MediaWiki` infrastructure to determine the overall performance impact during
|
|
execution.
|
|
|
|
When using `git`, it is relatively easy to run tests and see if a change
|
|
introduces a significant regression or improvement in terms of performance over
|
|
the existing `master` branch by comparing test results of the `master` against
|
|
a `feature` branch.
|
|
|
|
## Designing benchmark tests
|
|
|
|
The definition of what benchmarks are executed is specified by a `JSONScript`
|
|
found in the `TestCases` directory. Supported types are:
|
|
|
|
- `import` to import data from an external source
|
|
- `contentCopy` copy content from an internal source
|
|
- `editCopy` edit content from an internal source
|
|
- `job` running selected jobs
|
|
- `maintenance` running selected maintenance scripts
|
|
- `query` executing `#ask` queries
|
|
|
|
## Running benchmark tests
|
|
|
|
Running `composer benchmark` from the Semantic MediaWiki base directory should
|
|
output something similar to what can be seen below.
|
|
|
|
```
|
|
- mediawiki: "1.28.0-alpha"
|
|
- semantic-mediawiki: "2.5.0-alpha"
|
|
- environment: {"store":"SMWSQLStore3","db":"mysql"}
|
|
- benchmarks
|
|
- 35a205a6fa1db2cda4c484d3007953b3
|
|
- type: "import"
|
|
- source: "import-001.xml"
|
|
- memory: 5564360
|
|
- time: {"sum":5.9888241}
|
|
- 054543b5702e6fcbccafd00bf6dd27ac
|
|
- type: "contentCopy"
|
|
- source: "import-001.xml"
|
|
- import
|
|
- memory: 351392
|
|
- time: {"sum":0.900929}
|
|
- copy
|
|
- copyFrom: "Lorem ipsum"
|
|
- copyCount: 10
|
|
- memory: 75056
|
|
- time: {"sum":7.9295225,"mean":0.7929523,"sd":0.1008703,"norm":0.0792952}
|
|
- time: 8.8402690887451
|
|
|
|
```
|