This commit introduces a few new parts:
- A new deqp & piglit runner
- New lists of previous results
- New ignore lists for unreliable tests
- A switch to using CTS provided test-list for deqp
The new runner supports a few different modes:
- Running a single test using the '-t' flag
- Comparing tests using the same driver with and without vtest using '--compare-backends'
- Running piglit or deqp suites using '--piglit' or '--deqp'
- Running suites of piglit or deqp tests using, '--gles2'/'--gles3','--gles31'
- Selecting a specific backend to use for example, '-b vtest-softpipe' or '-b softpipe'
For example:
# Run piglit gles2 tests using gpu&softpipe drivers through vtest
$ ci/run_test_suite.sh --piglit --gles2 --backend vtest-gpu --backend vtest-softpipe
# Run ignored&unreliable CTS gles3 tests using the softpipe driver
$ ci/run_test_suite.sh --deqp --gles3 --backend softpipe --unreliable
# Run CTS GLES2 using softpipe driver, and then compare results with results
# stored in ci/previous_results/, while ignoring known unreliable tests.
$ ci/run_test_suite.sh --piglit --gles2 --backend softpipe --compare-previous
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
The selected subsets only contains tests that are currently passing with
a softpipe GL/GLES host.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Robert Foss
Gert Wollny