To try this out, start a run like using multiple `--test`
arguments.
ci/run_test_suite.sh \
--deqp \
--host-gles \
--backend vtest-gpu \
--backend gpu \
--test TEST1 \
--test TEST2
Signed-off-by: Robert Foss <robert.foss@collabora.com>
This patch cleans up the CI path handling, and
allows for external path variables to be used.
Environment variables are now respected for MESA_PATH,
VIRGL_PATH and CTS_PATH.
Additionally these variables can be picked up from a
config file `~/.virgl-ci.conf`
$ cat ~/.virgl-ci.conf
MESA_PATH=/opt/virgl/mesa
VIRGL_PATH=/opt/virgl/virglrenderer
Or given as a local environment variable like below.
CTS_PATH=/opt/virgl/ ci/run_test_suite.sh --piglit --gles3 --backend softpipe
Signed-off-by: Robert Foss <robert.foss@collabora.com>
This commit introduces a few new parts:
- A new deqp & piglit runner
- New lists of previous results
- New ignore lists for unreliable tests
- A switch to using CTS provided test-list for deqp
The new runner supports a few different modes:
- Running a single test using the '-t' flag
- Comparing tests using the same driver with and without vtest using '--compare-backends'
- Running piglit or deqp suites using '--piglit' or '--deqp'
- Running suites of piglit or deqp tests using, '--gles2'/'--gles3','--gles31'
- Selecting a specific backend to use for example, '-b vtest-softpipe' or '-b softpipe'
For example:
# Run piglit gles2 tests using gpu&softpipe drivers through vtest
$ ci/run_test_suite.sh --piglit --gles2 --backend vtest-gpu --backend vtest-softpipe
# Run ignored&unreliable CTS gles3 tests using the softpipe driver
$ ci/run_test_suite.sh --deqp --gles3 --backend softpipe --unreliable
# Run CTS GLES2 using softpipe driver, and then compare results with results
# stored in ci/previous_results/, while ignoring known unreliable tests.
$ ci/run_test_suite.sh --piglit --gles2 --backend softpipe --compare-previous
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Robert Foss