This commit introduces a few new parts:
- A new deqp & piglit runner
- New lists of previous results
- New ignore lists for unreliable tests
- A switch to using CTS provided test-list for deqp
The new runner supports a few different modes:
- Running a single test using the '-t' flag
- Comparing tests using the same driver with and without vtest using '--compare-backends'
- Running piglit or deqp suites using '--piglit' or '--deqp'
- Running suites of piglit or deqp tests using, '--gles2'/'--gles3','--gles31'
- Selecting a specific backend to use for example, '-b vtest-softpipe' or '-b softpipe'
For example:
# Run piglit gles2 tests using gpu&softpipe drivers through vtest
$ ci/run_test_suite.sh --piglit --gles2 --backend vtest-gpu --backend vtest-softpipe
# Run ignored&unreliable CTS gles3 tests using the softpipe driver
$ ci/run_test_suite.sh --deqp --gles3 --backend softpipe --unreliable
# Run CTS GLES2 using softpipe driver, and then compare results with results
# stored in ci/previous_results/, while ignoring known unreliable tests.
$ ci/run_test_suite.sh --piglit --gles2 --backend softpipe --compare-previous
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-by: Jakob Bornecrantz <jakob@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Also remove some trailing whitespace, while we're in here.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
check is more practical than distcheck since it doesn't
remove logs after the face, which allows us to always
keep logs. Even for non-failures.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
When the CI is run locally, the results directory has
to be cleared between runs in order to no exit early
or contain junk.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Apart from just doing a local run, this script enables
a local mesa repository to be used for the CI run.
For this to work the mesa repository is assumed to
be located in a directory next to the virgilrenderer.
$ ls -la
.
..
mesa
virgilrenderer
$ virgilrenderer/ci/run_ci_locally.sh
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
The only reason for wanting this is to lower the chance
of being unable to used a cached image incase of
changes to the Dockerfile.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
The selected subsets only contains tests that are currently passing with
a softpipe GL/GLES host.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
- rename options
- switch to softpipe since it supports up to GLES 3.1
- force host GL version 4.4 because otherwise we don't get GLES 3.1 in the
guest
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Define a CI run withing a specified docker image that uses pre-defined
versions of mesa, libdrm, and VK-GL_CTS.
The gitlab CI file .gitlab-ci.yml has been placed under ci/ in order to
avoid enabling the Ci automatically.
If the host provides a render device /dev/dri/render128D then forward
this device, use it as host device and run the dEQP GLES 2, 3, and 3.1
test suites as well as piglit by using vtest, a GL and a GLES
host context.
If this device is not available use the llvmpipe driver as host device
and run only dQEP GLES 2 with the GL and GLES host contexts.
The initial work for getting the CI running on normal hardware drivers
has been done by
Tomeu Vizoso <tomeu.vizoso@collabora.com> and
Jakob Bornecrantz <jakob@collabora.com>
Getting it to work with the llvmpipe host context was done by
Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Robert Foss
Gert Wollny