To try this out, start a run like using multiple `--test`
arguments.
ci/run_test_suite.sh \
--deqp \
--host-gles \
--backend vtest-gpu \
--backend gpu \
--test TEST1 \
--test TEST2
Signed-off-by: Robert Foss <robert.foss@collabora.com>
This patch cleans up the CI path handling, and
allows for external path variables to be used.
Environment variables are now respected for MESA_PATH,
VIRGL_PATH and CTS_PATH.
Additionally these variables can be picked up from a
config file `~/.virgl-ci.conf`
$ cat ~/.virgl-ci.conf
MESA_PATH=/opt/virgl/mesa
VIRGL_PATH=/opt/virgl/virglrenderer
Or given as a local environment variable like below.
CTS_PATH=/opt/virgl/ ci/run_test_suite.sh --piglit --gles3 --backend softpipe
Signed-off-by: Robert Foss <robert.foss@collabora.com>
gdc-6 is currently not in testing and gdc will pull in the default version
(currently gdc-8) which should work.
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
This reason for wanting to split them up is just to
provide a faster overview of what is failing and
what isn't.
This may be a small amount slower than running them
together, but the impact should be low.
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Make the script names easier to remember by harmonizing the names
to use underscores instead of hyphens.
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
These lists are no longer used, and should be pulled
in from the CTS directly either way.
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
This commit introduces a few new parts:
- A new deqp & piglit runner
- New lists of previous results
- New ignore lists for unreliable tests
- A switch to using CTS provided test-list for deqp
The new runner supports a few different modes:
- Running a single test using the '-t' flag
- Comparing tests using the same driver with and without vtest using '--compare-backends'
- Running piglit or deqp suites using '--piglit' or '--deqp'
- Running suites of piglit or deqp tests using, '--gles2'/'--gles3','--gles31'
- Selecting a specific backend to use for example, '-b vtest-softpipe' or '-b softpipe'
For example:
# Run piglit gles2 tests using gpu&softpipe drivers through vtest
$ ci/run_test_suite.sh --piglit --gles2 --backend vtest-gpu --backend vtest-softpipe
# Run ignored&unreliable CTS gles3 tests using the softpipe driver
$ ci/run_test_suite.sh --deqp --gles3 --backend softpipe --unreliable
# Run CTS GLES2 using softpipe driver, and then compare results with results
# stored in ci/previous_results/, while ignoring known unreliable tests.
$ ci/run_test_suite.sh --piglit --gles2 --backend softpipe --compare-previous
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-By: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Reviewed-by: Jakob Bornecrantz <jakob@collabora.com>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Also remove some trailing whitespace, while we're in here.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
check is more practical than distcheck since it doesn't
remove logs after the face, which allows us to always
keep logs. Even for non-failures.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
When the CI is run locally, the results directory has
to be cleared between runs in order to no exit early
or contain junk.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Apart from just doing a local run, this script enables
a local mesa repository to be used for the CI run.
For this to work the mesa repository is assumed to
be located in a directory next to the virgilrenderer.
$ ls -la
.
..
mesa
virgilrenderer
$ virgilrenderer/ci/run_ci_locally.sh
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
The only reason for wanting this is to lower the chance
of being unable to used a cached image incase of
changes to the Dockerfile.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Robert Foss <robert.foss@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
The selected subsets only contains tests that are currently passing with
a softpipe GL/GLES host.
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
- rename options
- switch to softpipe since it supports up to GLES 3.1
- force host GL version 4.4 because otherwise we don't get GLES 3.1 in the
guest
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Jakob Bornecrantz <jakob@collabora.com>
Define a CI run withing a specified docker image that uses pre-defined
versions of mesa, libdrm, and VK-GL_CTS.
The gitlab CI file .gitlab-ci.yml has been placed under ci/ in order to
avoid enabling the Ci automatically.
If the host provides a render device /dev/dri/render128D then forward
this device, use it as host device and run the dEQP GLES 2, 3, and 3.1
test suites as well as piglit by using vtest, a GL and a GLES
host context.
If this device is not available use the llvmpipe driver as host device
and run only dQEP GLES 2 with the GL and GLES host contexts.
The initial work for getting the CI running on normal hardware drivers
has been done by
Tomeu Vizoso <tomeu.vizoso@collabora.com> and
Jakob Bornecrantz <jakob@collabora.com>
Getting it to work with the llvmpipe host context was done by
Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Robert Foss
Gert Wollny