Testing your distribution with LAVA

Andrej Shadura, Guillaume Tucker

Collabora

25 August 2018

Presenter Notes

What is LAVA?

Presenter Notes

  • Linaro Automated Validation Architecture
  • Runs your tests on the target hardware

Presenter Notes

LAVA overview

  • Power on a device
  • Boot into the target OS
  • Run tests
  • Collect results

Presenter Notes

LAVA overview

Architecture overview

Presenter Notes

LAVA test labs

Kevin

Presenter Notes

LAVA test labs

Cambridge

Presenter Notes

LAVA test labs

Sjoerd

Presenter Notes

LAVA jobs

Pipelines defined in YAML:

  • boot
  • deploy
  • test

Jobs specify high-level details only

LAVA ‘knows’ the low-level details

Presenter Notes

KernelCI

  • Build all new commits for a number of architectures
  • Boot them on the actual hardware
  • Report all failures

Presenter Notes

KernelCI

KernelCI architecture

Presenter Notes

Why use Kernel CI

  • Detect failures early
  • Test on many devices
  • Test branches before they’re merged
  • Automatic bisection (WIP)

Presenter Notes

How to use LAVA

 1 device_type: bcm2836-rpi-2-b
 2 job_name: boot test
 3 visibility: public
 4 
 5 timeouts:
 6   job:
 7     minutes: 10
 8 
 9 actions:
10 - deploy: 
11 - boot: 
12 - test: 

Presenter Notes

Deploy

 1 - deploy:
 2     to: tftp
 3     kernel:
 4       url: https://server.org/path/to/zImage
 5       type: zimage
 6     dtb:
 7       url: https://server.org/path/to/raspberry-pi-2b.dtb
 8     ramdisk:
 9       url: https://server.org/path/to/ramdisk-armhf.cpio.gz
10       compression: gz
11     os: oe

Presenter Notes

Boot

1 - boot:
2     method: u-boot
3     commands: ramdisk
4     auto_login:
5       login_prompt: 'root login:'
6       username: root
7     prompts: ['# ']

Presenter Notes

Test

 1 - test:
 2     timeout:
 3       minutes: 15
 4     name: sanity-check
 5     definitions:
 6       - repository: https://git.server.org/tests/tests
 7         revision: master
 8         from: git
 9         path: sanity-check.yaml
10         name: sanity-check

Presenter Notes

More complex setups

  • Need to boot a full OS image?
  • Two-stage setup:
    1. Boot minimal OS
    2. Flash the image
    3. Reboot into the image

Presenter Notes

Deploy the second stage

 1 - deploy:
 2     namespace: system
 3     timeout:
 4       minutes: 20
 5     to: usb
 6     device: sd-card
 7     tool:
 8       prompts: ['copying time: [0-9ms\.\ ]+,
 9         copying speed [0-9\.]+ MiB\/sec']
10     images:
11       image:
12         url: "https://…/apertis_18.09-target-arm64-\
13             uboot_20180801.0.img.gz"
14       bmap:
15         url: "https://…/apertis_18.09-target-arm64-\
16             uboot_20180801.0.img.bmap"

Presenter Notes

Deploy the second stage (part 2)

1     os: ubuntu
2     writer:
3       tool: /usr/bin/bmaptool
4       options: copy {DOWNLOAD_URL} {DEVICE}
5       prompt: 'bmaptool: info'

Presenter Notes

How to define tests?

Presenter Notes

YAML!

Presenter Notes

Test definition: metadata

 1 metadata:
 2   name: check-dbus-services
 3   format: "Lava-Test-Shell Test Definition 1.0"
 4   description: "Sanity-check all installed D-Bus services"
 5   maintainer: "simon.mcvittie@collabora.co.uk"
 6   scope:
 7   - functional
 8   devices:
 9   - i386
10   environment:
11   - lava-test-shell

Presenter Notes

Test definition: dependencies

1 install:
2   deps:
3   - apertis-tests
4   - dbus-tests

Presenter Notes

Test definition: run stuff

1 run:
2   steps:
3   - common/run-test-in-systemd --user=user
4         dbus/check-dbus-services
5   - common/run-test-in-systemd dbus/check-dbus-services

Presenter Notes

Test definition: results parsing

1 parse:
2   pattern: 'RESULT:(?P<result>\w+):(?P<test_case_id>[^:]+):'
3   # LAVA doesn't seem to have the concept of
4   # an expected failure, so calling it skipped
5   # is the next best thing
6   fixupdict:
7     xfail: skip

Presenter Notes

Full CI loop with LAVA

LAVA CI loop

Presenter Notes

Thanks to Guillaume Tucker

Presenter Notes

Presenter Notes