Riot: examples / tests: LoRa tests fail on platforms that don't support LoRa

Created on 15 Jul 2020 · 3Comments · Source: RIOT-OS/RIOT

Description

The tests introduced in #11915 fail for non-LoRa platforms. See also https://github.com/RIOT-OS/Release-Specs/issues/156#issuecomment-658250837 task 2 and 3.

Steps to reproduce the issue

$ BOARD=iotlab-m3 make -C examples/lorawan/ flash
...
$ BOARD=iotlab-m3 make -C examples/lorawan/ test

$ BOARD=iotlab-m3 make -C tests/pkg_semtech-loramac/ flash
...
$ BOARD=iotlab-m3 make -C tests/pkg_semtech-loramac/ test

Expected results

The tests are ignored or marked as expected to fail.

Actual results

The tests fail.

Versions

2020.07-RC1

Operating System Environment
----------------------------
         Operating System: "Arch Linux" 
                   Kernel: Linux 5.7.7-arch1-1 x86_64 unknown
             System shell: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
             make's shell: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)

Installed compiler toolchains
-----------------------------
               native gcc: gcc (GCC) 10.1.0
        arm-none-eabi-gcc: arm-none-eabi-gcc (Arch Repository) 10.1.0
                  avr-gcc: missing
         mips-mti-elf-gcc: missing
               msp430-gcc: missing
     riscv-none-embed-gcc: missing
     xtensa-esp32-elf-gcc: missing
   xtensa-esp8266-elf-gcc: missing
                    clang: clang version 10.0.0 

Installed compiler libs
-----------------------
     arm-none-eabi-newlib: "3.3.0"
      mips-mti-elf-newlib: missing
  riscv-none-embed-newlib: missing
  xtensa-esp32-elf-newlib: missing
xtensa-esp8266-elf-newlib: missing
                 avr-libc: missing (missing)

Installed development tools
---------------------------
                   ccache: missing
                    cmake: cmake version 3.17.3
                 cppcheck: missing
                  doxygen: 1.8.18
                      git: git version 2.27.0
                     make: GNU Make 4.3
                  openocd: Open On-Chip Debugger 0.10.0+dev-01089-g3bfe49266 (2020-02-26-14:18)
                   python: Python 3.8.3
                  python2: Python 2.7.18
                  python3: Python 3.8.3
                   flake8: 3.8.3 (mccabe: 0.6.1, pycodestyle: 2.6.0, pyflakes: 2.2.0) CPython 3.8.3 on
               coccinelle: missing

LoRa tests bug

Source

miri64

Most helpful comment

It's just that compile_and_test_for_board.py doesn't take this into account apparently.

Yes, for other (e.g. driver) tests that works however.

And for other drivers it fails or triggers hardfaults like groove_ledbar, I think this test is just one of many that fall under the issues partially addressed in #11954, and this one in particular is addressed perfectly by that, its a PR I would like to revive if it gets more support.