Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest: use records to store RIOT version and experiment ID of test #188

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

miri64
Copy link
Member

@miri64 miri64 commented Aug 5, 2020

I noticed that I had this still lying around from before my vacation. This should make it easier to track after the fact which

  1. RIOT version was used for a given test and
  2. Which IoT-LAB experiment a test ran in.

Testing procedures

Run

tox  -- -k "(spec03 or spec04) and task01" --non-RC

and after success (given that you have everything set-up.. the tests should not skip ;-))

junit2html test-report.xml test-report.html

and open the test-report.html in a browser (you can of course also read just the XML and skip the junit2html step). The new properties should show up:

exp_id
225847
riot_commit
075b35a71a (HEAD -> gh-actions/enh/release-tests, origin/gh-actions/enh/release-tests) fixup! gh-actions: add release-specs tests

@miri64
Copy link
Member Author

miri64 commented Aug 5, 2020

(this is based on an offline suggestion by @MrKevinWeiss)

@miri64
Copy link
Member Author

miri64 commented Aug 5, 2020

Mh after re-running the tests I get warnings

03-single-hop-ipv6-icmp/test_spec03.py::test_task01[nodes0]
  03-single-hop-ipv6-icmp/test_spec03.py:25: PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
    @pytest.mark.skipif(not bridged(["tap0", "tap1"]),

04-single-hop-6lowpan-icmp/test_spec04.py::test_task01[nodes0]
  04-single-hop-6lowpan-icmp/test_spec04.py:22: PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
    @pytest.mark.iotlab_creds

-- Docs: https://docs.pytest.org/en/stable/warnings.html

@MrKevinWeiss do you know of an xunit2 compliant way to do this?

@miri64
Copy link
Member Author

miri64 commented Aug 6, 2020

Rebased to current master and also added the Release-Specs version as a property.

@MrKevinWeiss
Copy link
Contributor

@MrKevinWeiss do you know of an xunit2 compliant way to do this?

I don't know about this but I like the idea of the PR.

@MrKevinWeiss
Copy link
Contributor

Do we now have a way to get this info or should we continue with this?

@miri64
Copy link
Member Author

miri64 commented Jul 7, 2021

Do we now have a way to get this info or should we continue with this?

If we can live with the warning, I guess we can continue. Alternatively, we of course also could choose a different output format than xunit2 (which according to my last google search on that topic seems to be pytest specific anyways?)

@MrKevinWeiss
Copy link
Contributor

Or we could just use junit_family=legacy, was there a reason that we went with xunit2?

@MrKevinWeiss
Copy link
Contributor

I will give it a try and see what breaks.

@miri64
Copy link
Member Author

miri64 commented Jul 7, 2021

Or we could just use junit_family=legacy, was there a reason that we went with xunit2?

I think there were warnings to use xunit2 ^^.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants