Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

atf-c-api cannot expect both failure and timeout #76

Closed
jmmv opened this issue Feb 22, 2014 · 5 comments
Closed

atf-c-api cannot expect both failure and timeout #76

jmmv opened this issue Feb 22, 2014 · 5 comments
Labels
bug invalid

Comments

@jmmv
Copy link
Member

jmmv commented Feb 22, 2014

From asomers on January 16, 2014 18:34:38

What steps will reproduce the problem? 1. Create an ATF testcase in C that will both fail and timeout
2. Add atf_tc_expect_fail() and atf_tc_expect_timeout() statements to the test
3. Run it What is the expected output? What do you see instead? The expected output is something like "expected failure: blah blah blah". Instead, this is the output:
timeout_and_fail:expect_both -> failed: Test case was expecting a failure but none were raised [0.008s] What version of the product are you using? On what operating system? kyua 0.8

uname -a

FreeBSD alans-fbsd-head 11.0-CURRENT FreeBSD 11.0-CURRENT #23 r260786M: Thu Jan 16 13:06:43 MST 2014 alans@ns1.eng.sldomain.com:/vmpool/obj/usr/home/alans/freebsd/head/sys/GENERIC amd64 Please provide any additional information below. Attaching a test program that demonstrates the issues.

Attachment: timeout_and_fail.c Makefile Kyuafile

Original issue: http://code.google.com/p/kyua/issues/detail?id=76
@jmmv
Copy link
Member Author

jmmv commented Feb 22, 2014

From julio@meroh.net on January 24, 2014 15:20:53

Is this the same as issue 27 ?

@jmmv
Copy link
Member Author

jmmv commented Feb 22, 2014

From asomers on January 24, 2014 15:29:04

Nope. This test is deterministic. It always fails and always times out. The bug is that there isn't any way to set an expectation for that behavior.

@jmmv
Copy link
Member Author

jmmv commented Feb 22, 2014

From julio@meroh.net on January 24, 2014 15:38:11

Oh, I see. This is because you are using ATF_CHECK_* instead of ATF_REQUIRE_*; otherwise you'd never get to the timeout part because the test would abort right away.

Interesting, need to think about it some more.

Can you share some more details on the actual test you are writing, not just the simplified demonstration? Thanks.

@jmmv
Copy link
Member Author

jmmv commented Feb 22, 2014

From asomers on January 24, 2014 15:49:35

The actual test is in FreeBSD at sys/kern/unix_seqpacket_test:pipe_128k_8k. It times out because of kern/185812, which drops packets. And ATF_CHECK_EQ_MSG fails in the reader thread due to a miscompare, which is also due to the dropped packets.

@jmmv
Copy link
Member Author

jmmv commented Oct 13, 2014

ATF issue, so moved (again) to freebsd/atf#20 .

@jmmv jmmv closed this as completed Oct 13, 2014
@jmmv jmmv added the invalid label Oct 13, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug invalid
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jmmv