Failing .py tests, that I pass in "make check"

I just implemented Task 3.1.
When I run “run_tests.py” it’ll pass “public.statistics.brightness_small2” but nothing else.

HOWEVER!

If I use “make check”, it passes every test that includes “public.statistics…”

What am I missing, what is the problem?
Konsole

1 Like

Same for me over here :raised_hand:
I passed a lot tests using make check but only half of them with run_tests.py.
Weird… D:

image

Additional information: Public Tests on the Git Server do also pass as they did with make check. So, is run_tests.py broke, maybe?

The makefile extracts the reference images, builds the _opt, _debug, and testrunner binary, and calls the python script.
Therefore, calling the script manually should work the same way after building using make as make check.
This is also the behavior shown locally to me.

What could be the reason for not behaving like that? I mean, in theory it doesn’t matter for me, since I’m using the make command anyway. But I’d like to “fix” that issue :smiley:

I have the same error. My code passes only half of the public tests if tested with the Python script compared to testing with make check. But public tests on the git server are passed. For passed tests, I get the following in the Python script: FAIL: application did not return EXIT_SUCCESS – although my function return the values they should return (except void functions). :sweat_smile:

The problem in your screenshot here is that you execute the python script from the test folder and not the project root.

You have to be in the same folder as the makefile and run ./test/run_tests.py or python test/run_tests.py.

6 Likes

thanks, that should work :slight_smile: