Do we get the first evaluation results before the second deadline?

As the title says.
It would be nice to know, if all my tests were correct or I missed something, before the project deadline, so that I can at least check my implementation with corrected tests.

The daily test runs will continue.
Therefore, you can check that your tests are sound.
You can not check completeness as daily tests do not fully cover the specification as stated in the project description.

You will also receive points for the evaluation run of the test exercise part soon.

You should now be able to see your points for the tests exercise.

1 Like

Are they evaluated with eval tests or only based on the daily tests?

The points on your status page reflect all tests (daily and eval) and will not change (we will not evaluate your tests with more tests).

Note of caution:
If you have full points, your tests are sufficient for our test evaluation suite. But by the nature of testing, even that does not guarantee the completeness of your tests. Your tests still might overlook some bugs.

Will we see the failing eval tests on the statistics page in the CMS like the times before? So that we can fix our implementation?

As in previous projects, you will get the names of all eval tests after the project is over.
But you will not get them earlier (there also will be no result E-Mail for the same reason).
However, by inspecting the points you get more additional information on whether you overlooked more bugs that were not tested in the daily tests.

However, by inspecting the points you get more additional information on whether you overlooked more bugs that were not tested in the daily tests.

Is the number of eval tests 13?

The number of evaluation tests is not relevant.
There are multiple cases:

  • You have covered every reasonable bug and have full points in part one
  • You have full points in part one but your tests suite is not able to cover some critical bugs that by chance our eval tests-tests missed
  • You have not full points but pass all daily tests => you know for sure that you are missing bugs with your test suite
  • You have not full points and do not pass all daily tests => there are more bugs that you miss and for some of them you know the area where they are