Concerning eval tests of project


I am not able to find the bug in my code that caused all POSITIVE eval tests to fail except for the scheme test (all NEGATIVE tests work). My code has passed a solid amount of tests (over 200), roughly half of which were testing “positive” input strings (I assume that positive means that the string is a valid URI). My question now is: Are the eval tests acting differently than plugging in some input string and checking whether the outcome is equal to the respective part of the URI as a string? If not, I am quite puzzled with respect to the probability that my code fails on ALL inputs while passing all my own tests.

I might add that I implemented the part after the scheme (insofar, remember, my code seems to work) by using the JAVA.util.stack to create an abstract machine. But this should be no problem since it works in the VM, right?

I would highly appreciate some ideas on what could go wrong… (and learn from that)

Thanks a lot!

The quantity of tests does not matter as much as the quality of tests.
For instance, all tests of the form

[some text]://

are basically the same.
But you are missing more advanced tests.
For instance, there is no test in your test suite that checks whether the scheme in a URI with anything behind the scheme delimiter :// is correct.

A test suite with 20-40 tests should be able to catch most bugs.
On the other hand, 1000 tests might miss many bugs if they are quite similar to each other.

Well, what can I say, thanks a lot. I thought a “modular” approach would be sufficient, however, this way I missed (at least) one stupid bug in my code: the storage of the parsed scheme in my parsed object “dropped” INSIDE an if branch which checks whether there is no remainder left to parse (and in that case returns the object), while the storage should obviously be independent thereof (hence to be executed BEFORE the if statement). Thus, every time the input string contains more than a scheme I will not reset the initialization value; if you now check the correctness of the scheme in any case it makes perfect sense that my code breaks any time except for the case where a string containing only a scheme is to be parsed.

In any case, that blindspot really hurts but I think it was educational;) thanks again!

1 Like