Each programming problem has one or more test cases. Some test cases are samples included in the problem statement. Other test cases are hidden and only used to evaluate the program you submit.
There are two ways Toph may evaluate your submission. By default, if your program fails to produce the correct output for any test case, the evaluation process is stopped, and the verdict for that test case becomes the verdict for your submission. If all test case passes, the submission verdict will be “Accepted”.
Programming problems that allow partial scoring will work similarly, except the evaluation process will not stop after a failing test case. The submission verdict will still be the first failing test case verdict or “Accepted” if all test cases pass.
Verdicts that you may come across on Toph:
Accepted: “Accepted” is what you will be aiming for. It indicates that the program has passed all of the available test cases.
Wrong answer: The program failed to produce the correct output.
CPU limit exceeded: “CPU limit exceeded” indicates that your program took longer than expected to complete for any test case.
Memory limit exceeded: “Memory limit exceeded” indicates that your program attempted to consume more memory than allowed.
Output limit exceeded: “Output limit exceeded” indicates that your program tried to print way too much data, more than what is necessary to solve this problem.
Runtime error: The program crashed or returned a non-zero exit status.
Compilation error: The compiler returned an error when compiling your program, possibly because of syntax errors in the code.
Internal error: A submission may show “Internal error” when there is an issue with the programming problem’s tests or in the evaluation process. If you see “Internal error”, please let us know at support@toph.co.
