Writing a semi-public test?

[JedidiahTsang]

Hi there! Pretty new to using otter-grader, but I wanted to write a test that my students could run in order to determine whether they got the answer correct or not, but without leaking the actual solution (this specific question asks a student to select all correct options from a list of 11 statements, and place all correct options in an array).

My initial stab at the test looked something like:
// Hash the correct solution
correct_answer = make_array(1, 4, 7, 8, 9, 11)
correct_hash = hashlib.sha256(correct_answer.tobytes()).hexdigest()

// Check if the hash of answer_5a matches the correct hash
student_hash = hashlib.sha256(answer_5a.tobytes()).hexdigest()

student_hash == correct_hash

However, I found that when the student answers incorrectly, it'll actually show the contents of the test, line by line. My next try looked something like this:

import hashlib
correct_hash = '663ab247c7eca8e9f2e510654b74b431eb76275e5970ce414a0aef285844f0a7'
student_hash = hashlib.sha256(answer_5a.tobytes()).hexdigest()

student_hash == correct_hash

I also added one other test checking to see if the type of the answer was a numpy array.

Is this the best way to write semi-public tests? Thanks!

[chrispyles]

Yes, this is probably the best way to write public tests that check for correctness without giving away the answer. Since the public test files are available to students (even if the output of the test failure didn't print their contents), hashing is the only way that I can think of to include the correct answer without revealing it.