SURVEY:SUMMARY:BUILD_DIFFICULTY[not_applicable, reasonable_effort, code_problematic or string] not_applicable
SURVEY:SUMMARY:CLASSIFICATION[practical,theoretical,hardware] practical
SURVEY:SUMMARY:CORRECT_CODE_LOCATION[string] https://github.com/yaledb/calvin
SURVEY:SUMMARY:PUBLISHED_CODE[not_applicable, yes, no] yes
SURVEY:SUMMARY:SAME_VERSION[not_applicable, yes, no_but_available, no_and_not_available] yes
SURVEY:SUMMARY:STUDY_FOUND_CORRECT_CODE[not_applicable, yes, no] no
SURVEY:AUTHOR1:BUILD_COMMENT[string] 
SURVEY:AUTHOR1:BUILD_DIFFICULTY[not_applicable, reasonable_effort, code_problematic or string] not_applicable
SURVEY:AUTHOR1:BUILD_DIFFICULTY_COMMENT[string] none
SURVEY:AUTHOR1:CLASSIFICATION[practical,theoretical,hardware] practical
SURVEY:AUTHOR1:CLASSIFICATION_COMMENT[string] 
SURVEY:AUTHOR1:CORRECT_CODE_LOCATION[string] https://github.com/yaledb/calvin
SURVEY:AUTHOR1:PUBLIC_COMMENT[string] 
SURVEY:AUTHOR1:PUBLISHED_CODE[not_applicable, yes, no] yes
SURVEY:AUTHOR1:SAME_VERSION[not_applicable, yes, no_but_available, no_and_not_available] yes
SURVEY:AUTHOR1:SAME_VERSION_COMMENT[string] 
SURVEY:AUTHOR1:STUDY_FOUND_CORRECT_CODE[not_applicable, yes, no] no
SURVEY:AUTHOR2:BUILD_COMMENT[string] 
SURVEY:AUTHOR2:BUILD_DIFFICULTY[not_applicable, reasonable_effort, code_problematic or string] not_applicable
SURVEY:AUTHOR2:BUILD_DIFFICULTY_COMMENT[string] none
SURVEY:AUTHOR2:CLASSIFICATION[practical,theoretical,hardware] practical
SURVEY:AUTHOR2:CLASSIFICATION_COMMENT[string] 
SURVEY:AUTHOR2:CORRECT_CODE_LOCATION[string] 
SURVEY:AUTHOR2:PUBLIC_COMMENT[string] A 30-minute verification that an author's code compiles, runs, and prints the expected output is interesting but is not a convincing test of scientific reproducibility.<br/><br/>A better test of reproducibility of a publication's results would be to independently implement and measure the algorithm or system described. This would be unlikely to replicate any bugs or systematic measurement errors, increasing the meaningfulness of a positive reproducibility result. Negative results would also be more likely to provide technical insight and foster scientific discussion.<br/><br/>Note that publishing code is also a rather thankless task. Likely outcomes include:<br/>(a) Nobody will look at it and you will have wasted your time.<br/>(b) Anyone who looks at it will see that it perfectly matches the paper's specification (except for being MUCH harder to read) and won't have learned anything.<br/>(c) Someone will try to use it, run into some kind of trouble, and bother you with annoying questions.<br/>(d) Someone will catch some trivial bug, approximation, or limitation and use it to cast doubt on your results.<br/>(e) Some professor will convince a naive first-year student to use your code as a starting point for their experiments and you will have contributed to that poor student's misery and anguish.
SURVEY:AUTHOR2:PUBLISHED_CODE[not_applicable, yes, no] no
SURVEY:AUTHOR2:SAME_VERSION[not_applicable, yes, no_but_available, no_and_not_available] not_applicable
SURVEY:AUTHOR2:SAME_VERSION_COMMENT[string] none
SURVEY:AUTHOR2:STUDY_FOUND_CORRECT_CODE[not_applicable, yes, no] not_applicable