70 %
Chris Biscardi

I have some JSON with a ton of test cases, how do I use that in Rust?

OK, so let's imagine you have a file called fixtures.json that holds a bunch of test cases from a spec and looks like this.

fixtures.json
json
[
{
"markdown": "\tfoo\tbaz\t\tbim\n",
"html": "<pre><code>foo\tbaz\t\tbim\n</code></pre>\n",
"example": 1,
"start_line": 352,
"end_line": 357,
"section": "Tabs"
},
{
"markdown": " \tfoo\tbaz\t\tbim\n",
"html": "<pre><code>foo\tbaz\t\tbim\n</code></pre>\n",
"example": 2,
"start_line": 359,
"end_line": 364,
"section": "Tabs"
},
...
]

This file has 10, 100, or even 1000+ of these test cases for a new parser you're writing. What do we not want to do? Any sort of manual process. Luckily Rust has a crate for this, it's called datatest.

commonmark_tests.rs
rust
#![feature(custom_test_frameworks)]
#![test_runner(datatest::runner)]
use serde::Deserialize;
use std::fmt;
#[derive(Deserialize)]
struct CommonmarkTestCase {
markdown: String,
html: String,
example: usize,
start_line: usize,
end_line: usize,
section: String
}
impl fmt::Display for CommonmarkTestCase {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "example: {} ({})", self.example, self.section)
}
}
#[datatest::data("tests/fixtures.json")]
fn sample_test(case: CommonmarkTestCase) {
assert_eq!(case.html, case.markdown);
}

The important bit here is that we create a struct to represent the test cases CommonmarkTestCase that can deserialize using serde. This allows us to call all of the tests using datatest::data("tests/fixtures.json").

Each test is named according to the Display trait, so we implement it to give some nicer test output.

output.txt
txt
commonmark_test::example: 9 (Tabs) (line 66)
commonmark_test::example: 90 (Fenced code blocks) (line 714)
commonmark_test::example: 91 (Fenced code blocks) (line 722)
commonmark_test::example: 92 (Fenced code blocks) (line 730)
commonmark_test::example: 93 (Fenced code blocks) (line 738)
commonmark_test::example: 94 (Fenced code blocks) (line 746)
commonmark_test::example: 95 (Fenced code blocks) (line 754)
commonmark_test::example: 96 (Fenced code blocks) (line 762)
commonmark_test::example: 97 (Fenced code blocks) (line 770)
commonmark_test::example: 98 (Fenced code blocks) (line 778)
commonmark_test::example: 99 (Fenced code blocks) (line 786)

And a specific failing test looks like:

failing-test.txt
txt
---- commonmark_test::example: 92 (Fenced code blocks) (line 730) stdout ----
thread 'commonmark_test::example: 92 (Fenced code blocks) (line 730)' panicked at 'assertion failed: `(left == right)`
left: `"<pre><code>aaa\n~~~\n</code></pre>\n"`,
right: `"```\naaa\n~~~\n```\n"`', tests/commonmark.rs:27:3

Woah that's a lot of failing tests

yeah, if you're not ready to face them, throw them behind a conditional compilation flag:

rust
#[cfg(feature = "commonmark")]
#[datatest::data("tests/fixtures.json")]
fn commonmark_test(case: CommonmarkTestCase) {
assert_eq!(case.html, case.markdown);
}