Assertion reporting

This post is part of an ongoing series about tests in Rust:

Little recap

In the last post, we basically designed Assertion so we would use it like this.

let some_value = String::from("foo") + "bar";
Assertion::from(&some_value).is_equal_to("foobar");

or, since we provided a short macro for building Assertion.

let some_value = String::from("foo") + "bar";
assert_that!(some_value).is_equal_to("foobar");

And remember, we can chain assertions too. For example, let’s say we know our function life() should never return 0 nor 42.

let life = life();
assert_that!(life)
    .is_not_equal_to(0)
    .is_not_equal_to(42);

So far, the actual assertion will panic if one of the condition is not met. But since we’re chaining assertions, it might actually be useful to know the results for the entire chain instead of failing on the first incorrect one.

Storing the results

This should actually be pretty easy to store all the results since we’re having the Assertion object all along. We can stick a Vec into Assertion. This Vec will contain a Report that needs a few information:

  • did the assertion pass?

  • what was the test (so we can report to the end-user)?

  • what was the tested value?

  • what was the compared value?

#[derive(Debug)]
struct Report {
    pass: bool,
    condition: String,
    value: String,
    compared: String,
}

#[derive(Debug)]
pub struct Assertion<'v, V>
where
    V: std::fmt::Debug,
{
    value: &'v V,
    reports: Vec<Report>,
}

We can now implement is_equal_to() as follows.

pub fn is_equal_to<C>(mut self, compared: C) -> Self
where
    V: PartialEq<C> + std::fmt::Debug,
    C: std::fmt::Debug,
{
    let value: &'v V = self.value;
    let pass = value.eq(&compared);
    let reporter = crate::Report {
        pass,
        condition: "be equal to".to_string(),
        value: format!("{:?}", value),
        compared: format!("{:?}", compared),
    };
    self.reports.push(reporter);
    self
}

There is a few things going on. First of all, we changed the signature of the function is_equal_to so it now takes ownership of self. This will allow us to mutate it.[1]

We also transformed value and compared into their Debug formatted string.

And know, we can try to run the following to see how it goes.

let life = life();
assert_that!(life).is_not_equal_to(42);

And it passes. Great! Hmm, just to be sure, does this fails?

let life = 42;
assert_that!(life).is_not_equal_to(42);

What, it pass too? Something is wrong. And indeed, remember how we had the following in is_equal_to() implementation before…​ but we do not have it anymore!

assert!(t.eq(&expected));

This is the line that made the test panic or not, and therefore the test pass or fail.

Failing again

[2] Where do we insert the assert!() back into our code? We cannot put it in is_equal_to nor is_not_equal_to as this would defeat the purpose of not failing early. However, there is one piece of code that is always executed when a object goes out of scope: Drop.

Let’s try it.

impl<V> std::ops::Drop for Assertion<'_, V>
where
    V: std::fmt::Debug,
{
    fn drop(&mut self) {
        let nb_fails = self.reports.iter().filter(|r| r.pass == false).count();
        let message = self.reports.iter().fold(String::from("\n"), |msg, r| {
            let emoji = if r.pass { '✓' } else { '⨯' };
            msg + &format!(
                "\t{} should be {} `{}`, was `{}`\n",
                emoji, r.condition, r.compared, r.value,
            )
        });
        assert!(nb_fails == 0, "{}", message);
    }
}

Does it work? Let try back the following code.

let life = 42;
assert_that!(life).is_not_equal_to(42);

Here the output result. It does work!

stderr:
thread 'main' panicked at '
    ⨯ should not be equal to `42`, was `42`
'

More information in the report

We can add a little more useful information about where the assertion failed thanks to the file!() and line!() macros.

Let’s make these modifications to Assertion.

#[derive(Debug)]
struct Location {
    file: String,
    line: u32,
}

#[derive(Debug)]
pub struct Assertion<'v, V>
where
    V: std::fmt::Debug,
{
    value: &'v V,
    location: Option<Location>,
    reports: Vec<Report>,
}

impl<'v, V> From<&'v V> for Assertion<'v, V>
where
    V: 'v + std::fmt::Debug,
{
    fn from(value: &'v V) -> Self {
        Assertion {
            value,
            location: None,
            reports: Vec::new(),
        }
    }
}

impl<'v, V> Assertion<'v, V>
where
    V: 'v + std::fmt::Debug,
{
    pub fn with_location(mut self, file: &str, line: u32) -> Self {
        self.location = Some(Location {
            file: file.to_owned(),
            line,
        });
        self
    }
}

Note that file!() and especially line!() must be invoked at the exact place where the Assertion is built. If we stick them in the implementation of From above, this would not work because it would be expanded to our library file and line, not the one of the end-users. Once again, we need to rely on macros which will be expanded on-site and give use the correct result. Let’s modify assert_that.

#[macro_export]
macro_rules! assert_that {
    ($value:expr) => {{
        runit_assertions::Assertion::from(&($value)).with_location(file!(), line!())
    }};
}

Now, we can modify the Drop implementation like this.

impl<V> std::ops::Drop for Assertion<'_, V>
where
    V: std::fmt::Debug,
{
    fn drop(&mut self) {
        let fails = self.reports.iter().filter(|r| r.pass == false).count();
        let message = if let Some(location) = &self.location {
            format!("\n{}:{}\n", location.file, location.line)
        } else {
            String::from("\n")
        };
        let message = self.reports.iter().fold(message, |msg, r| {
            let emoji = if r.pass { '✓' } else { '⨯' };
            msg + &format!(
                "\t{} should be {} `{}`, was `{}`\n",
                emoji, r.condition, r.compared, r.value,
            )
        });
        assert!(fails == 0, "{}", message);
    }
}

Let run some tests again.

let life = 42;
assert_that!(life)
    .is_not_equal_to(0)
    .is_not_equal_to(42);

The output now looks like this.

stderr:
thread 'main' panicked at '
runit-assertions/src/lib.rs:66
    ✓ should be not equal to `0`, was `42`
    ⨯ should be not equal to `42`, was `42`
'

Some notes

I found out this little exercises pretty cool. Especially the need trick about displaying the file and line. There is however and drawback with this feature: it will not work for RustDoc, or at least, the line number will be wrong. Test it, you’ll see!

Again, you can browse the current implementation in woshilapin/runit.


1. We could have used &mut self instead of self, but there is no good reason to not consume it.
2. I like this title!

links

social