(std test expect)

Snapshot tests: expect-output compares what a test wrote with a text in the test, and –promote updates the text.

(import (std prelude))
(import (std test))
(import (std test expect))

(deftest "printing a card"
  (print-value (Card (rank 12) (suit Hearts)))
  (expect-output """
    (Card (rank 12) (suit Hearts))
    """))

(run-tests)

expect-output compares the output of a test with a text that is written in the test. When they differ, the test fails, and the report shows a line diff. With --promote, the run writes the output into the test file in place of the text, and the test does not fail.

Which tests capture their output. deftest examines its body. When the body has the name expect-output anywhere in it, the output of the test goes to a string port, and current-output-port does not print it. A function that is defined outside the test and calls expect-output does not make the test capture its output. Then expect-output has no output to read, and the test fails.

What is compared. Each expect-output reads what the test wrote since the test started, or since the previous expect-output. Both texts are split into lines. Whitespace at the end of each line is removed, and empty lines at the start and the end of the text are removed. So a final newline, or a carriage return, is not a difference.

The diff. A line that only the expected text has starts with -, a line that only the output has starts with +, and a line that both have starts with a space:

FAIL printing two cards (cards_test.bjo:10)
  cards_test.bjo:13: the output is not the expected text (- expected, + written; --promote updates the text):
     (Card (rank 12) (suit Hearts))
    -(Card (rank 1) (suit Spades))
    +(Card (rank 1) (suit Clubs))

The longest common subsequence of the lines decides which lines both texts have. When the product of the two numbers of lines is more than a million, the diff shows all the expected lines with - and all the written lines with +.

Promotion. With --promote, an expect-output whose text differs records the output, and the test does not fail. At the end of (run-tests), through at-run-end!, the run writes the output into the test file in place of each text that differed, and prints a line for each file that it changed. The text must be a string literal that comes directly after expect-output, because the run looks for it there.

Finding the text. A promotion records the line of its expect-output and the text that the test ran with. The run looks on that line for an (expect-output whose literal still holds that text, and replaces only that literal. So two forms on one line are each found, and a form that was edited after the test was compiled is left alone: the run prints a line for it, and writes nothing in its place.

The new text. Output of one line, without a quote, a backslash or a tab, becomes a plain string: "done". Other output becomes a raw string. Its lines start 2 columns to the right of the column of the (expect-output, and the closing quotes are on a line of their own, in the same column. A raw string has more quotes than the longest run of quotes in the text, and at least three. Output that has no lines becomes "".

The file is changed from its last line to its first, and the forms on one line from the right, so that a change does not move the texts that are still to come. When one expect-output runs more than once in a run, for example in a for-all, the output of its first mismatch is written. run-test alone does not write the file.

Output that no expect-output reads. What the test wrote after its last expect-output is printed when the test ends, so that the prints of a test are not lost.

See also: expect-output, deftest

(: expect-output-check! (-> string int string void))
(expect-output-check! file line wanted)

(expect-output text)

(deftest "a greeting"
  (println "hello")
  (println "world")
  (expect-output """
    hello
    world
    """)
  (println "again")
  (expect-output "again"))

(deftest "prints nothing"
  (expect-output ""))