diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index e2c298f..c0ed80e 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -1626,27 +1626,27 @@ For code you want to keep measuring, write a benchmark file. Benchmarks belong i (def/bench "full-deck" (full-deck)) (def/bench "shuffle" - #:setup (full-deck) - (fun (deck) (shuffle deck))) + #:setup ((deck (full-deck))) + (shuffle deck)) (def/bench "hand points" #:params ((size [5 13 52])) - #:setup (fun (size) (vec-slice (shuffle (full-deck)) 0 size)) - (fun (hand) (vec-fold (fun (c total) (+ total (card-points c))) 0 hand))) + #:setup ((hand (vec-slice (shuffle (full-deck)) 0 size))) + (vec-fold (fun (c total) (+ total (card-points c))) 0 hand)) ;; Sorting changes the deck it sorts, so every call gets a freshly ;; shuffled one, made outside the time. (def/bench "sort a shuffled deck" - #:setup-every (vec->list (shuffle (full-deck))) - (fun (cards) (list-sort cards))) + #:setup-every ((cards (vec->list (shuffle (full-deck))))) + (list-sort cards)) (def/bench "deal with vec-slice" #:group "dealing" - #:setup (shuffle (full-deck)) - (fun (deck) (vec-slice deck 0 5))) + #:setup ((deck (shuffle (full-deck)))) + (vec-slice deck 0 5)) (def/bench "deal with a loop" #:group "dealing" - #:setup (shuffle (full-deck)) - (fun (deck) {vecing (vec-ref deck i) (:for i (range 0 5))})) + #:setup ((deck (shuffle (full-deck)))) + {vecing (vec-ref deck i) (:for i (range 0 5))}) (def/bench "a million hands" #:once 3 (loop (:for i (up-from 0 #:to 1000000)) @@ -1659,7 +1659,7 @@ For code you want to keep measuring, write a benchmark file. Benchmarks belong i (run-benches) }| -It is laid out just like a test file. @code{def/bench} registers a benchmark where it is written, and @code{(run-benches)} at the end becomes the file's @code{main}. @code{bjo bench} builds every file under @code{benches/} and runs it: +It is laid out just like a test file. @code{def/bench} registers a benchmark where it is written, and @code{(run-benches)} at the end becomes the file's @code{main}. The body of a benchmark is ordinary forms, like a function body, and the names its options bind are in scope in it. @code{bjo bench} builds every file under @code{benches/} and runs it: @codeblock[#:lang "text"]|{ $ bjo bench --all @@ -1719,29 +1719,41 @@ The body of a benchmark should be the thing you want to measure and nothing else @read-table{ | Option | What it does | |------------------------------+--------------------------------------------------------------------------------------------------------| -| @code{#:setup e} | runs @code{e} once, outside the time; the body is then a function of what it made | -| @code{#:setup-every e} | runs @code{e} before every call, outside the time; for a body that uses up its input | -| @code{#:params ((p values) ...)} | a row for each combination of values; the body, or the setup, is a function of the parameters | +| @code{#:setup ((x e) ...)} | binds names once, outside the time, for the body to use | +| @code{#:setup-every ((x e) ...)} | binds names again before every call, outside the time; for a body that uses up its input | +| @code{#:params ((p values) ...)} | a row for each combination of values; the setups and the body use the parameters by name | | @code{#:once n} | for a body of seconds: @code{n} cold runs, timed one at a time, no warm-up | | @code{#:group "g"} | a heading in the menu, and what @code{--compare-within} compares within | | @code{#:skip "why"} | not run; the reason is shown | } -@code{#:setup} is the one to reach for first. In the example, @code{shuffle} measures shuffling a deck, not making one: the deck is made once, before the warm-up, and every call shuffles that same deck. +@code{#:setup} is the one to reach for first. Its bindings are written like a @code{let*}'s, so a binding can use the ones before it, and the body uses them by name. In the example, @code{shuffle} measures shuffling a deck, not making one: @code{deck} is made once, before the warm-up, and every call shuffles that same deck. -@code{#:setup-every} is for a body that changes or uses up what it is given, such as a sort that sorts in place, or a body that consumes a list. Each call gets a fresh input, and each call is then timed on its own, with its setup outside the time and outside the allocation. Timing calls one at a time has a cost, because the clock is read twice per call. A body that takes less than a microsecond is marked, since its time is close to the clock's resolution. +@code{#:setup-every} is for a body that changes or uses up what it is given, such as a sort that sorts in place, or a body that consumes a list. Its bindings are made again before every call, so each call gets a fresh input, and each call is then timed on its own, with its setup outside the time and outside the allocation. Timing calls one at a time has a cost, because the clock is read twice per call. A body that takes less than a microsecond is marked, since its time is close to the clock's resolution. -@code{#:params} gives the parameters' names and values, like a @code{let}. The values are a vec of any expressions, evaluated once. There is a row for every combination, in the order written, with the last parameter varying fastest. Without a setup, the body is a function of the parameters. With one, the setup is, and the body gets what the setup made: +The two can be used together. What @code{#:setup} binds is made once, and @code{#:setup-every} can use it to make each call's input. That is the way to keep an expensive input out of the time and still give every call its own copy: + +@codeblock[#:lang "bjolang"]|{ +(def/bench "sort a big shuffled deck" + #:setup ((cards (shuffle (repeat-deck 1000)))) ; 52 000 cards, made once + #:setup-every ((copy (vec->list cards))) ; a fresh list for each call + (list-sort copy)) +}| + +@code{#:params} gives the parameters' names and values, like a @code{let}. The values are a vec of any expressions, evaluated once. There is a row for every combination, in the order written, with the last parameter varying fastest. The setups and the body use the parameters by name: @codeblock[#:lang "bjolang"]|{ (def/bench "sort" #:params ((n [1000 100000]) (order ['sorted 'reversed 'random])) - #:setup (fun (n order) (make-cards n order)) - (fun (cards) (list-sort cards))) + #:setup ((cards (make-cards n order))) + (list-sort cards)) + +;; Without a setup, the body uses them directly. +(def/bench "fib" #:params ((n [10 20 30])) (fib n)) }| -Here @code{make-cards} stands for a function of your own that makes @code{n} cards in the given order. +Here @code{make-cards} and @code{repeat-deck} stand for functions of your own. Each combination is a benchmark of its own for everything else: it is warmed up, calibrated, saved and compared on its own. To try a size the file does not list, give it on the command line: @code{--params n=500,5000} replaces the values of @code{n} for that run. A value is read as the parameter's values are typed: an @code{int}, a @code{double}, a string, a symbol or a @code{bool}. Implementing the trait @code{BenchParam} lets other types be read too.