bjoli/BjoManual
SHA256
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
BjoManual/_02_.Writing-programs.sz
Linus Björnstam 5a9f65accf Traits and fmt brought up to the compiler of 21 October 
- Where an implementation is written: with its trait or its type, a
  newtype for anything else, and traits, types and impls usable above
  where they are written.
- One implementation that fits decides the type of a call.
- A function is written as #<procedure> wherever it is; Procedure.
- A (dyn T) implements the traits T requires.
- Formatting: the lambda that returns a vec of mixed pieces, in the
  table of errors; the short fmt tour in Language basics in the style of
  a string being a block.

🤖 Generated with [ECA](https://eca.dev)

Co-Authored-By: eca-agent <git@eca.dev>
2026-10-22 07:59:36 +02:00

2536 lines
161 KiB
Text
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

@section{Writing programs}
While the first part focused on the core language constructs in single files, this part covers the wider ecosystem: breaking programs into modules, managing project dependencies, handling I/O, concurrency with bjoroutines, .NET interop, macros, and testing.
Throughout this part we will continue building on the card game from Part 1, gradually turning it into a full project complete with a deck module, a persistent high-score file, concurrent players, and a custom @code{#card(Q hearts)} literal.
@subsection{Modules}
In Bjolang, every @code{.bjo} file is implicitly a module named after its file. There are no namespace declarations or enclosing module blocks; instead, a module simply exports whatever public API it wants other files to see, leaving everything else private by default.
@subsubsection{Splitting the game into files}
Here is the deck from the first part as a module of its own. Put it in @code{deck.bjo}:
@codeblock[#:lang "bjolang"]|{
(import (std random))
(export Suit Rank Card full-deck shuffle card-points)
(type/derive (Eq Ord)
(: Suit (Union Clubs Diamonds Hearts Spades))
(: Rank (Union (: Pip int) Jack Queen King Ace))
(: Card (Struct (: rank Rank) (: suit Suit))))
(impl (Displayed Suit)
(defun (display-into s b)
(stringbuilder-add-string! b
(match s (Clubs "♣") (Diamonds "♦") (Hearts "♥") (Spades "♠")))))
(impl (Displayed Rank)
(defun (display-into r b)
(stringbuilder-add-string! b
(match r
((Pip n) (int->string n))
(Jack "J") (Queen "Q") (King "K") (Ace "A")))))
(impl (Displayed Card)
(defun (display-into c b)
(display-into (record-ref c rank) b)
(display-into (record-ref c suit) b)))
;; Not exported: only this module needs them.
(def all-suits [Clubs Diamonds Hearts Spades])
(def all-ranks [(Pip 2) (Pip 3) (Pip 4) (Pip 5) (Pip 6) (Pip 7) (Pip 8)
(Pip 9) (Pip 10) Jack Queen King Ace])
(: full-deck (-> (Vec Card)))
(defun (full-deck)
(loop (:for s all-suits)
(:subloop)
(:for r all-ranks)
(:acc deck (vecing (Card (rank r) (suit s))))
=> deck))
(: shuffle (-> (Vec Card) (Vec Card)))
(defun (shuffle deck) (shuffle-vec deck))
(: card-points (-> Card int))
(defun (card-points c)
(match c
((Card (rank (Pip n))) n)
((Card (rank Ace)) 11)
(_ 10)))
}|
And the game, in @code{game.bjo} next to it:
@codeblock[#:lang "bjolang"]|{
(import "deck.bjo")
(defun (main)
(def hand (vec-slice (shuffle (full-deck)) 0 5))
(println #"Your hand: ${hand}")
(println #"Points: ${(vec-fold (fun (c acc) (+ acc (card-points c))) 0 hand)}")
0)
}|
Running @code{bjo run game.bjo} outputs something like @code{Your hand: [3♥ 10♠ 8♥ 10♦ 7♣]}.
Notice how the imports and exports interact here:
@read-list{
- Relative imports: @code{(import "deck.bjo")} specifies a file path relative to the importing file. Bjolang compiles @code{deck.bjo} into @code{deck.dll} ahead of time, automatically rebuilding it whenever it (or any of its dependencies) changes.
- Type exports are complete: Exporting a type exposes all its constructors, union cases, and fields. Because @code{Card} is exported, @code{game.bjo} can freely construct @code{(Card (rank Ace) (suit Spades))} and pattern-match on its fields.
- Trait implementations are automatic: Although @code{Displayed} was not explicitly listed in the export list, its implementations still work in @code{game.bjo}. Because @code{impl} blocks define anonymous implementations rather than named bindings, they cannot be exported individually; they are automatically carried along whenever the module defining them is loaded.
- Private definitions: Unexported bindings like @code{all-suits} remain private to @code{deck.bjo} and are inaccessible from outside.
}
Every exported definition must have an explicit top-level type signature. The compiler needs this signature to write the metadata into the compiled @code{.dll}:
@codeblock[#:lang "text"]|{
Export Error: Exported item 'helper' is missing a mandatory type signature at bad1.bjo:1
}|
@subsubsection{The standard library, and the prelude}
A list-shaped module path such as @code{(std random)} refers to a standard library module. The compiler resolves these relative to the Bjolang installation rather than the current directory, so @code{(std random)} always points to the same library regardless of where you invoke @code{bjo}. A few notable modules include:
@read-table{
| Module | What it has |
|---------------------------------+------------------------------------------------------|
| @code{(std random)} | random numbers, @code{shuffle-vec} |
| @code{(std hashset)} | sets; @code{(std orderedset)}, @code{(std orderedmap)} |
| @code{(std ports)} | reading a whole port: @code{port->lines} and friends |
| @code{(std fmt)} | text layout |
| @code{(std rx)} | regular expressions |
| @code{(std run)} | running other programs |
| @code{(std effect)} | effect handlers |
| @code{(std simpletest)} | tests |
| @code{(std syntax-match)} | writing macros |
| @code{(text json)} | JSON |
}
By default, @code{(std prelude)} is imported implicitly into every module, providing core functions like @code{println}, @code{list-map}, and the basic operations introduced in Part 1. If you need to hide or override names from the prelude, you can import it explicitly using import modifiers (described below), which replaces the default implicit import:
@codeblock[#:lang "bjolang"]|{
(import (except (std prelude) list-map))
}|
@subsubsection{Exporting types}
Types are private unless listed in @code{export}. Furthermore, the compiler forbids exporting a function whose signature references an unexported private type, since importers would have no way to understand or satisfy that signature. The compiler catches this as an export error in the defining module.
When you want external modules to use a type without inspecting or constructing its internal representation directly, mark the type definition with @code{#:opaque}. For example, here is a card draw pile in @code{pile.bjo} that only permits drawing from the top:
@codeblock[#:lang "bjolang"]|{
(import "deck.bjo")
(export Pile new-pile draw pile-size)
;; Importers can hold a Pile, but only this module can look inside.
(type (: Pile #:opaque (Record (: cards (Vec Card)))))
(: new-pile (-> Pile))
(defun (new-pile) (Pile (cards (shuffle (full-deck)))))
(: pile-size (-> Pile int))
(defun (pile-size p) (vec-length (record-ref p cards)))
;; The top card and the rest of the pile, or None when it is empty.
(: draw (-> Pile (Option (Tuple Card Pile))))
(defun (draw p)
(match (record-ref p cards)
([] None)
([top rest ...] (Some (Tuple top (Pile (cards rest)))))))
}|
With this setup, another module can receive @code{Pile} values, use @code{Pile} in type annotations, and call functions like @code{new-pile} or @code{draw}. However, it cannot construct a pile directly with @code{(Pile ...)}, access its internal @code{cards} field, or pattern-match on its structure:
@codeblock[#:lang "text"]|{
Type Error at game3.bjo:4: 'cards' cannot be read here. pile/Pile is exported #:opaque,
so its representation is visible only to the code of pile. A value of it can be held and
passed on here, and built and taken apart through the functions that module exports.
}|
Even though the type's representation is private, any trait implementations defined in @code{pile.bjo} remain available to consumers. For instance, an @code{(impl (Displayed Pile) ...)} inside @code{pile.bjo} allows any importing module to print a @code{Pile}. Without one, a @code{Pile} is written as @code{#<Pile>}, so printing it shows nothing of its representation.
@subsubsection{Import modifiers}
Imports can be wrapped in modifiers to filter which definitions are brought into scope or rename them to prevent naming collisions:
@read-table{
| Modifier | Does |
|---------------------------------+-------------------------------------------------------|
| @code{(only m a b ...)} | only these functions and macros |
| @code{(except m a b ...)} | everything but these |
| @code{(prefix m "p/")} | puts @code{p/} in front of every name |
| @code{(postfix m "/p")} | the same, at the end |
| @code{(prefix-defs m "p/")} | a prefix on functions and macros only |
| @code{(prefix-types m "P/")} | a prefix on types, constructors and traits only |
| @code{(rename m (old new) ...)} | renames functions and macros |
}
@codeblock[#:lang "bjolang"]|{
(import "deck.bjo"
(prefix "pile.bjo" "pile/"))
(defun (main)
(match (pile/draw (pile/new-pile))
((Some (Tuple c rest)) (println #"Drew ${c}, ${(pile/pile-size rest)} left"))
(None (println "empty")))
0)
}|
Modifiers can be nested and evaluate from the inside out. For example, @code{(prefix (except (std hashset) hashset-map) "s/")} excludes @code{hashset-map} and prefixes all remaining names with @code{s/}.
Note that @code{only} and @code{except} apply exclusively to functions and macros, not types. A module's exported types are always imported, ensuring that signatures referencing those types remain valid and resolvable:
@codeblock[#:lang "bjolang"]|{
(import (only "deck.bjo" full-deck)
(rename (std random) (shuffle-vec mix)))
(defun (main)
(println (vec-length (mix (full-deck)))) ;; 52
(println (Card (rank Ace) (suit Spades))) ;; A♠, Card came along anyway
0)
}|
Types are strictly scoped to the module that declared them. If two imported libraries each declare their own @code{Card} type, they remain distinct types; the unqualified name @code{Card} simply resolves to whichever import came last. To disambiguate between them, use @code{prefix-types}:
@codeblock[#:lang "bjolang"]|{
(import (prefix-types "poker.bjo" "P/")
(prefix-types "bridge.bjo" "B/"))
(: convert (-> P/Card B/Card))
}|
@subsubsection{Which name wins}
When multiple definitions share the same identifier, name resolution follows three precedence rules:
@read-list{
- Local definitions take precedence over imported names.
- For collisions between two imports, the later import shadows the earlier one. This shadowing behavior is how modules can freely override names from the prelude, which is loaded as the very first import.
- Explicit renames or aliases created by import modifiers or @code{:alias} must be unique. If an alias collides with an existing name, the compiler raises an error to prevent ambiguity.
}
@subsubsection{re-export and :alias}
While @code{export} publishes definitions authored in the current module, @code{re-export} publishes definitions that were imported from elsewhere. This makes it easy to create aggregator modules that bundle several sub-modules behind a single import. The prelude uses this technique to pass through @code{=}, @code{compare}, and @code{list-sort} from @code{(std eq)}. In our card game:
@codeblock[#:lang "bjolang"]|{
;; cards.bjo: one import for everything about cards
(import "deck.bjo" "pile.bjo")
(re-export Card Rank Suit full-deck shuffle Pile new-pile draw)
}|
Re-exporting a type exposes the exact original type, not a distinct copy; a @code{Card} referenced via @code{cards.bjo} is identical to one from @code{deck.bjo}. However, trait implementations (@code{impl}) do not transfer through re-exports. They remain bound to the module where they were defined, meaning any file that wants to print cards with their @code{Displayed} implementation must still import @code{deck.bjo}.
You can provide alternative names for bindings or macros using @code{(:alias new old)}. Combining @code{:alias} with @code{export} or @code{re-export} allows libraries to offer cleaner or more idiomatic public APIs:
@codeblock[#:lang "bjolang"]|{
(:alias deal full-deck)
(export deal)
}|
@subsubsection{include}
Unlike @code{import}, which compiles a separate module, @code{(include "file.bjo")} textually splices the included file's forms directly into the current file at compile time. No separate module boundary is created, no exports are required, and all declarations in the included file share the enclosing module's scope:
@codeblock[#:lang "bjolang"]|{
;; helpers.bjo
(: double (-> int int))
(defun (double x) (* x 2))
}|
@codeblock[#:lang "bjolang"]|{
(include "helpers.bjo")
(defun (main)
(println (double 21)) ;; 42
0)
}|
As a general guideline: use @code{include} when you want to divide a single large module across multiple files, and use @code{import} when the target file represents an independent component with its own encapsulation boundaries.
@subsection{Projects and packages}
Up to this point, our programs have consisted of standalone files importing neighboring modules directly by relative path. While that approach is convenient for quick scripts and small experiments, any software that pulls in external dependencies or is meant to be consumed by other developers should be structured as a project.
A project is simply a directory containing a @code{manifest.bjodat} file. When operating inside a project directory, @code{bjo} understands the package's identity, resolves dependencies, and manages builds automatically. Outside of a project, the CLI falls back to single-file execution just as before.
@subsubsection{Making a project}
@codeblock[#:lang "text"]|{
mkdir cards && cd cards
bjo init
bjo run
}|
Running @code{bjo init} scaffolds a new project layout:
@codeblock[#:lang "text"]|{
cards/
manifest.bjodat package metadata and dependencies
src/
main.bjo executable entry point
tests/
.gitignore
}|
The generated manifest defines the package identity:
@codeblock[#:lang "bjolang"]|{
(package
(name (cards))
(version "0.1.0"))
}|
All source files in a package live under @code{src/}, where each module's name mirrors its relative path within that directory. For example, moving @code{deck.bjo} into @code{src/} exposes it as the module @code{(cards deck)}, which @code{src/main.bjo} can import just like any library module:
@codeblock[#:lang "bjolang"]|{
(import (cards deck))
(defun (main args)
(println #"Your hand: ${(vec-slice (shuffle (full-deck)) 0 5)}")
(println #"Arguments: ${args}")
0)
}|
Because @code{bjo} automatically detects the project root by walking upward until it finds a @code{manifest.bjodat}, you can invoke CLI commands from any sub-directory within the project:
@read-table{
| Command | What it does |
|---------------------------+----------------------------------------------------------------|
| @code{bjo init [--lib]} | Initializes a new project in the current directory |
| @code{bjo fetch} | Fetches external dependencies without compiling |
| @code{bjo build} | Fetches dependencies and compiles the program or library |
| @code{bjo run . args ...} | Builds and runs the project with the specified arguments |
| @code{bjo check} | Type-checks the entire project without emitting output binaries |
| @code{bjo repl} | Starts the REPL preloaded with access to the project's modules |
}
In @code{bjo run . one two}, the dot specifies the project itself. Any arguments following the target are forwarded directly to the program's @code{main} function, binding @code{args} to @code{[one two]}. You can also point @code{bjo run} at a specific test file, such as @code{bjo run tests/deck-test.bjo}; the file will be compiled and linked against the project's declared packages.
@subsubsection{Libraries}
A package that lacks an executable entry point (@code{src/main.bjo}) is treated as a library. Running @code{bjo init --lib} initializes a library project, and running @code{bjo build} inside it compiles all modules found under @code{src/}.
To see this in action, we can split out our card deck into a separate library residing alongside the game:
@codeblock[#:lang "text"]|{
mkdir cardlib && cd cardlib
bjo init --lib
mv ../cards/src/deck.bjo src/
}|
We can then declare a path dependency in @code{cards/manifest.bjodat} and import the library module as @code{(cardlib deck)}:
@codeblock[#:lang "bjolang"]|{
(package
(name (cards))
(version "0.1.0")
(depends
(package (name (cardlib))
(source (path (dir "../cardlib"))))))
}|
@codeblock[#:lang "bjolang"]|{
(import (cardlib deck))
(defun (main)
(println (vec-slice (shuffle (full-deck)) 0 5))
0)
}|
Path dependencies are resolved relative to the manifest file that specifies them. Because a path dependency points straight to source files on disk, it does not require a version number—the build simply consumes whatever code is currently in the directory, making local development and iteration seamless.
@subsubsection{Dependencies from git}
Once a library is hosted in a Git repository, you can switch from a local path to a Git dependency:
@codeblock[#:lang "bjolang"]|{
(depends
(package (name (cardlib))
(version (version-at-least "0.1.0"))
(source (git (url "https://github.com/someone/cardlib")))))
}|
Package versions correspond to Git tags formatted strictly as @code{vX.Y.Z}. Git dependencies specify version constraints using @code{(version-at-least "1.2")}, @code{(version-between "1.2" "2.0")}, or @code{(version-at-most "2.0")}.
Bjolang resolves versions using Minimal Version Selection (MVS). Rather than greedily selecting the latest release available in the wild, MVS inspects the minimum version requested by each package across the entire dependency graph, and then selects the highest of those minimum requirements. In other words, you get the oldest possible release that satisfies everyone's stated requirements. This guarantees reproducible builds: new upstream releases won't alter your build until someone explicitly raises their requirement in a manifest, ensuring your code compiles identically a year from now. Any upper bound constraints are treated strictly as validation checks, never as criteria to pick an older package.
Commands like @code{bjo fetch}, @code{build}, and @code{run} clone external dependencies into a local @code{.bjo/} directory and record the exact resolved commits in @code{bjo.lock}. You should always check @code{bjo.lock} into version control, while leaving @code{.bjo/} ignored (as configured in the default @code{.gitignore}). If an upstream repository rewires a tag to point to a different commit after it was locked, the build aborts immediately. On CI servers, pass @code{--locked} to guarantee that the build fails if any dependency deviates from the locked snapshot.
You can also override dependency sources: if your root manifest defines an explicit source for a package, that source overrides any locations requested deeper in the dependency tree. This makes it straightforward to substitute a local checkout or an emergency fork while debugging.
@subsubsection{NuGet packages}
You can pull in third-party .NET packages directly from NuGet by listing them in the @code{packages} section:
@codeblock[#:lang "bjolang"]|{
(package
(name (shop))
(version "0.1.0")
(packages (nuget (id "Npgsql") (version "9.0.3"))))
}|
Once restored, types and methods from NuGet packages can be consumed through Bjolang's .NET interop mechanisms (described in the .NET section below). Version strings follow standard NuGet versioning semantics: @code{"9.0.3"} matches version 9.0.3 or higher, while bracketed syntax like @code{"[9.0.3]"} locks to that exact version. Bjolang delegates package restoration to the .NET SDK via the @code{dotnet} tool, storing the resolution graph in @code{packages.lock.json} (which should also be committed). Currently, compiled binaries load dependencies directly from the local NuGet cache, so executables expect to run in an environment with the restored packages available.
@subsubsection{Native executables}
A program usually runs on .NET: @code{bjo run} starts the runtime, and the JIT compiles each method the first time it runs. With @code{(aot #t)} in the manifest, the program is also compiled ahead of time, with .NET's Native AOT, to one native executable:
@codeblock[#:lang "bjolang"]|{
(package
(name (cards))
(version "0.1.0")
(aot #t))
}|
The executable is beside the source, with no extension: @code{src/main}, or @code{src/main.native.exe} on Windows. It holds the program, the standard library and the .NET runtime, so it runs on a machine without .NET installed.
@read-list{
- @strong{It starts in a few milliseconds.} A program on .NET spends about 70 ms starting the runtime, and then compiles its methods as it runs them. On 17 days of Advent of Code, the solutions together took 3.0 s on .NET and 1.2 s as native executables.
- @strong{@code{bjo run} starts the native executable, and @code{bjo publish} copies it} into the output directory. The ordinary @code{.exe} is still built, for the debugger and for @code{-d}.
- @strong{The build takes longer}, about 10 s more, and needs clang or gcc. The first build downloads .NET's ILCompiler package from NuGet.
- @strong{The runtime settings are built into the executable.} The GC settings, such as @code{(runtime (gc-server #t))}, take effect. The settings of tiered compilation do nothing, because there is no JIT.
- @strong{A long-running program can be slower.} The JIT optimizes the code with what it sees the program do (dynamic PGO), and a native executable can not. This matters for programs where the call graph stops the monomorphization or when trait calls cannot be determined at compile time.
- @strong{Reflection that is resolved at run time may fail.} Native AOT keeps only the code that it can see the program use. Bjolang does not use reflection by default, so this should be a small problem.
}
By default the executable runs on any x86-64 CPU, so it uses only the instructions that all of them have: no AVX, and 128-bit vectors. @code{aot-cpu} names a CPU to build for instead:
@codeblock[#:lang "bjolang"]|{
(aot #t)
(aot-cpu "native")
}|
@code{"native"} is the CPU of the machine that builds. A level, such as @code{"x86-64-v3"}, is every CPU that has its instructions: @code{x86-64-v3} has AVX2, and @code{x86-64-v4} has AVX-512. An executable built for a CPU stops at start on a CPU that does not have the instructions, so use @code{"native"} for a program that runs where it is built. This matters most to @code{(std array)} and @code{(std simd)}; see the section on vectors under performance.
Only the project being built reads @code{aot} and @code{aot-cpu}, as with @code{runtime}: a dependency is not a program. A change to either rebuilds the program. The compiler itself takes them as @code{--aot} and @code{--aot-cpu}, for a file built without a project.
@subsubsection{Publishing a package}
Publishing a Bjolang package requires three basic steps:
@read-list{
- Organize your source files under @code{src/}, and create a @code{manifest.bjodat} defining the package @code{name} and @code{version}.
- Run @code{bjo build} to verify that all modules compile cleanly without errors.
- Tag your commit with the semantic version—e.g. @code{git tag -a v0.1.0 -m "First release"}—and push the tag to your remote repository. Dependencies require valid @code{vX.Y.Z} tags to resolve.
}
@subsection{Files and I/O}
@subsubsection{Whole files}
For files that comfortably fit into memory, the prelude provides straightforward functions to read or write entire files in a single call:
@read-table{
| Function | Description |
|-------------------------------------------+----------------------------------------------------|
| @code{(file-read-text path)} | Reads the entire file into a string |
| @code{(file-write-text path s)} | Writes @code{s}, overwriting any existing contents |
| @code{(file-append-text path s)} | Appends @code{s} to the file, creating it if needed |
| @code{(file-read-lines path)} | Reads lines as a @code{(Vec string)} |
| @code{(file-write-lines path lines)} | Writes a @code{(Vec string)}, one line per element |
| @code{(file-read-bytes path)} | Reads the raw bytes as an @code{(Array byte)} |
| @code{(file-exists? path)} | Checks whether a file exists at the given path |
| @code{(file-delete path)} | Deletes a file; succeeds silently if file is absent |
| @code{(file-copy from to)} | Copies a file, raising an error if target exists |
| @code{(file-move from to)} | Moves or renames a file or directory |
| @code{(file-info path)} | Returns size, modification time, and kind as an @code{Option} |
}
Because these wrap the underlying .NET I/O methods directly, any I/O failure (such as missing permissions or a missing file on read) throws a .NET exception. If you expect a failure under normal conditions, wrap the call in a @code{try} expression.
To illustrate, we can add a persistent high-score ledger to our card game. Each line records a player name and their score separated by a space. Malformed lines are safely discarded rather than crashing the program:
@codeblock[#:lang "bjolang"]|{
(: score-file string)
(def score-file "scores.txt")
(: parse-score (-> string (Option (Tuple string int))))
(defun (parse-score line)
(match (string-split line " ")
([name points]
(match (try (string->int points) #:catch (System.FormatException))
((Ok n) (Some (Tuple name n)))
((Err _) None)))
(_ None)))
(: read-scores (-> (Vec (Tuple string int))))
(defun (read-scores)
(if (file-exists? score-file)
(loop (:for line (file-read-lines score-file))
(:when-let (Some score) (parse-score line))
(:acc (vecing score)))
[]))
(: save-score (-> string int void))
(defun (save-score name points)
(file-append-text score-file #"${name} ${points}\n"))
(defun (main)
(save-score "ada" 31)
(save-score "bo" 27)
(println (read-scores)) ;; [(Tuple ada 31) (Tuple bo 27)]
0)
}|
@subsubsection{Paths, directories and the environment}
Paths in Bjolang are represented as standard strings. The path manipulation utilities perform purely lexical operations on the string representations without touching the filesystem:
@codeblock[#:lang "bjolang"]|{
(path-combine "a" "b" "c.txt") ;; "a/b/c.txt"
(path-file-extension "deck.bjo") ;; (Some ".bjo")
(path-filename "/tmp/x/") ;; None: it names a directory
(path-directory "/tmp/x/y.txt") ;; (Some "/tmp/x")
(path-absolute "deck.bjo") ;; the full path
}|
When you do need to inspect or modify the filesystem layout, use the directory functions:
@codeblock[#:lang "bjolang"]|{
(directory-create "saves/2026") ;; creates every missing directory on the way
(directory-exists? "saves") ;; #t
(directory-files ".") ;; the files directly in ".", as a (Vec string)
(directory-subdirectories ".") ;; and the directories
(directory-delete-tree "saves") ;; everything under it, and it
;; There is no pattern argument. Filter instead:
(vec-filter #(string-ends-with? & ".bjo") (directory-files "."))
;; Every file under a directory, lazily, without going into .git:
(filter #(string-ends-with? & ".bjo")
(directory-walk "." #:into? (fun (d) (not (= (path-filename d) (Some ".git"))))))
}|
Process environment variables and working directory state can be inspected and updated similarly:
@codeblock[#:lang "bjolang"]|{
(get-environment-variable "HOME") ;; (Some "/home/linus"), or None
(set-environment-variable! "MODE" "x") ;; for this process and the ones it starts
(current-directory)
}|
@subsubsection{Ports}
When dealing with large files, streaming data, or incremental I/O, Bjolang uses *ports*. An input port provides a stream from which characters and lines can be read incrementally, while an output port collects written data. File ports and in-memory string ports share the same underlying interface, so functions written against ports work seamlessly with both files and string buffers.
Attempting to open a file returns a @code{Result}, treating potential I/O errors (such as missing files) as expected outcomes. The @code{Err} variant wraps the underlying .NET exception, allowing you to match on specific exception types with @code{:is}:
@codeblock[#:lang "bjolang"]|{
(match (open-input-file "nope.txt")
((Ok port) (read-all port))
((Err (:is System.IO.FileNotFoundException e)) #"missing: ${(.-FileName e)}")
((Err e) "some other error"))
}|
To avoid leaking resources, you will usually want to use @code{call-with-input-file} or @code{call-with-output-file}. These higher-order functions open the port, pass it to your callback, and guarantee the port is cleanly closed when the function finishes—even if an exception is thrown. For example, counting lines in an arbitrary file looks like:
@codeblock[#:lang "bjolang"]|{
(: count-lines (-> string (Result Exception int)))
(defun (count-lines path)
(call-with-input-file path
(fun (port)
(loop (:until (port-eof? port))
(:let line (read-line port))
(:acc lines (counting line))
=> lines))))
(count-lines "scores.txt") ;; (Ok 2)
}|
@read-table{
| Function | Description |
|-----------------------------------+---------------------------------------------------------|
| @code{(open-input-file path)} | Returns a @code{(Result Exception TextInputPort)} |
| @code{(open-output-file path)} | Opens for writing; @code{#:mode} can be @code{Truncate}, @code{Append}, or @code{CreateNew} |
| @code{(read-line/opt p)} | Reads the next line, or returns @code{None} at EOF |
| @code{(read-char/opt p)} | Reads the next character, or returns @code{None} at EOF |
| @code{(read-all p)} | Reads the remaining contents into a single string |
| @code{(port-eof? p)} | Checks if the end of the input stream has been reached |
| @code{(write-string p s)} | Writes string @code{s} to the port |
| @code{(writeln p s)} | Writes string @code{s} followed by a newline |
| @code{(close-input-port p)} | Closes an input port |
| @code{(close-output-port p)} | Closes an output port |
| @code{(open-input-string s)} | Creates a port reading from an in-memory string |
| @code{(open-output-string)} | Creates a string-builder port; read with @code{get-output-string} |
}
The variants @code{read-line} and @code{read-char} (without the @code{/opt} suffix) also exist, but throw an end-of-file exception if called at EOF.
Convenience helpers in @code{(std ports)} can ingest an entire port into collections: @code{port->lines}, @code{port->list}, @code{port->vec}, as well as lazy sequences via @code{port->seq} and @code{file->seq}:
@codeblock[#:lang "bjolang"]|{
(import (std ports))
(call-with-input-file "scores.txt" port->lines) ;; (Ok ["ada 31" "bo 27"])
}|
In Bjolang, open file ports are also tracked by their enclosing structured concurrency scope (explained in the Concurrency section below). When a scope exits, any unclosed ports opened within it are automatically finalized. Even @code{main} acts as a root scope, ensuring leaked handles are collected when your process terminates. For long-lived processes, however, you should always explicitly close ports or rely on @code{call-with-input-file}.
@subsubsection{Running other programs}
The @code{(std run)} module lets you invoke external processes. Commands are written as S-expression forms using a concise DSL where pipelines and shell-style redirections are integrated directly into the syntax:
@codeblock[#:lang "bjolang"]|{
(import (std run))
(run/string '(echo "hello")) ;; (Ok "hello\n")
(run/strings '(pipe (cat "scores.txt") (sort -r))) ;; (Ok ["bo 27" "ada 31"])
(run/status '(into-file "sorted.txt"
(pipe (cat "scores.txt") sort))) ;; (Ok 0), the exit code
(def who "ada")
(run/strings '(grep ,who "scores.txt")) ;; (Ok ["ada 31"])
}|
Prefixing an identifier with a comma (e.g. @code{,who}) splices the variable's value into the command arguments. Each list element is treated as an individual command-line argument, preserving spaces without fragile string quoting or shell injection hazards. The DSL primitives @code{pipe}, @code{into-file}, @code{from-file}, @code{append-to-file}, and @code{errors-into-file} are validated at compile time; any unrecognized head identifier is executed as an external executable.
@read-table{
| Function | Returns |
|---------------------------+------------------------------------------------------|
| @code{(run/string form)} | Standard output as a single trimmed string |
| @code{(run/strings form)} | Standard output split into a vector of lines |
| @code{(run/status form)} | Process exit code as an integer |
| @code{(run/output form)} | Tuple of exit code and standard output string |
| @code{(run form)} | A running @code{Proc} handle with streaming I/O ports |
}
All of these functions return a @code{Result} and execute synchronously, blocking the calling thread until completion. When running inside a bjoroutine, non-blocking asynchronous alternatives are available; refer to the @code{(std run)} module documentation for details.
@subsection{Formatting text}
For one line of output, an interpolated string or @code{println*} is enough. @code{println*} is in @code{(std fmt)}, and takes strings, ints, doubles, chars and bools in one vec:
@codeblock[#:lang "bjolang"]|{
(import (std fmt))
(println #"${who} has ${points} points") ;; ada has 31 points
(println* [who " has " points " points"]) ;; ada has 31 points
}|
When the shape of the output matters, such as columns that line up, padding, a paragraph wrapped to a width or a table, use the layout part of @code{(std fmt)}. This section is a guide to it. The module's page has the reference.
@subsubsection{Blocks}
Layout is made of blocks. A @code{Block} is a piece of output, and every function that lays something out takes its content as a block and gives back a block. These are blocks:
@read-list{
- A string, an int or a char. Written out, or the value of a variable or a call: @code{"tea"}, @code{name}, @code{(string-upcase name)}.
- A vec of blocks: the blocks one after another. @code{["x = " n nl]} is a string, an int and a newline.
- What a function in @code{(std fmt)} gives back: @code{(padded/left 5 "x")}, @code{nl}, @code{nothing}.
- An @code{if}, @code{cond}, @code{match} or @code{let} whose results are blocks.
}
So one thing needs no brackets, and several things are a vec:
@codeblock[#:lang "bjolang"]|{
(def name "Ada")
(def n 3)
(show "Hello\n") ;; Hello
(show (padded/left 6 name)) ;; " Ada"
(show (padded/left 8 ["x = " n])) ;; " x = 3"
(show [(padded/left 5 "x") (padded/left 5 "y") nl]) ;; " x y", then a newline
(show ["Hello, " name "! You have " n " new messages" #\. nl])
;; Hello, Ada! You have 3 new messages.
}|
Read @code{(padded/left 6 name)} as "pad @code{name} to 6 columns", and @code{(padded/left 8 ["x = " n])} as "pad these pieces, together, to 8 columns". A vec in a vec is a block in a block, so extra brackets change nothing.
Nothing is written until @code{show} runs the block. A block is a value: you can bind it, pass it to a function and return it.
@subsubsection{What is not a block}
Anything else is converted first:
@read-table{
| You have | Write |
|-----------------------------------+---------------------------------------------------------------|
| A double @code{x} | @code{(numeric x)}, or @code{(numeric x #:precision 2)} |
| A bool @code{b} | @code{(if b "yes" "no")} |
| A long @code{n} | @code{(long->string n)}, or @code{(bytes n)} for a size |
| A @code{(Vec string)} @code{xs} | @code{(vec-map BStr xs)} |
| A @code{(List %a)} @code{xs} | @code{(joined f xs ", ")}, where @code{f} makes the block of one element |
}
@code{BStr}, @code{BInt}, @code{BChar} and @code{BMany} are the cases of @code{Block}. You rarely write them, but like any constructor they are functions, so @code{(vec-map BStr names)} is a vec of blocks.
@strong{A number bound by @code{def}.} @code{(def n 3)} does not say which kind of number @code{n} is: it could still become a long or a double. Where a block is wanted, it is read as the number written there would be, so @code{["n = " n]} is what @code{["n = " 3]} is, and @code{n} is an int from then on. A number that is used as a double later, as in @code{(+ n 0.5)}, needs @code{(: n double)} above its @code{def}.
@strong{Where a block is wanted.} A string or a vec is read as a block where the compiler knows that a block is wanted: as an argument, as what a function declared to return a @code{Block} returns, and as the value of a binding whose type is written, such as @code{(: parts Block)}. A @code{def} without a type is what its value is: @code{(def parts ["a" "b"])} is a @code{(Vec string)}, and @code{(show parts)} is refused. Write @code{(: parts Block)} above it.
@subsubsection{Blocks from your data}
Most layout code is a function from a value of your own to a block. Give it a signature: the signature is what tells the compiler that what it returns is a block.
@codeblock[#:lang "bjolang"]|{
(type (: Item (Record (: name string) (: price double) (: stock int))))
(: stock (Vec Item))
(def stock
[(Item (name "Green tea") (price 3.5) (stock 12))
(Item (name "Gaiwan") (price 1240.0) (stock 2))
(Item (name "Kettle") (price 89.9) (stock 0))])
(: item-line (-> Item Block))
(defun (item-line i)
[(padded/right 10 (record-ref i name))
(padded/left 9 (numeric (record-ref i price) #:precision 2 #:comma 3))
(if (= (record-ref i stock) 0) " sold out" nothing)
nl])
(show (vec-map item-line stock))
}|
@codeblock[#:lang "text"]|{
Green tea 3.50
Gaiwan 1,240.00
Kettle 89.90 sold out
}|
@code{vec-map} gives a vec of blocks, which is a block. A lambda given to @code{vec-map} is different from a function with a signature: @code{vec-map} does not say what its function returns, so a vec of mixed pieces that the lambda returns, such as a string and an int, is not read as blocks and is refused. Give it to @code{each}, which says that it is one:
@codeblock[#:lang "bjolang"]|{
(show (vec-map (fun (i) (each [(record-ref i name) ": " (record-ref i stock) nl]))
stock))
;; Green tea: 12
;; Gaiwan: 2
;; Kettle: 0
}|
A lambda given to a function in @code{(std fmt)}, such as @code{joined}, needs no @code{each}: their signatures say that the lambda returns a block.
@code{nothing} is the block that writes nothing, for the branch that has nothing to say:
@codeblock[#:lang "bjolang"]|{
(: greeting (-> string (Option string) Block))
(defun (greeting name title)
["Hello, "
(match title
((Some t) [t " "])
(None nothing))
name "!"])
(: medal (-> int Block))
(defun (medal place)
(cond ((= place 1) "gold")
((= place 2) "silver")
(else nothing)))
(show [(greeting "Ada" (Some "Dr")) nl (greeting "Bob" None) nl])
;; Hello, Dr Ada!
;; Hello, Bob!
(show [(medal 1) "/" (medal 2) "/" (medal 3) nl]) ;; gold/silver/
}|
A list is written with @code{joined}, which takes a function that makes each element's block, and a separator:
@codeblock[#:lang "bjolang"]|{
(show [(joined BStr (list "tea" "milk" "honey") ", ") nl]) ;; tea, milk, honey
(show [(joined/last BStr (fun (s) ["and " s])
(list "tea" "milk" "honey") ", ") nl]) ;; tea, milk, and honey
(show [(joined (fun (s) ["* " s]) (list "tea" "milk") nl) nl])
;; * tea
;; * milk
}|
@subsubsection{Padding, cutting and leaders}
@code{padded/left}, @code{padded/right} and @code{padded/both} pad content to a width: aligned right, left, or centred. Padding never cuts. @code{trimmed/left}, @code{trimmed/right} and @code{trimmed/both} cut content to a width, and keep its end, its start or its middle. @code{fitted/left}, @code{fitted/right} and @code{fitted/both} do both, so the content is exactly the width.
@codeblock[#:lang "bjolang"]|{
(show [(padded/right 8 who) (padded/left 4 points) nl]) ;; ada 31
(show [(with-pad #\0 (padded/left 4 42)) nl]) ;; 0042
(show [(with-pad #\= (padded/both 20 " Receipt ")) nl]) ;; ===== Receipt ======
(show [(trimmed/left 6 "/home/ada/scores.txt") nl]) ;; es.txt
(show [(with-fmt current-ellipsis "…"
(trimmed/left 12 "/home/ada/scores.txt")) nl]) ;; …/scores.txt
(show ["[" (fitted/both 6 "ab") "]" nl]) ;; [ ab ]
}|
@code{space-to} pads to a column, whatever was written before it, and @code{with-pad} changes what the padding is made of. Together they make a leader:
@codeblock[#:lang "bjolang"]|{
(show [(with-pad #\. ["Tea" (space-to 12) "3.50"]) nl
(with-pad #\. ["Gaiwan" (space-to 12) "1240.00"]) nl])
;; Tea.........3.50
;; Gaiwan......1240.00
}|
@code{nl} is a newline, and @code{fl} a newline unless the cursor is already at the start of a line.
@subsubsection{Numbers}
An int is a block. A double is not, as there is no one way to write one: @code{numeric} says how, with a precision, digits in groups, and the separators. @code{bytes} and @code{percent} are blocks for two common kinds of number.
@codeblock[#:lang "bjolang"]|{
(show [(numeric 1234567.891 #:precision 2 #:comma 3) " "
(numeric 0.5 #:precision 3) " "
(numeric 42.0 #:signed #t) " "
(percent 0.125) " "
(bytes 1500L #:si #t) nl])
;; 1,234,567.89 0.500 +42 12.5% 1.5 kB
}|
The digits are always the invariant culture's, so the machine's locale never shows. The separators are settings, described below.
@subsubsection{Paragraphs}
@code{wrapped} fills content to @code{current-width}, and @code{justified} makes it flush against both margins. Every run of whitespace in the content is a break between words.
@codeblock[#:lang "bjolang"]|{
(show [(with-fmt current-width 20
(justified "Bjolang lays text out with blocks, and blocks nest."))
nl])
;; Bjolang lays text
;; out with blocks, and
;; blocks nest.
}|
The first line starts where the cursor is, so after a label it has less room. The lines after it start at column 0.
@subsubsection{Tables}
A table is the place where the brackets matter. @code{tabular} takes a vec of columns, and @code{tabular/rows} a vec of rows. Each column or row is a @code{(Vec Block)}, with a block for each cell:
@codeblock[#:lang "bjolang"]|{
(show (tabular [["tea" nl "gaiwan" nl "kettle"]
[3 nl 1240 nl 90]]
#:sep " " #:align [AlignLeft AlignRight]))
(show (tabular/rows [["tea" 3] ["gaiwan" 1240] ["kettle" 90]]
#:sep " " #:align [AlignLeft AlignRight]))
}|
Both write the same table:
@codeblock[#:lang "text"]|{
tea 3
gaiwan 1240
kettle 90
}|
In @code{tabular} a column's cells are separated by @code{nl}. In @code{tabular/rows} each element of a row is one cell, so a cell of several pieces is a vec in the row, and a newline in a cell gives the row another line:
@codeblock[#:lang "bjolang"]|{
(show (tabular/rows [["tea" [3 " cups"]]
["kettle" ["boils in" nl "4 minutes"]]]
#:style StyleAscii))
}|
@codeblock[#:lang "text"]|{
+--------+-----------+
| tea | 3 cups |
| kettle | boils in |
| | 4 minutes |
+--------+-----------+
}|
Rows from data come from a function with a signature, for the reason given above: a row of a string and an int has no element type of its own. Here is the high-score ledger from "Files and I/O" as a table:
@codeblock[#:lang "bjolang"]|{
(: score-row (-> (Tuple string int) (Vec Block)))
(defun (score-row s)
(def (Tuple name points) s)
[name points])
(show (tabular/rows (vec-map score-row (read-scores))
#:titles ["Player" "Score"]
#:align [AlignLeft AlignRight]
#:style StyleAscii))
}|
@codeblock[#:lang "text"]|{
+--------+-------+
| Player | Score |
+--------+-------+
| ada | 31 |
| bo | 27 |
+--------+-------+
}|
The tables here are drawn with @code{StyleAscii}, whose characters every font has. @code{#:style} takes a @code{TableStyle}: @code{StyleNone}, the default, @code{StyleSimple}, @code{StyleAscii}, @code{StyleLight}, @code{StyleHeavy}, @code{StyleDouble}, @code{StyleRounded}, @code{StyleMarkdown}, or @code{StyleCustom} with pieces of your own. @code{#:align} takes an @code{Align} for each column, from the first: @code{AlignLeft}, @code{AlignRight}, @code{AlignCentre}, or @code{(AlignOn #\.)}, which puts every decimal point of the column in one place. @code{#:widths} and @code{#:overflow} make a table fit @code{current-width}, by cutting or wrapping its cells.
A table whose rows are not all known yet, such as results that come one at a time, is laid out a row at a time with @code{table-layout} and @code{table-row}. The widths are fixed beforehand, so each row is written when it is known:
@codeblock[#:lang "bjolang"]|{
(def layout (table-layout [8 6] #:sep " " #:align [AlignLeft AlignRight]))
(show (table-row layout ["Player" "Score"]))
(loop (:for (Tuple name points) (read-scores))
(:do (show (table-row layout [name points]))))
}|
@codeblock[#:lang "text"]|{
Player Score
ada 31
bo 27
}|
A table can also be a quoted literal, which the compiler checks where it is written. A column can say what kind of number it holds:
@codeblock[#:lang "bjolang"]|{
(show-table '(table (style ascii) header
(column "Player")
(column "Score" (align right))
(column "Share" (percent (precision 0)))
(rows (row "ada" 31 0.46) (row "bo" 27 0.40) (row "cy" 9 0.13))))
}|
@codeblock[#:lang "text"]|{
+--------+-------+-------+
| Player | Score | Share |
+--------+-------+-------+
| ada | 31 | 46% |
| bo | 27 | 40% |
| cy | 9 | 13% |
+--------+-------+-------+
}|
The rows can be spliced, @code{(rows ,scores)}, from a @code{(Vec (Vec TableCell))}. See @code{Table} on the module's page.
@subsubsection{Settings}
The pad character, the right margin, the ellipsis and the number formats are parameters. For a whole output, set them with @code{parameterize} around the @code{show}:
@codeblock[#:lang "bjolang"]|{
(parameterize ((current-width 20))
(show (wrapped "Bjolang lays text out with blocks, and blocks nest.")))
;; Bjolang lays text
;; out with blocks, and
;; blocks nest.
}|
A block reads its settings when it runs, which is in @code{show}, not when it is built. So the @code{parameterize} must be around the @code{show}: around the code that builds the block, it has ended before the block runs, and does nothing. @code{(show (parameterize ((current-width 20)) (wrapped text)))} wraps at the default width, 78.
For part of an output, such as one paragraph, one cell or the numbers of one column, put the setting in the layout. @code{with-fmt} binds one setting around a block while it runs, @code{with-fmt*} binds several, written as @code{parameterize} is, and @code{with-pad} is @code{with-fmt} for @code{current-pad}:
@codeblock[#:lang "bjolang"]|{
(show (with-fmt* ((current-comma 3)
(current-comma-sep #\space)
(current-decimal-sep #\,))
(numeric 1234567.5) nl))
;; 1 234 567,5
}|
The settings are @code{current-pad}, @code{current-width}, @code{current-ellipsis}, @code{current-string-width}, @code{current-precision}, @code{current-comma}, @code{current-comma-sep}, @code{current-decimal-sep} and @code{current-signed}. For text with wide characters or colours in it, which a terminal shows in more or fewer columns than it has characters, set @code{current-string-width} to @code{terminal-width}.
@subsubsection{When it does not compile}
The messages for the common mistakes, and what to write instead:
@read-table{
| The compiler says | Write |
|----------------------------------------------------------------------+-----------------------------------------------------|
| Block: ... found a double | @code{(numeric x)} |
| Block: ... found the number 1.5 | @code{(numeric 1.5)} |
| Block: ... found a boolean, or found a bool | @code{(if b "yes" "no")} |
| Block: ... found a (Vec string) | @code{(vec-map BStr xs)}, or @code{(: xs Block)} above @code{(def xs ...)} |
| Block: ... found a (Vec (Vec ...)), at a lambda that returns a vec | @code{(each [...])} around the vec the lambda returns |
| these types do not match, naming two of int, string and char, at a lambda that returns a vec | @code{(each [...])} around the vec the lambda returns |
}
The last two are one mistake, a lambda given to a function that does not say it returns a block. Its vec is a vec of strings if all its pieces are strings, and then @code{show} gets a vec of vecs. If the pieces are of different kinds, such as a name and a count, the vec has no type at all, and the compiler says so inside the lambda.
@subsubsection{A block of your own}
@code{captured} renders content to a string and gives it to a function, which returns the block to write in its place. It is how a block that needs the whole of its content is made, and how padding and trimming are written:
@codeblock[#:lang "bjolang"]|{
(: boxed (-> Block Block))
(defun (boxed b)
(captured (fun (s) ["[" s "]"]) b))
(show [(boxed ["x = " 7]) nl]) ;; [x = 7]
}|
A block that writes as it goes is a @code{BRun}: a function given the sink to write to and the column the cursor is at, which returns the column it ends at. @code{run-block} runs a block inside one. The module's page describes both.
@subsection{Concurrency}
Bjolang's concurrency model rests on three foundations: lightweight fibers called *bjoroutines*, communication channels, and structured concurrency scopes ensuring background work is cleanly contained. In addition, Bjolang adopts the Concurrent ML (CML) paradigm, where communication operations—such as sending or receiving on a channel—are first-class values that can be composed and combined before being synchronized.
@subsubsection{Bjoroutines}
Functions that may suspend execution—whether waiting on a channel, sleeping, or performing asynchronous I/O—are declared with @code{defbjo} rather than @code{defun}, and their type signatures use the @code{-bjo->} arrow. Suspending and waiting for an event is performed using @code{sync}:
@codeblock[#:lang "bjolang"]|{
(: think (-bjo-> int int))
(defbjo (think n)
(sync (timeout (milliseconds 50))) ;; wait 50 ms, without holding up a thread
(* n n))
}|
Bjoroutines execute as lightweight fibers scheduled across a managed thread pool. When a fiber suspends at a @code{sync} point, it yields its underlying thread back to the pool. As a result, maintaining thousands of waiting fibers incurs the memory overhead of small heap objects rather than expensive OS threads.
Because calling a suspending function means the caller may also have to wait, function coloring applies: bjoroutines can only be directly invoked from within other bjoroutines:
@codeblock[#:lang "text"]|{
Type Error at colour.bjo:7: calling 'think' is a yield point, and a yield point is not allowed here.
'twice' is defined with (defun ...), which is emitted as an ordinary C# method, and an ordinary
method cannot await.
Define it with (defbjo ...), or move the suspending call out of it. Note that (defbjo ...) spreads:
whoever calls 'twice' needs to be one too.
}|
To enter concurrent execution from the start, @code{main} can itself be defined with @code{defbjo}.
In practice, this coloring is less infectious than in many other languages. Most standard I/O functions (such as @code{file-read-text}) are automatically emitted with dual implementations: a synchronous blocking path and an asynchronous suspending path. An ordinary @code{defun} that calls standard I/O similarly gets both versions generated under the hood; when called from a bjoroutine, it transparently executes the non-blocking suspending version:
@codeblock[#:lang "bjolang"]|{
(: size-of (-> string int))
(defun (size-of path) (string-length (file-read-text path))) ;; an ordinary defun
(defbjo (main)
(println (size-of "deck.bjo")) ;; the file is read without blocking a thread
0)
}|
@subsubsection{spawn and channels}
The @code{(spawn (f args ...))} form creates a new concurrent fiber running the invocation @code{(f args ...)}. Arguments are evaluated eagerly in the spawning fiber before transferring execution to the new fiber.
Fibers communicate over channels created with @code{make-chan}. Rather than performing immediate I/O, expressions like @code{(chan-send ch v)} and @code{(chan-recv ch)} construct first-class *events*, which are then synchronized using @code{sync}. Unbuffered channels operate as rendezvous points: sending blocks until a receiver is ready, and receiving blocks until a sender arrives.
We can model our card players as concurrent bjoroutines. Each player receives a channel representing their seat at the table and plays one card per trick (descending by value). The table (@code{main}) draws cards round-robin from each seat:
@codeblock[#:lang "bjolang"]|{
(import "deck.bjo")
(: player (-bjo-> (Vec Card) (Chan Card) void))
(defbjo (player hand seat)
(loop (:for c (list-reverse (list-sort (vec->list hand))))
(:do (sync (chan-send seat c)))))
;; One trick: a card from every seat, in turn.
(: play-trick (-bjo-> (Vec (Chan Card)) (Vec Card)))
(defbjo (play-trick seats)
(loop (:for seat seats)
(:acc (vecing (sync (chan-recv seat))))))
(defbjo (main)
(def deck (shuffle (full-deck)))
(def seats [(make-chan) (make-chan) (make-chan)])
(loop (:for seat seats)
(:for p (range 0 3))
(:do (spawn (player (vec-slice deck (* p 4) 4) seat))))
(loop (:for trick (range 1 5))
(:do (println #"Trick ${trick}: ${(play-trick seats)}")))
0)
}|
@codeblock[#:lang "text"]|{
Trick 1: [K♥ K♦ J♥]
Trick 2: [8♣ 7♣ 4♣]
Trick 3: [6♠ 4♥ 3♣]
Trick 4: [6♦ 3♦ 2♦]
}|
Because channel sends and receives synchronize as a rendezvous, a player cannot get ahead of the table and play multiple cards into the same trick.
@subsubsection{Events are values}
Crucially, calling @code{(chan-recv seat)} does not perform the receive immediately; it simply constructs an event value describing the operation. The actual communication occurs only when the event is passed to @code{sync}. Because events are first-class values, they can be parameterized, composed, and transformed before being awaited:
@read-table{
| Function | Description |
|---------------------------+-------------------------------------------------------------|
| @code{(chan-recv ch)} | Receives a value from channel @code{ch} |
| @code{(chan-send ch v)} | Sends @code{v} on channel @code{ch} |
| @code{(timeout d)} | Fires after the @code{Duration} @code{d} |
| @code{(choose e1 e2 ...)} | Fires when the first of the supplied events occurs |
| @code{(wrap e f)} | Fires when @code{e} occurs, applying @code{f} to its value |
}
A @code{Duration} is a length of time, made with @code{milliseconds}, @code{seconds}, @code{minutes} or @code{hours}: @code{(timeout (seconds 5))}. It has its unit in its type, so a timer cannot be given a bare number, and one too long for its range raises rather than wrapping round.
With these combinators, deadlines and timeouts become simple functions that compose over any event. For instance, we can enforce a response timeout on our card players:
@codeblock[#:lang "bjolang"]|{
(: within (-> Duration (Event %a) (Event (Option %a))))
(defun (within d ev)
(choose (wrap ev #(Some &))
(wrap (timeout d) (fun (u) None))))
(: slow-player (-bjo-> (Chan string) void))
(defbjo (slow-player seat)
(sync (timeout (milliseconds 200)))
(sync (chan-send seat "7♣")))
(defbjo (main)
(def seat (make-chan))
(spawn (slow-player seat))
(println (sync (within (milliseconds 50) (chan-recv seat)))) ;; None
(println (sync (within (milliseconds 500) (chan-recv seat)))) ;; (Some 7♣)
0)
}|
Notice that @code{within} is defined with standard @code{defun} rather than @code{defbjo}. Simply constructing or composing event values does not suspend execution; suspending only occurs when an event is submitted to @code{sync}.
Also notice how the first timeout behaves: when the 50 ms deadline expires before the card arrives, the @code{chan-recv} event is cleanly cancelled without consuming the message. The card remains on the channel, available to be received by the subsequent 500 ms wait.
An operation that is synced right where it is written does not need to be an event first. @code{(chan-put ch v)} does what @code{(sync (chan-send ch v))} does, and @code{(chan-get ch)} what @code{(sync (chan-recv ch))} does. They wait, and are cancelled, exactly as @code{sync} is, but no event value is built, so they are cheaper. Use the event forms when you need an event: to @code{choose} between operations, to @code{wrap} one, or to hand one to a function such as @code{within}. Our players could just as well be written with @code{chan-put}:
@codeblock[#:lang "bjolang"]|{
(defbjo (player hand seat)
(loop (:for c (list-reverse (list-sort (vec->list hand))))
(:do (chan-put seat c))))
}|
@subsubsection{Collecting fiber results: bjo}
While @code{spawn} runs a fiber for its side effects and discards its return value, @code{bjo} returns a joinable promise. You can await this promise using @code{(promise-join p)}, which yields a @code{Result} wrapping either the computed value or an exception if the fiber failed:
@codeblock[#:lang "bjolang"]|{
(def a (bjo (think 3)))
(def b (bjo (think 4)))
(sync (promise-join a)) ;; (Ok 9)
(sync (promise-join b)) ;; (Ok 16)
}|
Because both fibers execute concurrently across thread pool workers, the combined operations complete in roughly 50 ms rather than running sequentially for 100 ms.
@subsubsection{Scopes}
Every fiber runs within an enclosing structured concurrency scope. A scope cannot complete until all fibers spawned within it have terminated, ensuring background tasks never outlive their caller or leak into other components. Furthermore, unhandled exceptions inside spawned fibers propagate outward when the scope terminates rather than being silently dropped.
The top-level @code{main} function acts as an implicit root scope, which is why our earlier card game example cleanly awaited all player fibers. You can create nested scopes with fine-grained cancellation and deadlines using these forms:
@read-table{
| Form | Description |
|---------------------------------------+------------------------------------------------------------|
| @code{(with-scope body ...)} | Waits for child fibers and finalizes open file ports |
| @code{(with-cancel (cancel) body ...)} | Binds a @code{cancel} procedure to signal child fibers |
| @code{(with-deadline d body ...)} | Cancels the scope automatically after the duration @code{d} |
| @code{(with-shield body ...)} | Shields cleanup operations from parent cancellation |
}
@codeblock[#:lang "bjolang"]|{
(: fetch (-bjo-> string void))
(defbjo (fetch name)
(sync (timeout (milliseconds 30)))
(println #"fetched ${name}"))
(: count-up (-bjo-> (Chan int) void))
(defbjo (count-up ch)
(let go ((i 0))
(sync (chan-send ch i))
(go (+ i 1))))
(defbjo (main)
;; Both are fetched when this returns, or the deadline raises.
(with-deadline (seconds 1)
(spawn (fetch "a"))
(spawn (fetch "b")))
(println "both done")
;; count-up never ends on its own. Cancelling the scope stops it
;; the next time it waits.
(def numbers (make-chan))
(with-cancel (cancel)
(spawn (count-up numbers))
(println (sync (chan-recv numbers))) ;; 0
(println (sync (chan-recv numbers))) ;; 1
(cancel (Requested "enough")))
0)
}|
Cancellation in Bjolang is cooperative rather than preemptive. Cancelling a scope signals its fibers, which check for cancellation whenever they next perform a suspending operation (such as waiting on a channel or timer). A fiber that exits due to cancellation is treated as a clean completion rather than a failure.
Depending on the desired lifecycle and supervision strategy, fibers can be started using four distinct forms:
@read-table{
| Form | Scope behavior |
|------------------------------------+----------------------------------------------------------|
| @code{(spawn (f x))} | Awaited by scope; propagates failure on error |
| @code{(bjo (f x))} | Awaited by scope; yields a promise holding value/failure |
| @code{(spawn/daemon (f x))} | Cancelled on scope exit without being awaited |
| @code{(spawn/detached (f x))} | Runs detached; unmonitored by the enclosing scope |
}
The scope forms work in both colours. In a bjoroutine, the wait at the end of the scope suspends. In ordinary code, a @code{defun} or a @code{(fun ...)}, it parks the thread until the children have ended, as @code{sync/blocking} does. So an ordinary function can start fibers and know that they are done when it returns:
@codeblock[#:lang "bjolang"]|{
(: fetch-all (-> (Vec string) void))
(defun (fetch-all names)
(with-scope
(ignore (own! (fun () (println "released"))))
(loop (:for n names)
(:do (spawn (fetch n))))
(println "all started")))
(defun (main)
(fetch-all ["a" "b"])
(println "back in main")
0)
}|
@codeblock[#:lang "text"]|{
all started
fetched a
fetched b
released
back in main
}|
A scope also owns resources. @code{(own! thunk)} registers a release on the innermost scope, and @code{(own-disposable x)} registers @code{x}'s @code{.Dispose}. When the scope ends, the releases run after the fibers have ended, newest first. @code{(release! h)} runs one early, with the handle @code{own!} returned. These are ordinary functions, so they also work in both colours. A release is a plain thunk: it cannot suspend, and a release that blocks also blocks the end of the scope.
Do not park a bjoroutine's thread by accident. A scope in an ordinary @code{(fun ...)} that a bjoroutine calls parks the thread of that bjoroutine, and the compiler warns about it. Write the scope in the bjoroutine itself. Or, if the function the lambda is given to takes a @code{-?->} parameter, write the lambda as a @code{(bjoroutine ...)}.
@subsubsection{Work that is not a fiber}
Performing long CPU-bound computations or calling legacy blocking .NET APIs directly inside a fiber can monopolize thread pool threads, preventing other fibers from running. To avoid blocking the pool, you can offload such operations onto dedicated background threads; both helpers return synchronization events:
@codeblock[#:lang "bjolang"]|{
;; Work that waits: a call that parks its thread.
(sync (blocking #(file-read-text "deck.bjo"))) ;; (Ok "...")
;; Work that computes: gets a thread of its own.
(sync (spawn/thread #(loop (:for i (range 0 1000)) (:acc (summing i))))) ;; (Ok 499500)
}|
Conversely, you may occasionally need to wait on a channel or event from inside synchronous code. While an ordinary @code{defun} cannot suspend with @code{sync}, it can block its caller thread using @code{sync/blocking}. Avoid calling @code{sync/blocking} inside a bjoroutine, as doing so freezes a thread pool worker that other fibers rely on:
@codeblock[#:lang "bjolang"]|{
(: wait-for-card (-> (Chan Card) Card))
(defun (wait-for-card ch)
(sync/blocking (chan-recv ch)))
}|
@subsubsection{Shared state: atomics and transactions}
Channels move values from one fiber to another. When many fibers must read and change the same data, a channel means one fiber that owns the data and answers requests for it. That is often the clearest design, but it is not the only one. Two modules let fibers share data directly:
@read-table{
| Module | What it is for |
|--------------------+-----------------------------------------------------------------------------------|
| @code{(std atomic)} | One value that threads change without a lock: a counter, a flag, a cached result |
| @code{(std stm)} | Changes to several values as one step, and waiting until the values are right |
}
An atomic is the simple case and is described in Part III. The rest of this section is about transactions: software transactional memory, built on the design of the OCaml library kcas.
@strong{Locations and transactions.} A @code{(Loc T)} holds one value. A transaction is a function of one argument, a @code{Tx}, which is the log of the transaction. The function reads and writes locations through the log with @code{tx-ref} and @code{tx-set!}, and @code{atomically} runs it and commits it:
@codeblock[#:lang "bjolang"]|{
(import (std stm))
(def pot (make-loc 0))
(def table (make-loc (list)))
;; Put a card on the table and add its points to the pot, as one step.
(: play! (-> Card int void))
(defun (play! card points)
(atomically
(fun (tx)
(tx-set! tx table (Cons card (tx-ref tx table)))
(tx-set! tx pot (+ (tx-ref tx pot) points)))))
}|
The writes stay in the log until the transaction commits, and the commit makes all of them visible at once. No other fiber sees the card on the table without its points in the pot. If another transaction changed a location that this one read before this one commits, this one runs again with the new values. So a transaction never acts on values that changed under it, and nothing is ever locked.
@strong{The rules for a transaction function.} Because it can run more than once, a transaction function must have no side effects except through its @code{Tx}: no printing, no I/O, no changes to other state. Do those after @code{atomically} returns. A transaction function also can not @code{sync}. It is an ordinary function, and the compiler says so:
@codeblock[#:lang "text"]|{
Type Error at play.bjo:7: calling 'sync' is a yield point, and a yield point is not allowed here.
}|
If a transaction raises an exception, it writes nothing, and @code{atomically} raises the exception. Store immutable values in locations: a change inside a mutable vec is not part of any transaction.
@strong{Waiting with tx-retry.} A transaction that can not go on yet calls @code{tx-retry}. That stops it, and it runs again when a location that it read has changed. There is no condition variable to signal and no wakeup to forget:
@codeblock[#:lang "bjolang"]|{
;; The top card of a pile. Waits while the pile is empty.
(: take-top (-> Tx (Loc (List Card)) Card))
(defun (take-top tx pile)
(match (tx-ref tx pile)
((Cons card rest) (tx-set! tx pile rest) card)
(Nil (tx-retry tx))))
}|
@strong{A transaction is an event.} @code{(atomically-evt f)} is the event that runs and commits @code{f}. While @code{f} retries, the event is not available. So waiting for a condition on shared data combines with everything else that @code{choose} combines: a timeout, a channel, another transaction, and the cancellation of the fiber's scope. A transaction whose branch loses has written nothing. @code{atomically} is @code{(sync (atomically-evt f))}:
@codeblock[#:lang "bjolang"]|{
;; A card from the pile, or None after 100 ms.
(sync (choose (wrap (atomically-evt (fun (tx) (take-top tx pile))) Some)
(wrap (timeout (milliseconds 100)) (fun (_) None))))
}|
Inside a transaction, @code{(tx-or-else tx a b)} runs @code{a}, and if @code{a} retries, undoes its writes and runs @code{b}. If both retry, the whole transaction waits for a change to anything either of them read.
@code{atomically} works in both colours: in a bjoroutine it syncs, and in an ordinary function it blocks the thread. @code{try-atomically} never waits: it gives @code{None} when the transaction retries, which makes it safe in a callback that .NET calls.
@strong{Queues, maps and counters.} Three modules build data structures out of locations, so that their operations are part of the transaction that calls them:
@read-table{
| Module | Type | What it is |
|----------------------------+--------------------+--------------------------------------------------------------------|
| @code{(std stm queue)} | @code{(TQueue T)} | A FIFO queue, bounded or not. Producers and consumers seldom conflict |
| @code{(std stm map)} | @code{(TMap K V)} | A hash map. Changes to different keys seldom conflict |
| @code{(std stm counter)} | @code{(TCounter T)} | A number that fibers on different CPUs add to without conflicts |
}
Their operations take the @code{Tx} first. Here a dealer takes the top card of a shared deck into a player's hand, and counts it, in one step:
@codeblock[#:lang "bjolang"]|{
(import (std stm queue))
(import (std stm map))
(import (std stm counter))
(def deck (make-queue)) ;; a (TQueue Card)
(def hands (make-tmap)) ;; a (TMap string (List Card))
(def dealt (make-tcounter 0)) ;; a (TCounter int)
;; Waits while the deck is empty: queue-pop! retries.
(: deal! (-> Tx string Card))
(defun (deal! tx player)
(def card (queue-pop! tx deck))
(tmap-update! tx hands player
(fun (hand) (Some (Cons card (match hand ((Some cards) cards) (None Nil))))))
(tcounter-add! tx dealt 1)
card)
(: dealer (-bjo-> string int void))
(defbjo (dealer player n)
(loop (:for i (range 0 n))
(:do (ignore (atomically (fun (tx) (deal! tx player)))))))
}|
Two dealers can run at the same time. Neither card can end up in two hands, or be counted without being dealt. The queue also has the channel names for use outside a transaction: @code{queue-send} and @code{queue-recv} are events, and @code{queue-put} and @code{queue-get} sync them. What a queue adds to a channel is the transaction: take from one queue and put in another, or take from whichever of two queues has a card, as one step.
@strong{Costs.} A transaction that reads and writes the same locations as many others runs again often. Many fibers that write one location cost the most. A @code{TMap} or @code{TCounter} spreads the writes over many locations for that reason. A transaction that only reads never makes another one run again. Transactions also allocate, and a program in which many fibers allocate at once is often limited by the garbage collector: set @code{(runtime (gc-server #t))} in the manifest to give it one heap per CPU. With it, sixteen fibers that add keys to one @code{TMap} have eight times the throughput of one fiber.
@subsubsection{Compared with Go and Hopac}
Go and Hopac are the two points of comparison. Go is the obvious one: goroutines and unbuffered channels are the same model of fibers meeting at a rendezvous. Hopac is the tougher one. It is a Concurrent ML library for F#, running on the same .NET runtime with the same JIT and garbage collector, so the differences between it and Bjolang are in the design rather than in the platform. The main repository has the same benchmarks in all three (@code{bench/bjolang/cmlbench.bjo}, @code{go run . -suite cmlbench} in @code{bench/go}, and @code{dotnet run -c Release -- cmlbench} in @code{bench/hopac}).
Every Bjolang channel operation races the cancellation token of its scope. Neither Go nor Hopac has that built in, so each is measured twice: plain, and with every operation raced against what a program in it has to use to stop its work. In Go that is a @code{select} against @code{ctx.Done()}; in Hopac it is an @code{IVar} (Hopac's write-once variable) that is filled to stop the jobs. Bjolang is measured twice as well: with events and @code{sync}, and with @code{chan-put} and @code{chan-get}, so the difference between the two Bjolang columns is what building an event and syncing it costs. The best of five runs, in nanoseconds per operation, on a Ryzen 9 5900X under Linux with .NET 10 and Go 1.25:
@read-table{
| Benchmark | Bjolang, @code{sync} | Bjolang, @code{chan-put} | Go | Go, @code{ctx.Done()} | Hopac | Hopac, cancel @code{IVar} |
|----------------------------------------+----------------------+--------------------------+-----+-----------------------+-------+---------------------------|
| Ring: 1000 fibers, a million messages | 70 | 73 | 117 | 241 | 51 | 3487 |
| The same ring, one cancel IVar per job | — | — | — | — | — | 195 |
| The same ring inside a nested scope | 63 | — | — | 242 | — | 3427 |
| Spawning a million fibers and waiting | 210 | — | 190 | — | 80 | — |
| An 8-way choose where one branch fires | 181 | 205 | 403 | 451 | 127 | 136 |
}
In the choose row only the sender can use @code{chan-put}, since the receiving side has to use channel-receive (and thus allocates). On the ring, syncing an event costs nothing measurable any more: a send or a receive synced alone does not build an event. For some reason, this ring inside a nested scope is faster in bjolang, and I haven't figured out why yet.
Against Go, Bjolang is ahead on the ring and the choose, and further ahead once Go has to watch @code{ctx.Done()}, which doubles its ring. Go spawns a little faster. Against Hopac it is the other way round without cancellation: Hopac's ring and choose take about 30% less time than Bjolang's, and its spawns about a third. Bjolang's numbers include watching the scope's token at every operation, which Hopac's plain rows do not do. Once cancellation is added, the ring turns. A thousand jobs racing one shared @code{IVar} all take that IVar's lock at every operation, and the ring slows to three and a half microseconds a message. Giving each job an IVar of its own brings it back to 195 ns, at the price of keeping track of a thousand IVars and filling all of them to stop the ring. The choose does not turn: with a cancel IVar Hopac's takes 136 ns, against Bjolang's 181.
The spawn row is about twice what it was with the scheduler Bjolang used before, which handed every fiber to the .NET thread pool, and the choose row about one and a half times. With it (@code{BJO_SCHEDULER=pool} in the environment, kept as a way back) a spawn costs 105 ns and the choose 113. The spawn row is a single fiber spawning a million fibers that do nothing, and it is slow only on many cores: on two or four cores the current scheduler spawns faster than the pool, 61 and 72 ns against 87 and 112. The scheduler that replaced it runs fibers from run queues of its own, in the shape of Go's, because the pool let work that had waited longest wait longer still under load. That is worth more in a loaded service, as the service benchmark below shows, than these two rows on an idle machine.
These are small benchmarks on one machine, and they only measure the cost of the concurrency primitives, not of the work between rendezvous. They do not show what each operation allocates either. A Go channel operation never allocates and a goroutine costs 16 bytes. A Bjolang send or receive synced alone allocates nothing, a @code{choose} 56 bytes and a spawn 136. Hopac allocates 208 bytes a message on the ring, 424 a choose and 184 a spawn. Go is ahead in other ways too: any Go function can block without being written as a bjoroutine and the Go scheduler preempts a goroutine that computes for too long, where a fiber has to be moved off the pool with @code{spawn/thread}.
What Bjolang offers that Go does not is in the CML vocabulary rather than in the numbers. A Go @code{select} is a statement, its cases are fixed in the source, and a channel operation cannot be passed to a function, wrapped, or combined with others. In Bjolang an event is a value, which is what made @code{within} above a two-line function that works on any event, and @code{choose}, @code{wrap}, @code{guard} and @code{with-nack} build new events out of old ones. Go has no structured concurrency either. Waiting for goroutines, stopping them, and reporting their failures are a @code{sync.WaitGroup}, a @code{context.Context} passed to every function and checked at every channel operation, and an @code{errgroup}, written by hand each time. A goroutine that nothing waits for leaks without a sound. In Bjolang a scope does all of that by construction: it waits for its fibers, raises their failures when it ends, releases the files they opened, and cancels them through a token that every @code{sync} watches without being asked. Hopac can run jobs in parallel and wait for all of them, but a job cannot be stopped from the outside, so stopping jobs when one of them fails, or when their owner is done with them, is the same hand-written race against an IVar as in the table.
@subsubsection{A service under load}
The rows above time one primitive at a time on an idle machine. @code{bench/service} is closer to a program: a sharded key-value store in all three languages, with 16 shard fibers that each serve a part of the keys, and clients that send gets, puts and four-key multi-gets, each with a fresh reply channel and a one-second deadline raced against the reply. The three programs do the same work and agree on a checksum. @code{bench/service/drive.py} runs them and takes the median of 15 reps.
The two .NET programs run with the server GC, which keeps a few hundred megabytes and collects a few times a rep. Go's collector keeps the heap small and collects about twenty times as often, so Go runs twice: with its default, and with as much memory as the .NET programs took (@code{GOGC=off} and @code{GOMEMLIMIT} at their peak).
In a closed loop, 1000 clients send 1000 requests each, each one as soon as the last is answered, so the machine is always saturated. Wall time, CPU time, p99.9 latency and peak memory:
@read-table{
| Program | all cores: wall | CPU | p99.9 | memory | 4 cores: wall | CPU | p99.9 |
|------------------+-----------------+---------+--------+--------+---------------+---------+--------|
| Go | 397 ms | 5552 ms | 5.7 ms | 27 MB | 475 ms | 1832 ms | 3.4 ms |
| Go, same memory | 258 | 4076 | 3.2 | 364 | 414 | 1641 | 3.5 |
| Hopac | 411 | 8903 | 9.8 | 387 | 296 | 1161 | 4.2 |
| Bjolang | 254 | 5211 | 5.5 | 369 | 407 | 1571 | 4.7 |
}
On all cores Bjolang and Go with the same memory finish together, and Go uses a fifth less CPU and has the shorter tail. On four cores Hopac is clearly the fastest, and Bjolang and Go are close.
In an open loop, 10000 clients send on a fixed schedule at a given rate, so the three are compared at the same load. Latency is taken from when a request was due, so a client that woke late, waited for a core, or slept through a GC pause counts that time too. p99 and p99.9 in milliseconds:
@read-table{
| Load | Go | Go, same memory | Hopac | Bjolang |
|----------------------+-------------+-----------------+-------------+-------------|
| 4 cores, 250k/s | 5.6 / 9.0 | 2.0 / 2.2 | 2.8 / 4.3 | 2.2 / 3.1 |
| 4 cores, 500k/s | 28.0 / 32.6 | 41.9 / 48.6 | 6.9 / 8.3 | 3.1 / 3.7 |
| all cores, 500k/s | 7.0 / 9.1 | 8.0 / 12.7 | 24.5 / 31.1 | 3.6 / 6.3 |
| all cores, 1M/s | 33.9 / 37.1 | 69.4 / 73.5 | 27.5 / 32.3 | 5.9 / 8.6 |
}
A client that is early sleeps until its request is due, in whole milliseconds rounded up, the same in all three programs, so every latency here has up to a millisecond of that in it, and below about 2 ms the numbers say little. At 250k/s on four cores the three are close, and Go with the same memory has the shortest tail. At every higher load Bjolang has the shortest, and at a million requests a second its tail is a quarter of Hopac's or less, and a fifth of Go's or less. Some rows move from run to run: over three runs at 500k/s on all cores, Go with the same memory had a p99 of 2.1 to 8.0 ms and Bjolang of 3.6 to 5.0, so that row is not settled. Go's tail at 500k/s on four cores is long with either memory, and its time once a request has started is as long, so the delay is in running the requests, not before them. Hopac keeps all 24 cores busy on all cores at either rate, and why has not been looked into.
The tables in @code{BjolangRuntime/Cml/design.md} in the main repository have every column, including the time a request took once its client had started it.
@subsection{Effects}
Our deck is shuffled using @code{shuffle-vec}, which is non-deterministic. While randomness is ideal during gameplay, it makes automated tests brittle when verifying deterministic outcomes like trick winners. The traditional object-oriented or functional workaround is dependency injection—passing an explicit shuffle function down through every layer of the call stack.
Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. With effects, a function simply declares and performs an abstract operation, allowing an enclosing caller higher up the call stack to dynamically decide how that operation is handled.
@subsubsection{Declaring an effect}
@codeblock[#:lang "bjolang"]|{
(import (std effect) "deck.bjo")
(defeffect Dealing
(shuffle-deck (-> (Vec Card) (Vec Card)))
#:default ((shuffle-deck shuffle)))
(: deal-hand (-> int (Vec Card)))
(defun (deal-hand n)
(vec-slice (shuffle-deck (full-deck)) 0 n))
}|
@code{shuffle-deck} is exposed as a regular function matching the declared signature. When called without an active handler in the dynamic scope, it falls back to the default implementation—in this case, the real shuffle. In a test, you can override this behavior using @code{with-handler}:
@codeblock[#:lang "bjolang"]|{
(deal-hand 5) ;; five random cards
(with-handler ((shuffle-deck (fun (deck) deck)))
(deal-hand 5)) ;; [2♣ 3♣ 4♣ 5♣ 6♣], every time
}|
The handler intercepts operations executed anywhere within the dynamic extent of the @code{with-handler} block, including inside deeply nested helper calls and child fibers, without requiring @code{deal-hand} to pass any parameters around.
Several key semantics govern effect handlers:
@read-list{
- Default implementations: If an effect omits @code{#:default} and is performed without an active handler, the runtime raises an exception. Omit defaults for operations that always require explicit environmental context.
- One-shot execution: Handlers in Bjolang behave like standard functions and return a single value. Full multi-shot delimited continuations are not supported: a handler cannot resume its caller multiple times or abort resumption except by throwing an exception.
- Effect coloring: Operations declared with @code{->} can be invoked from any standard function. Operations declared with @code{-bjo->} may only be invoked from within bjoroutines, and their handlers must likewise be bjoroutines.
- Handler composition: Invoking an operation from inside its own handler invokes that handler recursively. To delegate to an enclosing outer handler, retrieve it using @code{(handler-of op)}:
}
@codeblock[#:lang "bjolang"]|{
(with-handler ((log (let ((outer (handler-of log)))
(fun (s) (outer (str "[game] " s))))))
(log "dealing")) ;; "[game] dealing" on stderr
}|
@subsubsection{The prelude's own effects}
Several core runtime facilities in the prelude are implemented as effects out of the box, allowing you to intercept or virtualize them in tests without modifying application code:
@read-table{
| Operation | Default |
|----------------------------+----------------------------------------------------------|
| @code{(log s)} | Writes a line to standard error |
| @code{(warn s)} | Writes a warning line to standard error |
| @code{(getenv name)} | Reads an environment variable as an @code{Option} |
| @code{(monotonic-ms)} | High-resolution monotonic clock for elapsed timing |
| The @code{FS} operations | Physical filesystem operations |
}
Because all prelude file I/O routes through the built-in @code{FS} effect operations, tests can replace the entire filesystem with an in-memory mock using @code{with-fake-fs} (covered in the Testing section below).
Similarly, you can capture diagnostic output in memory instead of dumping it to the terminal by installing a custom handler for @code{log}:
@codeblock[#:lang "bjolang"]|{
(def lines (make-box (list)))
(with-handler ((log (fun (s) (box-set! lines (Cons s (box-ref lines))))))
(log "one")
(log "two"))
(box-ref lines) ;; '("two" "one")
}|
Keep in mind that @code{with-handler} only establishes a dynamic binding; unlike a concurrency scope, it does not await background fibers. If code inside the handler spawns asynchronous fibers that emit logs, ensure the concurrency scope enclosing those fibers is placed *inside* the @code{with-handler} block so the handler remains active until all fibers finish writing.
@subsection{.NET interop}
Because Bjolang compiles directly to C#, you have full, seamless access to the entire .NET ecosystem. By declaring the CLR classes, methods, and properties you need along with their signatures, the compiler exposes them as ordinary Bjolang functions and types.
@subsubsection{Methods: import/extern}
For example, we can render suit symbols in color in the terminal using .NET's @code{System.Console}:
@codeblock[#:lang "bjolang"]|{
(import "deck.bjo")
(import/class
(ConsoleColor (: System.ConsoleColor)))
(import/extern
(set-foreground! (: System.Console.ForegroundColor (-> ConsoleColor void) #:set))
(reset-colour! (: System.Console.ResetColor (-> void)))
(console-write (: System.Console.Write (-> string void))))
(: suit-colour (-> Suit ConsoleColor))
(defun (suit-colour s)
(match s
((or Hearts Diamonds) ConsoleColor.Red)
((or Clubs Spades) ConsoleColor.Blue)))
(: print-card (-> Card void))
(defun (print-card c)
(set-foreground! (suit-colour (record-ref c suit)))
(console-write #"${c} ")
(reset-colour!))
(defun (main)
(vec-for-each print-card (vec-slice (shuffle (full-deck)) 0 5))
(println "")
0)
}|
Each entry in @code{import/extern} specifies a local Bjolang identifier, the fully qualified CLR member, and its signature. For overloaded methods (such as @code{Console.Write}), the compiler determines which overload to bind based on the provided signature. Instance methods accept the target object as their first parameter. Properties and fields can be accessed via @code{#:get} and modified with @code{#:set}:
@codeblock[#:lang "bjolang"]|{
(import/extern
(trim (: System.String.Trim (-> string string)))
(max-int (: System.Int32.MaxValue #:get)))
(trim " hi ") ;; "hi"
max-int ;; 2147483647
}|
.NET's own string, @code{System.String}, is UTF-16 and a different type from @code{string}. A .NET method that takes or returns one, or an array of them, is converted where it is called, so @code{trim} above takes and gives a Bjolang string; a @code{null} from .NET arrives as @code{""}. A .NET string inside another type, such as a @code{(Seq System.String)}, is not converted: @code{string->clr-string} and @code{clr-string->string} do it by hand.
Avoid binding an external method to the name of an existing prelude function. Locally bound names and prelude symbols take precedence over @code{import/extern} declarations, which means an alias named @code{println} would simply be shadowed and ignored.
@subsubsection{Classes: import/class}
The @code{import/class} form maps a .NET class to a Bjolang type, exposing its constructor as a function suffixed with a dot (e.g. @code{StringBuilder.}). CLR enums map similarly, with individual members accessed using dot notation like @code{Type.Member} (seen with @code{ConsoleColor.Red} above).
@codeblock[#:lang "bjolang"]|{
(import/class
(StringBuilder (: System.Text.StringBuilder (-> StringBuilder))))
(def sb (StringBuilder.)) ;; new StringBuilder()
(ignore (.Append sb "hello, "))
(ignore (.Append sb "world"))
(.ToString sb) ;; "hello, world"
(.-Length sb) ;; 12
(.ToUpper "shout") ;; "SHOUT"
}|
A method that @code{string} does not have, like @code{ToUpper}, is called on the string converted to .NET's, so its indices and lengths are .NET's UTF-16 ones and not @code{string-length}'s bytes.
As shown in the example above, you do not need to declare every method up front with @code{import/extern}. Once an object's type is known to the type checker, you can invoke any public method using @code{(.Method obj args ...)} and inspect properties using @code{(.-Property obj)}. Because @code{Append} returns the builder itself and Bjolang enforces explicit handling of return values, discard unused returns intentionally using @code{ignore}.
@subsubsection{When .NET fails}
Idiomatic .NET code reports errors by throwing exceptions. Bjolang provides three distinct ways to translate these exceptions into safe value types:
@read-list{
- Generic @code{try} blocks: Wrap arbitrary code in @code{try}, as introduced in Part 1.
- Exception-mapped constructors: Annotating a constructor with @code{#:exceptions} in @code{import/class} causes the constructor to return a @code{Result} instead of throwing on specified exceptions.
- Output parameter conversion: Common .NET methods adhering to the @code{bool TryParse(..., out T result)} pattern automatically transform into Bjolang functions that return an @code{(Option T)}.
}
@codeblock[#:lang "bjolang"]|{
(import/class
(Uri (: System.Uri (-> string Uri)
#:exceptions (System.UriFormatException))))
(import/extern
(parse-int (: System.Int32.TryParse (-> string (out int) (Option int)))))
(match (Uri. "https://example.com/cards")
((Ok u) (.-Host u)) ;; "example.com"
((Err e) (.-Message e)))
(parse-int "42") ;; (Some 42)
(parse-int "forty-two") ;; None
}|
@subsubsection{Async methods, and blocking calls}
A .NET method returning a @code{Task} or @code{Task<T>} can be imported using the @code{#:async} modifier. Calling an async method from within a bjoroutine suspends the fiber until the task completes without blocking an OS thread, with no explicit @code{await} required:
@codeblock[#:lang "bjolang"]|{
(import/extern
(read-text-async (: System.IO.File.ReadAllTextAsync (-> string string) #:async)))
(defbjo (main)
(println (string-length (read-text-async "deck.bjo")))
0)
}|
Conversely, CLR methods known to perform synchronous, thread-blocking I/O should be tagged with @code{#:blocking}. Calling a blocking method from within a bjoroutine triggers a compiler warning; you can resolve this warning by offloading the call to a dedicated thread pool thread via @code{(sync (blocking (fun () ...)))}.
@subsubsection{Generics}
Generic .NET types are imported by parameterizing them with type variables, and generic methods specify those type variables in their signature:
@codeblock[#:lang "bjolang"]|{
(import/class
((Dict %k %v) (: System.Collections.Generic.Dictionary)))
(import/extern
(dict-try-get (: System.Collections.Generic.Dictionary.TryGetValue
(-> (Dict %k %v) %k (out %v) (Option %v)))))
}|
At the boundary between systems, Bjolang types map cleanly to standard .NET equivalents: a @code{Func<A, B>} corresponds to @code{(-> %a %b)}, @code{IEnumerable<T>} maps to @code{(Seq %a)}, C# tuples map to @code{Tuple}, and native arrays @code{T[]} correspond to @code{(Array %a)}. Note that constructing generic .NET classes dynamically and pattern matching on .NET objects via constructor patterns are not currently supported.
@subsubsection{Culture and formatting}
Bjolang programs execute using the invariant culture by default. For example, floating-point numbers format with a decimal point (@code{2.5}), avoiding regional discrepancies on machines where local culture settings would otherwise produce @code{2,5}. If your application requires localized formatting, you can configure @code{CultureInfo.CurrentCulture} explicitly.
@subsubsection{Calling Bjolang from C#}
Every Bjolang module compiles to a public static class named @code{<module>_Module}, exposing top-level functions defined with @code{defun} as static methods and @code{def} values as static fields. Identifiers with characters invalid in C# are automatically mangled (for example, hyphenated names like @code{full-deck} become @code{fullsubdeck}), while collections remain Bjolang's persistent data structures. You can inspect the emitted C# source code directly by passing the debug flag: @code{bjo build -d}, which generates @code{out.cs}.
@subsection{Macros}
Macros are functions executed by the compiler during the expansion phase. A macro receives an unevaluated syntax tree as input data and returns a new syntax tree that replaces the invocation form. In fact, many constructs in Bjolang that appear to be built-in language keywords—such as @code{cond}, @code{when}, @code{type/derive}, and @code{with-test}—are actually implemented as standard library macros.
@subsubsection{def/macro}
Macros are defined using @code{def/macro}. A macro transformer accepts three parameters: @code{form}, @code{inject}, and @code{compare}. Most macros are written using pattern matching via @code{syntax-match} from the @code{(std syntax-match)} library:
@codeblock[#:lang "bjolang"]|{
(import (std syntax-match))
;; (with-each-card (c deck) body ...) runs the body once for every card.
(def/macro (with-each-card form inject compare)
(syntax-match form
((_ (name deck) body ...)
#'(vec-for-each (fun (,name) ,@body) ,deck))
(bad (syntax-error bad "(with-each-card (name deck) body ...)"))))
}|
The @code{syntax-match} form decomposes syntax by structural shape. A wild-card pattern @code{_} matches the macro name itself, variable names bind matched syntax fragments, and ellipsis @code{...} matches repeated sequences. Templates are constructed using syntax quotation @code{#'(...)}, where comma @code{,x} unquotes an individual syntax value and @code|{,@xs}| splices a list of syntax objects into the form. Malformed invocations can be rejected at compile time using @code{syntax-error}, which highlights the offending source location.
Pattern fragments like @code{((name value) ...)} match lists of pairs and bind @code{name} and @code{value} to lists of syntax elements. Quoted symbols such as @code{'else} match that literal identifier.
Because macro transformers are compiled code executed directly by the compiler while reading subsequent source files, a macro cannot be used in the same module where it is defined:
@codeblock[#:lang "text"]|{
'twice' is a macro defined in this module, and a macro cannot be used where it is defined,
at bad2.bjo:5. Its transformer runs inside the compiler, so it has to be compiled before
whatever uses it is read — which cannot be true of the file it is written in. Move it to a
module of its own and import that. An (include ...) will not do: an included file becomes
part of this one.
}|
For our card game, we place macro definitions in a dedicated module, @code{cardmacros.bjo}. Importing that module with @code{(import "cardmacros.bjo")} automatically makes its macros available. Unlike functions, macros do not require an explicit export list—every macro defined in a module is exported automatically.
The input passed to a macro is a value of type @code{Syntax}, a union type with variants like @code{(SSym s)} for symbols, @code{(SInt text)} for numbers, and @code{(SList items)} for parenthesized forms. Because macros are ordinary Bjolang functions, you can invoke helper functions, inspect @code{Syntax} records directly, and use recursion.
@subsubsection{Hygiene and inject}
Bjolang macros are hygienic by default. Identifiers introduced within a syntax template are automatically renamed to ensure they never collide with local bindings in the calling scope:
@codeblock[#:lang "bjolang"]|{
;; (or-else a b) is a, unless a is 0.
(def/macro (or-else form inject compare)
(syntax-match form
((_ a b) #'(let ((tmp ,a)) (if (= tmp 0) ,b tmp)))))
;; in another module
(def tmp 9)
(or-else 0 tmp) ;; 9, not 0
}|
Here, the template's temporary variable @code{tmp} and the caller's outer variable @code{tmp} are treated as two distinct symbols, allowing the expression to evaluate correctly to @code{9}. Furthermore, free identifiers referenced in the template (such as @code{let}, @code{if}, and @code{=}) always resolve to their prelude definitions, even if the calling scope has shadowed those names locally.
If a macro intentionally needs to introduce an unhygienic binding into the caller's environment, it can explicitly opt out of renaming using @code{(inject 'name)}:
@codeblock[#:lang "bjolang"]|{
;; (aif test then else): in `then`, `it` is what was inside the Some.
(def/macro (aif form inject compare)
(syntax-match form
((_ test then else)
#'(match ,test
((Some ,(inject 'it)) ,then)
(None ,else)))))
(aif (map-try-ref #map(("a" 1)) "a")
(println #"found ${it}")
(println "nothing"))
}|
@subsubsection{Splicing multiple definitions: begin}
A macro returns a single syntax form. If you need a macro to expand into multiple definitions or expressions at the top level, return a @code{(begin ...)} form, whose child elements are spliced directly into the enclosing scope:
@codeblock[#:lang "bjolang"]|{
;; (def/counter name) defines a function that counts how often it is called.
(def/macro (def/counter form inject compare)
(syntax-match form
((_ name)
#'(begin
(def/mutable count 0)
(: ,name (-> int))
(defun (,name)
(set! count (+ count 1))
count)))))
;; in another module
(def/counter next-id)
(next-id) ;; 1
(next-id) ;; 2
}|
In this expansion, the internal counter variable @code{count} is generated hygienically and cannot collide with any @code{count} variable in the importing module, while @code{next-id} takes on the identifier provided by the caller.
@subsubsection{Pattern macros: def/pattern}
The @code{def/pattern} form defines macros intended for use within pattern-matching positions rather than ordinary expressions. Nullary pattern macros can be referenced simply by their bare capitalized name:
@codeblock[#:lang "bjolang"]|{
;; Face matches a jack, a queen or a king.
(def/pattern (Face form inject compare)
(syntax-match form
(_ #'(or Jack Queen King))))
;; in another module
(: court? (-> Card bool))
(defun (court? c)
(match (record-ref c rank)
(Face #t)
(_ #f)))
}|
Pattern macros expand before the compiler performs exhaustiveness checking, meaning the compiler inspects the expanded @code{or} pattern just as if it had been written by hand.
@subsubsection{Hash macros: def/hash-extend}
Literal prefixes like @code{#map(...)} are reader-level hash macros. You can register custom hash literals using @code{def/hash-extend}. For example, we can implement a custom literal syntax for playing cards, converting ranks and suits into their typed constructors:
@codeblock[#:lang "bjolang"]|{
(import (std syntax-match) "deck.bjo")
(re-export Card Rank Suit)
;; A number is a Pip, a letter a court card.
(: rank-syntax (-> Syntax Syntax))
(defun (rank-syntax r)
(syntax-match r
('J #'Jack) ('Q #'Queen) ('K #'King) ('A #'Ace)
(n (match n
((SInt _) #'(Pip ,n))
(_ (syntax-error n "a rank is 2 to 10, J, Q, K or A"))))))
(: suit-syntax (-> Syntax Syntax))
(defun (suit-syntax s)
(syntax-match s
('clubs #'Clubs) ('diamonds #'Diamonds) ('hearts #'Hearts) ('spades #'Spades)
(bad (syntax-error bad "a suit is clubs, diamonds, hearts or spades"))))
;; #card(Q hearts) is (Card (rank Queen) (suit Hearts)).
(def/hash-extend (card form inject compare)
(syntax-match form
((_ r s) #'(Card (rank ,(rank-syntax r)) (suit ,(suit-syntax s))))
(bad (syntax-error bad "#card takes a rank and a suit: #card(Q hearts)"))))
}|
@codeblock[#:lang "bjolang"]|{
(import "deck.bjo" "cardmacros.bjo")
(defun (main)
(println #card(Q hearts)) ;; Q♥
(println #card[10 spades]) ;; 10♠, either bracket works
(println (court? #card(K clubs))) ;; #t
(with-each-card (c [#card(A spades) #card(2 hearts)])
(println #"a card: ${c}"))
0)
}|
Invalid literal syntax is flagged with a compile-time error at the point of use:
@codeblock[#:lang "text"]|{
The hash macro 'card' failed at bad.bjo:3: a suit is clubs, diamonds, hearts or spades — in hurts
}|
Notice the @code{re-export} in @code{cardmacros.bjo}: because the macro expansion emits constructor symbols like @code{Card} and @code{Queen}, those types must be resolvable in the module invoking the macro. Re-exporting them ensures that importing @code{cardmacros.bjo} alone brings the required types into scope. Importing @code{deck.bjo} alongside it remains recommended so that the corresponding @code{Displayed} implementations are also present.
@subsubsection{Literals read by a grammar}
Many embedded domain-specific languages do not need macros. A grammar says how a quoted literal of a type is written, and declares the type. Where the type checker expects the type, a quoted literal becomes its constructors:
@codeblock[#:lang "bjolang"]|{
(def/grammar Action
(:or (: Play (play int))
(: Say (say string))
(: Pass 'pass)))
(: script (List Action))
(def script '((play 3) (say "your turn") pass (play 1)))
;; = (list (Play 3) (Say "your turn") Pass (Play 1))
(def n 7)
(def (: more (List Action)) '((play ,n) pass))
}|
@code{Action} is an ordinary union, and @code{match} takes its cases. The type checker reads the literal, so a misspelled keyword or a wrong number of items is a compile error that says what can come at that place:
@codeblock[#:lang "text"]|{
Type Error at elab2.bjo:5: Action: expected (play ...), (say ...) or `pass`; found (sya ...).
Type Error at elab2.bjo:5: (play ...): expected the end of the list; found the number 2.
}|
A grammar reads a literal only where the type checker already expects one of its types, which is why @code{script} and @code{more} have signatures. A grammar can also have several productions, items in any order with @code{:each}, optional items, repeats with @code{...} and @code{...+}, parameters, and atoms that a function checks when the literal is compiled. @code{(std run)}, @code{(std fmt)} and @code{(std url)} use grammars for pipelines, layouts and URLs. The language reference describes every form: see @code{Docs/Syntax.org} and @code{Docs/Grammar-design.org} in the Bjolang repository.
@subsubsection{Computing a value once}
A macro often builds a value out of constants, and that value is then built again every time the expansion is evaluated. When that is expensive, wrap it in @code{please-hoist-i-promise-i-am-not-naughty}: the form is evaluated the first time it is reached, and every later evaluation gets that same value. This is what @code{#rx(...)} does, so a regex is compiled once rather than once per match.
@codeblock[#:lang "bjolang"]|{
;; (tag-set "a" "b" ...) is a HashSet of the strings, built once.
(def/macro (tag-set form inject compare)
(syntax-match form
((_ tag ...)
#'(please-hoist-i-promise-i-am-not-naughty
(list->hashset (list ,@tag))))))
}|
If computing the value throws, nothing is kept, and the next evaluation tries again. The same form written in two places is one value.
The long name is a promise about what the compiler cannot check: that the form has no side effects worth repeating, and that it does not depend on things that change from call to call, such as a parameter or an effect handler. Everything it can check, it does, and a form that breaks a rule is a compile error that lists every reason. The form cannot:
@itemlist{
@item{refer to a local variable or a function parameter;}
@item{read a @code{def} of the module it is in, or any @code{def/mutable};}
@item{use a type variable of the function around it;}
@item{call a bjoroutine, @code{yield} to a sequence, or leave a @code{with-return};}
@item{be void, or produce something that can be changed in place, such as an array, a box or a mutable collection.}
}
It may use literals, functions, and the defs of other modules. A macro that puts its caller's code inside the form passes these rules on to its caller, so keep the form to parts the macro built itself.
@subsubsection{One expansion for both colours: by-colour}
A macro does not know if its expansion will land in a bjoroutine or in ordinary code. When the two need different code, the expansion writes @code{(by-colour (#:sync body ...) (#:bjo body ...))}, and the compiler keeps the body for the colour of the code the expansion is in. This is how @code{with-scope} works in both colours: it ends the scope with a suspending wait in a bjoroutine and with a parking wait in ordinary code.
@codeblock[#:lang "bjolang"]|{
;; (await ev) syncs on ev, in code of either colour.
(def/macro (await form inject compare)
(syntax-match form
((_ ev) #'(by-colour (#:sync (sync/blocking ,ev))
(#:bjo (sync ,ev))))))
}|
The colour is the colour of the innermost of these:
@read-table{
| Where | Body |
|------------------------------------------------------------------------+--------------------|
| @code{defun}, @code{(fun ...)}, @code{(seq ...)}, top-level @code{def} | @code{#:sync} |
| @code{defbjo}, @code{(bjoroutine ...)}, @code{(bjo ...)} | @code{#:bjo} |
| @code{defbjouble} | the body it is in |
| a @code{defun} inside a body | the body around it |
}
A @code{defun} with a @code{by-colour} in it gets a suspending copy, as a @code{defun} that calls @code{file-read-text} does, and a bjoroutine calls that copy. If the @code{#:bjo} body never suspends, the copy is dropped, and the @code{#:sync} body runs everywhere. So write a @code{#:bjo} body that suspends and a @code{#:sync} body that does the same work without suspending.
A lambda applied where it is written, @code{((fun () body))}, is replaced by its body before colours are found. A @code{by-colour} in it has the colour of the code around it.
@subsubsection{Macro limitations}
When designing macros, keep the following constraints in mind:
@read-list{
- Macros cannot be invoked within the module that defines them; they must be imported from a separate module.
- Macros can only appear in expression or definition positions (or in pattern positions when defined with @code{def/pattern}). They cannot be used as loop clauses, let bindings, or type annotations.
- Macros cannot override built-in special forms; a macro named @code{if} will never shadow the compiler primitive.
- Macros cannot inspect types, as macro expansion occurs prior to type checking.
- Macros are not first-class values and cannot be passed as function arguments.
- Macros cannot inject imports into the calling module; any imported dependencies must be declared explicitly in the consuming file.
}
@subsection{Testing and the REPL}
@subsubsection{Tests with (std simpletest)}
Bjolang includes a lightweight test harness in @code{(std simpletest)}. Its core assertions are straightforward: @code{(expect label actual wanted)} checks equality between values, @code{(expect-true label ok)} asserts a boolean condition, and @code{(fail label)} explicitly flags an unreachable branch. Passing checks output @code{ok: label}, while test failures emit descriptive @code{FAILURE:} diagnostics:
@codeblock[#:lang "bjolang"]|{
;; tests/deck-test.bjo
(import (std simpletest) (cardlib deck))
(defun (main)
(expect "a deck has 52 cards" (vec-length (full-deck)) 52)
(expect "an ace is worth 11" (card-points (Card (rank Ace) (suit Spades))) 11)
(expect-true "shuffling keeps the cards" (= (vec-length (shuffle (full-deck))) 52))
0)
}|
@codeblock[#:lang "text"]|{
$ bjo run tests/deck-test.bjo
ok: a deck has 52 cards
ok: an ace is worth 11
ok: shuffling keeps the cards
}|
Whenever an assertion fails, the output reports both the computed value and the expected value, each as @code{written} gives it: @code{FAILURE: deliberately wrong gave 2 but wanted 3}, or @code{gave "4" but wanted "5"} for strings. So @code{expect} needs a @code{Written} for the type, which every type has.
For asynchronous or resource-intensive testing, @code{(with-test "name" body ...)} executes the test body within an isolated structured concurrency scope configured with a default deadline. Any background fibers spawned during the test are automatically awaited, and the test fails if unclosed file ports remain open when the scope exits. Because @code{with-test} forms a concurrency scope, it must be invoked from within a bjoroutine.
You can also mock filesystem interactions using @code{(with-fake-fs ((path contents) ...) body ...)}, which redirects file operations to an in-memory map for the duration of the enclosed block. Here is how we can verify our earlier high-score tracking functions without touching real files on disk:
@codeblock[#:lang "bjolang"]|{
(import (std simpletest) (std effect) "scores.bjo")
(defbjo (main)
(with-test "no file means no scores"
(with-fake-fs ()
(expect "empty" (read-scores) [])))
(with-test "a bad line is skipped"
(with-fake-fs (("scores.txt" "ada 31\nnonsense\nbo 27\n"))
(expect "two scores" (read-scores) [(Tuple "ada" 31) (Tuple "bo" 27)])))
(with-test "a saved score reads back"
(with-fake-fs (("scores.txt" "ada 31\n"))
(save-score "cy" 12)
(expect "appended" (vec-length (read-scores)) 2)))
0)
}|
Because @code{with-fake-fs} is implemented via algebraic effects, test modules using it must also import @code{(std effect)}. In standard project layouts, test files live in @code{tests/} and can be executed individually using @code{bjo run tests/<test-name>.bjo}.
@subsubsection{The REPL}
You can launch an interactive Read-Eval-Print Loop by running @code{bjo repl}. When invoked inside a project directory, the REPL automatically discovers and allows importing any of the project's local modules. At a terminal the REPL edits its own lines and keeps a history across sessions. Pressing Enter while a bracket is still open starts a new line, so a long form can be written and edited over several lines. Submitting an expression evaluates it and prints the result, while entering definitions compiles them and prints the bound names:
@codeblock[#:lang "text"]|{
bjo> (+ 1 2)
3
bjo> (import "deck.bjo")
bjo> (vec-slice (full-deck) 0 2)
[(Card (rank (Pip 2)) (suit Clubs)) (Card (rank (Pip 3)) (suit Clubs))]
bjo> (displayed (vec-slice (full-deck) 0 3))
"[2♣ 3♣ 4♣]"
bjo> (defun (double (: x int)) : int (* x 2))
double
bjo> (double 21)
42
}|
The REPL shows a result with @code{written}, as the reader reads it: a string in quotes, a card as it is built. @code{println} and @code{displayed} give the text a person reads.
Unlike many Lisp REPLs, Bjolang's REPL does not interpret forms. Instead, each entered interaction is compiled on the fly into an in-memory assembly using the exact same compiler pipeline applied to files on disk, ensuring identical semantics between interactive sessions and compiled production builds.
Top-level functions declared in the REPL require explicit type signatures, mirroring the rules for modules. You can supply parameter and return types inline directly on the @code{defun}, or submit a standalone signature on its own line immediately preceding the definition:
@codeblock[#:lang "text"]|{
bjo> (: triple (-> int int))
bjo> (defun (triple x) (* x 3))
triple
}|
Redefining a symbol shadows the previous definition for subsequent interactions; however, any existing code already compiled against the earlier version continues to invoke the original binding:
@codeblock[#:lang "text"]|{
bjo> (defun (f (: x int)) : int (+ x 1))
f
bjo> (defun (g (: x int)) : int (f (f x)))
g
bjo> (defun (f (: x int)) : int (* x 100))
note: f shadows the one from entry 1. Anything already compiled against that one still calls it.
f
bjo> (g 10)
12
}|
The same incremental compilation model applies to trait implementations (@code{impl}): an implementation entered at the prompt is visible to subsequent entries, but cannot retroactively modify calls compiled earlier.
@code{:show name} prints the documentation of a name you can use at the prompt: what it is, where it comes from, how it is called, and the text its author wrote in a @code{(:doc ...)} form. It then lists the other usable names containing the text, in any case, with their modules and one-line summaries. Only names that an entry typed now could use are considered, that is builtins, names your imports bring in, and names defined at the prompt. So @code{:show random-bool} prints nothing until @code{(std random)} is imported:
@codeblock[#:lang "text"]|{
bjo> :show random-bool
bjo> (import (std random))
bjo> :show random-bool
random-bool: function from (std random)
(random-bool #:rng (parameter-ref current-random))
: (-> (#:rng RandomSource) bool)
#t or #f, equally likely.
#:rng The source to draw from.
returns One random bit. They are drawn 64 at a time.
See also: random-chance?
bjo> :show random-u
random-u32 (std random) 32 random bits.
random-u64 (std random) 64 random bits.
}|
Import modifiers apply as they would in an entry: after @code{(import (prefix (std random) "r/"))} the name to ask about is @code{r/random-bool}, and @code{:show} says that it is @code{random-bool} from @code{(std random)}. A name with no documentation is shown with where it comes from and "with no doc".
Tab completes the name at the cursor from the same names, together with the special forms and the names the entry you are typing has already bound. When only one name fits it is inserted, and when several do, the part they share is inserted and pressing Tab again opens a menu showing each name's signature and summary. Inside an import, Tab completes module paths instead: @code{(import (std ra} becomes @code{(import (std random}.
The same completions are available as a command, @code{:complete text}, which prints the names that could be written at the end of @code{text}, one per line followed by where each comes from. It is meant for editors rather than people: an editor can send the entry up to the cursor and offer what comes back.
@codeblock[#:lang "text"]|{
bjo> :complete (random-in
random-int (std random)
}|
Type @code{:help} to view available REPL commands, or press @code{Ctrl-D} (or type @code{:quit}) to exit.
@subsection{Benchmarking}
Sooner or later a program is fast enough everywhere except one place, and the question becomes how fast that place is, and whether a change made it better. Timing it with a stopwatch gives a number, but not one you can trust: the first calls run code the JIT has not optimized yet, a garbage collection can land in the middle of a measurement, and two runs of the same code on a busy laptop differ by more than most improvements do.
@code{(std bench)} measures in a way that holds up against all of that, and says how sure it is. There are three ways in:
@read-list{
- @code{time-it}, for a quick answer at the REPL: one line, measured properly.
- Benchmark files, with @code{def/bench} and @code{(run-benches)}, for the code you care about: a table, a menu, saved runs to compare with later.
- @code{bjo bench FILE --against REF}, for the question "did my change make this slower": the file run against itself at another commit, the two versions measured in turns.
}
All three are measured the same way, which the section "How a benchmark is measured" below describes.
@subsubsection{time-it}
@code{time-it} measures a body where you write it and prints one line. It is in @code{(std bench)}, and @code{(std stopwatch)} has it too:
@codeblock[#:lang "text"]|{
bjo> (import (cardlib deck))
bjo> (import (std bench))
bjo> (time-it (shuffle (full-deck)))
(shuffle (full-deck)): 1.06 µs ± 0.7% (min 1.05 µs), 3.64 KB/call, 67 samples
}|
The line gives the median time per call, its spread (the median absolute deviation, as a percentage of the median), the fastest sample, how much the body allocated per call, and how many samples it took. The body is warmed up, calibrated and sampled just as a benchmark is, with shorter time targets, so it is done in a second or two.
A body that takes more than a second the first time is not run again. That first run is the measurement, and the line says so. Here the body is a slow recursive Fibonacci:
@codeblock[#:lang "text"]|{
(fib 44): 1.46 s (one cold run: a body over a second is run once)
}|
@code{#:times n} and @code{#:warm-up-rounds w} ask for an exact number of runs instead: @code{w} untimed runs, then @code{n} timed ones, with no calibration. Use them when the body has an effect that has to happen a known number of times:
@codeblock[#:lang "text"]|{
bjo> (time-it #:times 1000 (shuffle (full-deck)))
(shuffle (full-deck)): 3.82 µs (min 3.02 µs), 3.64 KB/call, 1000 runs
}|
The median here is more than three times the first one: without a warm-up, the early runs are the JIT's unoptimized code. That is the reason to let @code{time-it} decide how often to run a body.
@subsubsection{A benchmark file}
For code you want to keep measuring, write a benchmark file. Benchmarks belong in @code{benches/} in a project, next to @code{src/} and @code{tests/}. Here is one for the deck from @code{cardlib}, in @code{benches/deck.bjo}:
@codeblock[#:lang "bjolang"]|{
(import (cardlib deck))
(import (std bench))
(def/bench "full-deck" (full-deck))
(def/bench "shuffle"
#:setup ((deck (full-deck)))
(shuffle deck))
(def/bench "hand points"
#:params ((size [5 13 52]))
#:setup ((hand (vec-slice (shuffle (full-deck)) 0 size)))
(vec-fold (fun (c total) (+ total (card-points c))) 0 hand))
;; Sorting changes the deck it sorts, so every call gets a freshly
;; shuffled one, made outside the time.
(def/bench "sort a shuffled deck"
#:setup-every ((cards (vec->list (shuffle (full-deck)))))
(list-sort cards))
(def/bench "deal with vec-slice" #:group "dealing"
#:setup ((deck (shuffle (full-deck))))
(vec-slice deck 0 5))
(def/bench "deal with a loop" #:group "dealing"
#:setup ((deck (shuffle (full-deck))))
{vecing (vec-ref deck i) (:for i (range 0 5))})
(def/bench "a million hands" #:once 3
(loop (:for i (up-from 0 #:to 1000000))
(:acc total (summing (card-points (vec-ref (full-deck) (% i 52)))))
=> total))
(def/bench "deal from a stacked deck" #:skip "the cheat is not written yet"
(full-deck))
(run-benches)
}|
It is laid out just like a test file. @code{def/bench} registers a benchmark where it is written, and @code{(run-benches)} at the end becomes the file's @code{main}. The body of a benchmark is ordinary forms, like a function body, and the names its options bind are in scope in it. @code{bjo bench} builds every file under @code{benches/} and runs it:
@codeblock[#:lang "text"]|{
$ bjo bench --all
bench: 10 benchmarks, Release build, .NET 10.0.4, X64, 24 cores, gcServer=false
bench: AMD Ryzen 9 5900X 12-Core Processor, Fedora Linux 43
name size median ± min alloc/op gc/1k 0/1/2
full-deck 309 ns 0.3% 307 ns 1.70 KB 0.20/0/0 1 outlier
shuffle 558 ns 0.3% 555 ns 1.94 KB 0.23/0/0 4 outliers
hand points 5 4.67 ns 1.9% 4.52 ns 0 B - 9 outliers
hand points 13 11.3 ns 1.4% 10.7 ns 0 B - 7 outliers
hand points 52 46.2 ns 0.5% 45.5 ns 0 B - 1 outlier
sort a shuffled deck 3.63 µs 0.5% 3.60 µs 3.30 KB 1.00/0/0 9 outliers
deal with vec-slice 19.4 ns 0.2% 19.3 ns 152 B 0.02/0/0 3 outliers
deal with a loop 51.7 ns 0.7% 51.2 ns 656 B 0.08/0/0 4 outliers
a million hands 320 ms - 320 ms 1.62 GB 208/0/0
(324 ms, 320 ms, 320 ms)
deal from a stacked deck skipped: the cheat is not written yet
}|
The first two lines say what the numbers were measured on, because a time is a fact about one machine. A Debug build, an attached debugger and a CPU frequency governor other than @code{performance} add a warning there. @code{bjo bench} always builds in Release, whatever @code{-d} says, since a Debug build measures code the JIT did not optimize.
The columns are:
@read-table{
| Column | What it says |
|----------------------+--------------------------------------------------------------------------------------------------|
| @code{name}, @code{size} | the benchmark, and a column for each of its parameters |
| @code{median} | the median time per call. Not the mean: one collection can move a mean anywhere |
| @code{±} | the median absolute deviation, as a percentage of the median: how much the samples agree |
| @code{min} | the fastest sample, which is what a quiet machine converges to |
| @code{alloc/op} | the bytes allocated per call |
| @code{gc/1k 0/1/2} | the collections of each generation per thousand calls (per call for a @code{#:once} benchmark) |
}
After the columns come the marks. @code{noisy} means a spread above 3%, @code{unsteady} means the warm-up ran out of time before the times settled, and outliers are samples far outside the rest. They are counted and shown, never dropped: a pause that happens once in fifty calls is part of what the call costs.
Each row is printed as soon as its benchmark is measured, so a long run can be read as it goes. At a terminal, a line below the table says where the run is: which benchmark, which phase, the median so far and an estimate of the time left. It is rewritten in place, and is left out when the output goes to a pipe or a file, under @code{CI}, and in an Emacs buffer.
@subsubsection{How a benchmark is measured}
You do not have to know this to use @code{(std bench)}, but it explains its numbers.
@read-list{
- @strong{The body is kept.} The value of the body's last form is written to a sink the JIT cannot remove. A body that computes a value and throws it away is therefore not optimized away, and the sink allocates nothing.
- @strong{Calibration.} The body is run in batches of @code{k} calls, and @code{k} grows until a batch takes 10 ms. One sample is one batch, so the clock's resolution does not matter even for a body of a few nanoseconds.
- @strong{Warm-up.} Batches are run until the medians of the last two windows of five batches agree within 2%, for at least half a second and at most one. The minimum is there because of the JIT. .NET recompiles a hot method into faster code only after a pause with no new compilation, and a body can look steady at the first, slow tier before the faster code arrives. A warm-up that runs out of time leaves the row marked @code{unsteady}.
- @strong{Samples.} Before each sample a collection runs, so that one benchmark's garbage is not collected in another one's time. At least 20 samples are taken, for at least 2 s. While the median's 95% interval is wider than ±1%, it takes more, up to 100 samples or three times the time. A noisy benchmark costs more time, and gets a more precise median for it.
}
Every benchmark of a file runs in the file's one process, one after another, so what one benchmark compiled is there for the next. That is usually what you want: it is how the code runs in a program.
@subsubsection{Setup, parameters, and the rest}
The body of a benchmark should be the thing you want to measure and nothing else. The options of @code{def/bench} are about keeping everything else out of the time:
@read-table{
| Option | What it does |
|------------------------------+--------------------------------------------------------------------------------------------------------|
| @code{#:setup ((x e) ...)} | binds names once, outside the time, for the body to use |
| @code{#:setup-every ((x e) ...)} | binds names again before every call, outside the time; for a body that uses up its input |
| @code{#:params ((p values) ...)} | a row for each combination of values; the setups and the body use the parameters by name |
| @code{#:once n} | for a body of seconds: @code{n} cold runs, timed one at a time, no warm-up |
| @code{#:group "g"} | a heading in the menu, and what @code{--compare-within} compares within |
| @code{#:skip "why"} | not run; the reason is shown |
}
@code{#:setup} is the one to reach for first. Its bindings are written like a @code{let*}'s, so a binding can use the ones before it, and the body uses them by name. In the example, @code{shuffle} measures shuffling a deck, not making one: @code{deck} is made once, before the warm-up, and every call shuffles that same deck.
@code{#:setup-every} is for a body that changes or uses up what it is given, such as a sort that sorts in place, or a body that consumes a list. Its bindings are made again before every call, so each call gets a fresh input, and each call is then timed on its own, with its setup outside the time and outside the allocation. Timing calls one at a time has a cost, because the clock is read twice per call. A body that takes less than a microsecond is marked, since its time is close to the clock's resolution.
The two can be used together. What @code{#:setup} binds is made once, and @code{#:setup-every} can use it to make each call's input. That is the way to keep an expensive input out of the time and still give every call its own copy:
@codeblock[#:lang "bjolang"]|{
(def/bench "sort a big shuffled deck"
#:setup ((cards (shuffle (repeat-deck 1000)))) ; 52 000 cards, made once
#:setup-every ((copy (vec->list cards))) ; a fresh list for each call
(list-sort copy))
}|
@code{#:params} gives the parameters' names and values, like a @code{let}. The values are a vec of any expressions, evaluated once. There is a row for every combination, in the order written, with the last parameter varying fastest. The setups and the body use the parameters by name:
@codeblock[#:lang "bjolang"]|{
(def/bench "sort"
#:params ((n [1000 100000])
(order ['sorted 'reversed 'random]))
#:setup ((cards (make-cards n order)))
(list-sort cards))
;; Without a setup, the body uses them directly.
(def/bench "fib" #:params ((n [10 20 30])) (fib n))
}|
Here @code{make-cards} and @code{repeat-deck} stand for functions of your own.
Each combination is a benchmark of its own for everything else: it is warmed up, calibrated, saved and compared on its own. To try a size the file does not list, give it on the command line: @code{--params n=500,5000} replaces the values of @code{n} for that run. A value is read as the parameter's values are typed: an @code{int}, a @code{double}, a string, a symbol or a @code{bool}. Implementing the trait @code{BenchParam} lets other types be read too.
@code{#:once n} is for bodies too slow to run in batches. There is no calibration and no warm-up, just @code{n} runs timed one at a time, the first of them cold. Each time is shown under the row, as for @code{a million hands} above, and the median is the row's.
@subsubsection{Choosing what runs}
Names on the command line choose benchmarks. A benchmark runs when its name contains one of them, and a parameter written the way the table shows it chooses rows:
@codeblock[#:lang "text"]|{
bjo bench benches/deck.bjo shuffle # shuffle, and sort a shuffled deck
bjo bench benches/deck.bjo "hand points" size=52
bjo bench --list # the names, and nothing run
}|
Note that @code{shuffle} also matches @code{sort a shuffled deck}: a name is a part of a name.
With no names, at a terminal, a benchmark file asks what to run:
@codeblock[#:lang "text"]|{
$ bjo bench
Benchmarks in deck.bjo:
#0 full-deck
#1 shuffle
#2 hand points size: 5 13 52
#3 sort a shuffled deck
dealing
#4 deal with vec-slice
#5 deal with a loop
#6 a million hands
#7 deal from a stacked deck
Choose by number (0 2-4), by name (sort), with parameters (2 n=1000),
or * for all. Nothing quits, or repeats the last choice once there is one.
bench> 2 size=5,52
}|
A choice is numbers, ranges of numbers (@code{2-4}), parts of names, or @code{*} for everything, in any mix and separated by spaces. A parameter after a benchmark narrows that benchmark to some of its values: @code{2 size=5,52} runs @code{hand points} at sizes 5 and 52 only. The groups and the parameters' values are shown, so a choice can be made without opening the file.
After a run the menu asks again. Pressing Enter repeats the last choice, so you can change the code in another terminal and run the same benchmarks again with one key. @code{q} or the end of the input leaves.
With several files and no names, @code{bjo bench} makes one menu of all of them, numbered across the files under a heading for each, and runs each file with the part of the choice that is its own. Not at a terminal, when the output is redirected or @code{CI} is set, there is no menu: a run with no names runs everything, as @code{--all} does.
@subsubsection{More numbers}
@code{--detail} adds three lines under each row: the mean, the standard deviation and the quartiles; the samples in the order they were taken; and a histogram of them:
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo "hand points" --detail
name size median ± min alloc/op gc/1k 0/1/2
hand points 5 6.50 ns 6.6% 5.21 ns 0 B - noisy, 10 outliers
mean 6.69 ns, sd 1.02 ns, q1 6.04 ns, q3 6.90 ns, 100 samples of 1600000 calls
in order ██████▄▂▃▃▃▃▃▂▂▃▁▃▃▃▃▁▂▂▄▂▂▃▃▃▃▂▁▂▄▄▂▂▂▃▃▃▃▃▃▂▂▅▄▁▂▃▃▁▁▃▃▅▂▃
histogram 5.21 ns ▃▂▅▄▄▆█▇▅▂▃▂▂▂ ▁ ▃▂▂▂ 9.64 ns
}|
The "in order" line is the one that explains a strange row. Here the first six samples were slower than the rest, and the spread of 6.6% and the @code{noisy} mark come from those six; the other 94 agree closely. A second run tells whether it was the machine or the code: run again, the same benchmark had a spread of 1.9%. A line that rises from left to right is a body that slows down as it runs, for example because its heap grows. The histogram shows the shape: two humps are two different speeds, not one noisy speed.
@code{--scaling} adds a column to a benchmark with one numeric parameter: how much the time grew from the row before, against how much the parameter grew.
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo "hand points" --scaling
name size median ± min alloc/op gc/1k 0/1/2 scaling
hand points 5 4.94 ns 5.4% 4.47 ns 0 B - noisy
hand points 13 9.99 ns 2.7% 8.41 ns 0 B - ×2.02 for ×2.60
hand points 52 41.5 ns 0.7% 40.5 ns 0 B - ×4.16 for ×4.00 3 outliers
}|
From 13 to 52 cards the time grew by as much as the hand did, which is what a linear function does. From 5 to 13 it grew less than the hand, because a few nanoseconds of every call are not spent on the cards at all, and with five cards those nanoseconds are a large part of the time. A column that keeps growing faster than the parameter is the sign of something quadratic.
@subsubsection{Which of two ways is faster}
"Is dealing with @code{vec-slice} faster than with a loop, and by how much?" is a question about two benchmarks in the same file. Put them in a @code{#:group}, and run with @code{--compare-within}:
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo deal --compare-within
name median ± min alloc/op gc/1k 0/1/2
deal with vec-slice 20.0 ns 0.9% 19.0 ns 152 B 0.02/0/0 2 outliers
deal with a loop 47.3 ns 0.6% 46.8 ns 656 B 0.08/0/0 14 outliers
deal from a stacked deck skipped: the cheat is not written yet
bench: within dealing, against deal with vec-slice: 20 rounds in this process, paired (Wilcoxon signed-rank); 1 comparison
bench: seed 11 (--seed repeats the order of the cases)
name deal with vec-slice this change 95% interval p ratio verdict
deal with a loop 19.5 ns 47.0 ns +140.9% +140.0% .. +141.7% <0.001 ×2.41 SLOWER
}|
After the usual table, the benchmarks of each group are measured again in rounds: every round takes one sample of each, in an order shuffled anew each round. Each one is then compared with the first of its group. Taking turns is what makes the comparison fair. If the machine gets slower halfway through, both sides get slower in the same rounds, and the comparison, which pairs the samples of a round, does not move.
The ratio is the number to quote. "A loop is 2.4 times as slow as @code{vec-slice}" is roughly true on the next machine too, where "47 ns" is not. @code{-N 50} takes 50 rounds rather than 20, and @code{--seed 11} repeats the order of an earlier run. The columns of the comparison, and how a verdict is reached, are explained in the next section.
@subsubsection{Saving a run and comparing with it}
@code{--save FILE} keeps a run: every sample of every benchmark, with the facts of the machine it was taken on. @code{--compare FILE} measures again and compares with what was kept. Nothing is saved unless you ask, and nothing is saved anywhere you did not name. Where a baseline lives, and whether it is committed, is up to you.
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo full-deck shuffle "hand points" --save before.bjodat
...
bench: saved to before.bjodat
}|
The file is bjodat, a benchmark to a line, so a diff of two saved runs shows which benchmarks changed:
@codeblock[#:lang "text"]|{
(bench-run
(version 1)
(saved "2026-10-16T17:25:06Z")
(machine "95c59c6948245e2e")
(header (build "Release") (dotnet ".NET 10.0.4") (arch "X64") (cores "24") ...)
(case (name "full-deck") (labels) (k 60000) (once #f) (bytes 1744.0) (samples [371.62 312.41 ...]))
...)
}|
Now suppose @code{full-deck} is rewritten to collect the cards in a list and turn the list into a vec at the end:
@codeblock[#:lang "bjolang"]|{
(defun (full-deck)
(list->vec
(loop (:for s all-suits)
(:subloop)
(:for r all-ranks)
(:acc deck (listing (Card (rank r) (suit s))))
=> deck)))
}|
The comparison prints the run's own table, and then:
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo full-deck shuffle "hand points" --compare before.bjodat
...
bench: compared with before.bjodat, saved 2026-10-16T17:25:06Z; unpaired (Mann-Whitney U); 6 comparisons, Holm-corrected
bench: one process each: a difference between processes is not in the p-values
name size was now change 95% interval p verdict
full-deck 311 ns 597 ns +92.1% +91.7% .. +92.5% <0.001 SLOWER
shuffle 623 ns 548 ns -12.0% -12.0% .. -11.8% <0.001 faster
hand points 5 4.58 ns 5.07 ns +10.7% +10.1% .. +11.3% <0.001 SLOWER
hand points 13 10.6 ns 10.2 ns -4.2% -5.0% .. -3.7% <0.001 faster
hand points 52 44.2 ns 44.7 ns +1.0% +0.4% .. +1.4% <0.001 no change (< 2%)
sort a shuffled deck 3.68 µs 3.92 µs +6.8% +6.6% .. +7.0% <0.001 SLOWER
}|
Each benchmark gets four answers:
@read-list{
- @strong{The change} is the ratio of the two medians, as a percentage.
- @strong{The interval} is a bootstrap of that ratio: the samples are resampled 10 000 times, and the middle 95% of the resampled ratios is shown. It makes no assumption about the shape of the timings, which are lopsided and have long tails.
- @strong{The p-value} is the probability of a difference at least this large if nothing had changed, from the Mann–Whitney U test. A run that compares many benchmarks at once will show some differences by chance alone: with 40 benchmarks at a level of 5%, two are expected to look different when nothing changed. So the p-values are corrected for the number of comparisons, by Holm's method, and the corrected value is the one shown and used. @code{--p-raw} shows the uncorrected values beside them.
- @strong{The verdict} is @code{faster} or @code{SLOWER} only when the corrected p is below 0.05 (@code{--alpha}) @em{and} the change is larger than 2% (@code{--threshold}). A change that is real but smaller is @code{no change (< 2%)}: it is there, and it was decided not to matter. Anything else is @code{no change}.
}
The reason for asking for both is that a p-value says how unlikely a difference is, not how large it is: with enough samples, a change of 0.1% has a tiny p.
Now look at the table again. @code{full-deck} is 92% slower, as it should be: that is the change. But @code{shuffle} did not change at all, since its deck is made in its setup, outside the time. Yet it says @code{faster}, by 12%. The rows for @code{hand points} and the sorting moved too, and they never call @code{full-deck} in their time at all.
This is the weakness of comparing with a saved run. The two runs are two processes, and @strong{two processes of the same program can differ by a few percent}, in every benchmark at once. Where the code ends up in memory, which core the process runs on, and what the JIT decided all differ from one process to the next. Within one run the samples agree to a fraction of a percent, so the test is very sure about a difference that is really between the processes. The report says this in its second line. Treat a saved-run comparison as a tool for large changes, like @code{full-deck}'s. For small ones, use @code{--against} with several process pairs, as described next.
A saved run from another machine is still compared, but under a warning above the table, and every verdict is marked @code{(other machine)}: the numbers are shown, but no verdict is to be believed. Differences in the build, the .NET version, the garbage collector's mode or the CPU governor are warned about too.
@code{--fail-slower PCT} makes the run exit with 1 when anything is @code{SLOWER} by more than @code{PCT}, for a CI job that guards a hot path. It does not apply against a run from another machine. A file that cannot be read stops the run with exit code 2 before anything is measured.
@subsubsection{Two versions against each other: @code{--against}}
The real question behind most benchmarking is "did my change make this slower?". @code{--against} answers it in one run. It builds the benchmark file as it is and as it was at a git ref, and measures the two versions in turns:
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo full-deck shuffle "hand points" --against HEAD~1
bench: 6 benchmarks, Release build, .NET 10.0.4, X64, 24 cores, gcServer=false
bench: AMD Ryzen 9 5900X 12-Core Processor, Fedora Linux 43
bench: A/B against HEAD~1, 20 rounds, paired (Wilcoxon signed-rank); 6 comparisons, Holm-corrected
bench: one process per side: a difference between processes is not in the p-values (--processes P measures it)
bench: seed 42 (--seed repeats the order of the cases)
name size HEAD~1 now change 95% interval p verdict
full-deck 360 ns 649 ns +80.2% +73.9% .. +93.9% <0.001 SLOWER
shuffle 659 ns 616 ns -6.5% -63.1% .. +2.4% 0.097 no change
hand points 5 5.60 ns 5.46 ns -2.4% -5.5% .. +1.5% 0.046 faster
hand points 13 11.5 ns 12.8 ns +11.0% +5.7% .. +17.6% <0.001 SLOWER
hand points 52 47.2 ns 50.3 ns +6.6% +4.7% .. +10.9% <0.001 SLOWER
sort a shuffled deck 4.09 µs 4.14 µs +1.3% -4.5% .. +8.4% 0.50 no change
}|
This is what happens:
@read-list{
- @code{REF} is checked out in a temporary git worktree, which is removed afterwards, also when something fails. It can also be a path: another checkout, or another @code{.bjo} file.
- Both sides are built in Release, by the same compiler. Both programs are started, and @code{bjo bench} drives them over a small protocol on their standard input and output.
- Every benchmark both sides have is warmed up and calibrated on both, with the smaller of the two batch sizes, so that a sample of each is the same number of calls.
- Then come @code{-N} rounds, 20 by default. Each round takes one sample of every benchmark from each side, in an order shuffled from @code{--seed}, which is printed so that a run can be repeated. Which side goes first alternates every round, so that neither side always runs on a cache the other one warmed.
- Each round's two samples are a pair. The p-value is the Wilcoxon signed-rank test on the pairs, and the interval a bootstrap that resamples whole rounds. If the machine slows down during the run, both samples of a round slow down, and the comparison does not move.
}
A benchmark that only one side has is listed, not compared.
Look at the table, though. Rounds cancel a slowdown @em{during} the run, but not the difference between the two processes as a whole, and that difference is still there: @code{hand points} is called @code{SLOWER} by 11% at size 13, and it has not changed. The report says so in its fourth line. This is why there is @code{--processes}:
@codeblock[#:lang "text"]|{
$ bjo bench benches/deck.bjo full-deck shuffle "hand points" --against HEAD~1 -N 60 --processes 6
bench: A/B against HEAD~1, 60 rounds, 6 process pairs (t-test on their log ratios); 6 comparisons, Holm-corrected
bench: seed 42 (--seed repeats the order of the cases)
name size HEAD~1 now change 95% interval p between processes verdict
full-deck 344 ns 614 ns +81.2% +72.4% .. +90.4% <0.001 4.7% SLOWER
shuffle 596 ns 585 ns -2.1% -10.3% .. +6.9% 0.73 8.3% no change
hand points 5 5.55 ns 5.13 ns -9.6% -18.7% .. +0.6% 0.24 10.1% no change
hand points 13 12.1 ns 11.4 ns -7.6% -20.8% .. +7.7% 0.73 14.7% no change
hand points 52 45.2 ns 45.0 ns -3.1% -10.5% .. +4.9% 0.73 7.6% no change
sort a shuffled deck 3.91 µs 4.04 µs +3.6% +0.5% .. +6.7% 0.15 2.8% no change
}|
With @code{--processes 6}, the rounds are spread over six pairs of processes. Each pair gives one ratio of medians, and the test is a t-test on the six ratios, so a difference between processes is now part of what is tested rather than hidden from it. The "between processes" column says how large it was. For these tiny benchmarks it was 3 to 15%, more than any of the "changes" the single pair reported. Now only @code{full-deck} gets a verdict, which is the right answer.
So which to use?
@read-table{
| Question | Use |
|----------------------------------------------------------+-------------------------------------------------------------|
| is it about this fast | @code{time-it}, or a benchmark file |
| which of two ways is faster, and by how much | @code{#:group} and @code{--compare-within}: one process |
| did my change make something a lot slower | @code{--against REF}, or @code{--compare} with a saved run |
| did my change make something a few percent slower | @code{--against REF --processes 6} or more, and more rounds |
}
@code{--against} has a few more options. @code{--against-warmup S} and @code{--against-sample MS} set the warm-up limit and the sample length of both sides. A warm-up much shorter than the default second can mistake the JIT's first tier for steady, and a benchmark that did not get steady is named in a warning. @code{--against-build "CMD"} builds the other side with a command of your own, for a side that is not built the way this one is. @code{--save FILE} keeps both sides' samples: this code's in @code{FILE}, and the other side's beside it, with @code{.against} before the extension. @code{--fail-slower}, @code{--alpha}, @code{--threshold} and @code{--p-raw} work as they do for @code{--compare}.
A CI runner is one machine for the length of one job, which is exactly what @code{--against} needs. A job that guards performance runs @code{bjo bench --against origin/main --processes 6 --fail-slower 5} and fails when something got more than 5% slower.
@subsubsection{The flags}
@read-table{
| Flag | What it does |
|----------------------------+-----------------------------------------------------------------------------|
| @code{NAME} | runs the benchmarks whose names contain it; @code{n=10} chooses rows |
| @code{--all} | runs every benchmark, without the menu |
| @code{--list} | names the benchmarks and their parameters, and runs nothing |
| @code{--quick}, @code{--long} | halves or quadruples the time targets |
| @code{--min-samples N} | takes at least @code{N} samples (20) |
| @code{--max-time S} | lets a noisy benchmark sample for up to @code{S} seconds |
| @code{--params P=V,...} | replaces a parameter's values for the run |
| @code{--detail} | the mean, the deviation, the quartiles, the samples in order, a histogram |
| @code{--scaling} | how the time grows with a numeric parameter |
| @code{--compare-within} | compares each @code{#:group}'s benchmarks with its first, in rounds |
| @code{-N ROUNDS} | the rounds of @code{--compare-within} and @code{--against} (20) |
| @code{--seed N} | repeats the order of an earlier run's rounds |
| @code{--save FILE} | writes every sample of the run to @code{FILE} |
| @code{--compare FILE} | compares with the run saved in @code{FILE} |
| @code{--alpha P} | the significance level of a verdict (0.05) |
| @code{--threshold PCT} | the smallest change a verdict counts (2%) |
| @code{--p-raw} | shows the uncorrected p-values too |
| @code{--fail-slower PCT} | exits with 1 when something is @code{SLOWER} by more than @code{PCT} |
| @code{--against REF} | runs the file against itself at a git ref, or against a path |
| @code{--processes P} | with @code{--against}: @code{P} pairs of processes (1) |
| @code{--against-warmup S} | with @code{--against}: the warm-up limit of both sides (1 s) |
| @code{--against-sample MS} | with @code{--against}: the sample length of both sides (10 ms) |
| @code{--against-build CMD} | with @code{--against}: builds the other side with @code{CMD} |
}
A percentage can be written @code{2} or @code{2%}. A benchmark file is an ordinary program, so @code{bjo run benches/deck.bjo --quick shuffle} runs it just as well. @code{bjo bench} adds the Release build, the files under @code{benches/}, the menu over several files and @code{--against}.
@subsubsection{Driven by another program}
@code{--against} works because a benchmark program can be driven. Started with @code{--serve}, it reads commands on its standard input and answers each with one line of bjodat on its standard output. Below, @code{>} marks what is sent and @code{<} what comes back:
@codeblock[#:lang "text"]|{
> (list)
< (cases (case 0 (name "full-deck") (labels) (group "") (once 0) (skip "")) (case 1 (name "shuffle") (labels) ...) ...)
> (prepare 1 :warmup 1000 :sample 10)
< (prepared 1 :k 20000 :steady #t)
> (sample 1 :k 20000)
< (sample 1 :ns 13663495 :bytes 39680000 :gc (4 0 0))
> (sample 9 :k 1)
< (error "case 9 is not prepared")
> (quit)
}|
@code{prepare} runs a benchmark's setup and warms it up and calibrates it, as a run would. @code{sample} takes one sample of @code{k} calls, and @code{:ns} is the whole sample's time. Anything wrong is answered with @code{(error "why")}, and the session goes on. A served program prints nothing else, so its output is the protocol and nothing more. A script that compares two builds made by two different compilers, which is not something @code{bjo bench} does by itself, can drive both of them this way, or use @code{--against PATH --against-build "CMD"}.
@subsection{Performance}
Bjolang is compiled into pretty straight-forward c#, but without spans or ref structs. It performs pretty similar to c#, except for some small corners of the language that are different.
@subsubsection{Strings}
Strings in bjolang are utf8, as such they do @em{not} support random access. This means we navigate them differently, using string-cursors, which is explained in the part about strings. Navigating a string using cursors is slower than blasting through integer indices, and if your code needs to be really fast, there are some things you can think about:
@read-list{
- Use the provided search functions and the loop form. They work on raw memory, and can thus work around some of the limits that the safety of string-cursor give.
- use @code{string-cursor-next+ref} instead of next and then ref.
- some of the higher order functions are also faster, depending on use case. Benchmark and see.
- Use slices when copying is the bottleneck: an operation ending with @code{/slice} will return a view into the string instead of copying. This will retain the whole string in memory, so use it wisely.
}
When parsing strings with a known format, (std scan) will be a lot faster than (std rx) and the good old string-split. Often by an order of magnitude. The only cost is IL code size, since it builds the parser in the code, which will have other costs, such as no inlining.
@subsubsection{loops}
Most of the time you should use the @code{loop} form. It will be the fastest, and is usually flexible enough to do what you need. The only case is for hot loops (ones that are run often) that are performance sensitive. For those, using the various @code{-fold}, @code{-for-each}, and @code{-iter} functions will be faster for the collections that are tree-based. Those collections are: @code{Vec}, @code{Map}, @code{HashSet}, @code{OrderedMap}, and @code{OrderedMap}. The overhead of loop vs. fold is often something like 3x, with fold being as fast or slightly faster thana @code{foreach} on a C# @code{List<T>}. It does not, however, beat using indexed iteration over the @code{List<T>} due to the @code{foreach} doing some safety checks to detect concurrent mutation.
The absolutely fastest collection to loop over is, unsurprisingly, the Array. In bjolang it should be just as fast as in c# (again, without spans).
The loop form gets turned into a kind of goto soup. For the single level case, it is just as efficient as a c# @code{while} loop, but for really complex cases with many levels and subloops with clauses the JIT in dotnet may struggle. In that case you either live with the clarity and simplicity it gives, or you reach for the glorified gotos of names lets. This is probably not going to be a problem though. I tried to rewrite the worst loop form in bjolang (a 16 line monstrosity with one large multi-level claused subloop and a regular subloop) and there was no difference.
@subsubsection{Choosing a collection}
Bjolang has two pairs of collections that do mostly the same job: @code{List} and @code{Vec}, and @code{Map} and @code{OrderedMap}. Which one you pick matters less for small collections than you think, and more for large ones than you would hope. The tables below come from the benchmarks in @code{bench/collections/} in the bjolang repository (run them with @code{./run.sh}). They were measured on one machine with .NET 10 on Linux, so trust the ratios more than the exact times.
@strong{List or Vec}
@read-table{
| Operation | List | Vec |
|--------------------------------------------+--------------------------+---------------------------|
| memory per int | 32 B | about 7 B |
| first item | 1 ns | 2 ns |
| drop the first item | 1 ns | 21 ns |
| add at the front | 1 ns | 14–200 ns |
| add at the back | copies the whole list | 10 ns |
| last item, length | 1 ns per item | 1–2 ns |
| item at an index | 1 ns per item before it | 1–2 ns |
| read every item (@code{loop}, fold) | 1 ns per item | 0.6 ns per item |
| build with an accumulator, 1000+ items | 1 | 5–8 times faster |
| map, reverse, append, slice, 100+ items | 1 | much faster |
| recursive match that binds the rest | 1 ns per item | 30 ns and 230 B per item |
| match the first items, ignore the rest | 2 ns | 2 ns |
}
Use a @code{List} when the work happens at the front and goes item by item: a stack, a result built with @code{cons}, or a recursive function that matches @code{(Cons h t)} and continues with @code{t}. That is what a list does in constant time, and there it is 3 to 30 times faster than a vec.
Use a @code{Vec} for everything else. It knows its length and last item, it reads any index in a couple of nanoseconds, it is faster to iterate, and it uses about a fifth of the memory. Some things to keep in mind:
@read-list{
- Build vecs with an accumulator (@code{vecing} in @code{loop}, or a vec builder), not with @code{vec-add} in a loop. Each @code{vec-add} copies the tail of the vec, which makes a @code{vec-add} loop 3–4 times slower than a @code{cons} loop.
- Do not bind the rest of a vec in a recursive match. @code{[a rest ...]} slices the vec at every step. @code{[a b _ ...]} ignores the rest and does not slice. To walk a vec, use @code{loop}, a fold or an index.
- @code{list-ref}, @code{length} and the last item of a list read the list from the start. 64 random reads in a list of 100,000 items take 3 ms. In a vec they take 240 ns.
}
@strong{Map or OrderedMap}
@read-table{
| Operation | Map | OrderedMap |
|--------------------------------------+---------------------+------------------------------------|
| lookup, int keys | 1 | 1.6–3 times slower |
| lookup, string keys | 1 | 2–6.5 times slower |
| update, insert, remove | 1 | 1.2–1.9 times slower, 2× the allocation |
| build | 1 | up to 2.9 times slower |
| read every entry, 10,000+ entries | 1 | 2–6 times faster |
| smallest key | reads every entry | 2 ns |
| 64 entries in a key range | reads every entry | about 400 ns at any size |
| every key, in order | collect and sort | about 4 times faster |
}
Use a @code{Map} to look up, update, insert and remove by key. It wins at all of those at every size. The difference is largest for string keys, because a map hashes the key once and an ordered map compares it at every step down the tree.
Use an @code{OrderedMap} when the order of the keys matters: the smallest or largest key, the entries between two keys, or the entries in key order. A map can only answer those by reading every entry. An ordered map is also faster to iterate in full, so a program that walks a large map more often than it looks things up in it may prefer one.
@subsubsection{Simd vectors}
A CPU can do one operation on several numbers at once: 4 doubles, 8 ints or 32 bytes in one 256-bit register with AVX2. Bjolang does not do this for you, because the .NET JIT does not turn a loop into vector code. Two modules do it explicitly.
Try @code{(std array)} first. @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-count}, @code{array-index-of}, @code{array-add!} and @code{array-scale!} are each one vector loop in the runtime, with the tail handled for you. On 4 million elements a sum of doubles took 0.8 ms instead of 2.6 ms for a loop, and a sum of ints 0.21 ms instead of 0.88 ms.
For a loop that the kernels do not cover, write it with @code{(std simd)}. A loop over an array has two parts: whole vectors, and the elements after the last whole vector, one at a time:
@codeblock[#:lang "bjolang"]|{
(import (std simd))
(: dot (-> (Array double) (Array double) double))
(defun (dot a b)
(def w (simd256-lanes a)) ; 4 doubles in a Float64x4
(def whole (simd256-whole a)) ; where the tail starts
(def acc
(loop (:for i (up-from 0 #:to whole #:by w))
(:acc v (folding (simd256-zero)
(simd-add v (simd-mul (simd256-load a i) (simd256-load b i)))))
=> v))
(+ (simd-sum acc)
(loop (:for i (up-from whole #:to (array-length a)))
(:acc s (folding 0.0 (+ s (* (array-ref a i) (array-ref b i)))))
=> s)))
}|
A comparison gives a mask instead of a bool: @code{(simd-lt v limit)} has every bit of a lane set where the lane is less. Use @code{simd-select} to choose lanes by a mask, @code{simd-count-true} to count them, and @code{simd-and} and @code{simd-or} to combine masks. A branch in a scalar loop is often a select in a vector loop.
Some things to keep in mind:
@read-list{
- Use more than one accumulator in a sum. An addition waits for the one before it. On 4 million doubles the loop above took 2.9 ms, the same with four @code{Float64x4} accumulators 1.4 ms, and a scalar loop 9.9 ms.
- Put a margin of zeros round a grid. A load at a neighbour's offset then needs no bounds test, and the zeros match nothing. Make each row longer than the grid by a whole vector, so that the last vector of a row does not read the next row.
- A vector loop pays when the work is dense and does not stop early. On Advent of Code 2025 day 4, a neighbour count over a 3000x3000 grid, the vectors were 9 times faster than the same loop one byte at a time. On day 9, a search that usually stops after a few comparisons, they were 1.6 times faster. On a small input, reading the input takes almost all of the time, and vectors change nothing. The @code{-avx} solutions in @code{Examples/aoc} show all three.
- Use 128 bits for code that must run everywhere. A 256-bit vector is emulated on a CPU without AVX2, and on ARM64, and is then much slower than scalar code. @code{simd256-accelerated?} says which you have.
- A Native AOT build targets a baseline x86-64 CPU, without AVX2, unless the manifest says @code{(aot-cpu "native")} or a level such as @code{(aot-cpu "x86-64-v3")}. The executable then does not start on a CPU without those instructions. A program that runs on .NET uses what the CPU has.
- Shifts, conversions between element types and shuffles are not there yet.
}
Reference in a new issue View git blame Copy permalink