From 15bfc181675a0118f9ede8e0d2ff920e542d95d6edf1c0d7f581c4ae6b0d9e22 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 18 Oct 2026 17:16:42 +0200 Subject: [PATCH 01/38] Native executables, std/array and std/simd MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Projects and packages" gets a section on Native AOT: (aot #t), where the executable goes, what it gives and costs, and (aot-cpu ...) for the CPU it is built for. The standard library chapter gets std/array and std/simd after std/maths, and "Performance" a section on vectors: the kernels first, a loop with its tail, masks, and when vectors pay, with the measurements from the Advent of Code solutions. modules/std/array.sz and modules/std/simd.sz are the reference pages, generated from the modules' docs. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-4-7) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 71 ++++++++++++++++++++++++++++++++++++++++ _03.Standard-library.sz | 5 +++ modules/std/array.sz | 3 ++ modules/std/simd.sz | 3 ++ 4 files changed, 82 insertions(+) create mode 100644 modules/std/array.sz create mode 100644 modules/std/simd.sz diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 5885198..d6d4bd9 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -375,6 +375,39 @@ You can pull in third-party .NET packages directly from NuGet by listing them in Once restored, types and methods from NuGet packages can be consumed through Bjolang's .NET interop mechanisms (described in the .NET section below). Version strings follow standard NuGet versioning semantics: @code{"9.0.3"} matches version 9.0.3 or higher, while bracketed syntax like @code{"[9.0.3]"} locks to that exact version. Bjolang delegates package restoration to the .NET SDK via the @code{dotnet} tool, storing the resolution graph in @code{packages.lock.json} (which should also be committed). Currently, compiled binaries load dependencies directly from the local NuGet cache, so executables expect to run in an environment with the restored packages available. +@subsubsection{Native executables} + +A program usually runs on .NET: @code{bjo run} starts the runtime, and the JIT compiles each method the first time it runs. With @code{(aot #t)} in the manifest, the program is also compiled ahead of time, with .NET's Native AOT, to one native executable: + +@codeblock[#:lang "bjolang"]|{ +(package + (name (cards)) + (version "0.1.0") + (aot #t)) +}| + +The executable is beside the source, with no extension: @code{src/main}, or @code{src/main.native.exe} on Windows. It holds the program, the standard library and the .NET runtime, so it runs on a machine without .NET installed. + +@read-list{ + - @strong{It starts in a few milliseconds.} A program on .NET spends about 70 ms starting the runtime, and then compiles its methods as it runs them. On 17 days of Advent of Code, the solutions together took 3.0 s on .NET and 1.2 s as native executables. + - @strong{@code{bjo run} starts the native executable, and @code{bjo publish} copies it} into the output directory. The ordinary @code{.exe} is still built, for the debugger and for @code{-d}. + - @strong{The build takes longer}, about 10 s more, and needs clang or gcc. The first build downloads .NET's ILCompiler package from NuGet. + - @strong{The runtime settings are built into the executable.} The GC settings, such as @code{(runtime (gc-server #t))}, take effect. The settings of tiered compilation do nothing, because there is no JIT. + - @strong{A long-running program can be slower.} The JIT optimizes the code with what it sees the program do (dynamic PGO), and a native executable can not. A program that calls through many traits in a hot loop gains the most from that. Measured on one such program: 0.73 s on .NET and 0.89 s native. + - @strong{Reflection that is resolved at run time may fail.} Native AOT keeps only the code that it can see the program use. +} + +By default the executable runs on any x86-64 CPU, so it uses only the instructions that all of them have: no AVX, and 128-bit vectors. @code{aot-cpu} names a CPU to build for instead: + +@codeblock[#:lang "bjolang"]|{ + (aot #t) + (aot-cpu "native") +}| + +@code{"native"} is the CPU of the machine that builds. A level, such as @code{"x86-64-v3"}, is every CPU that has its instructions: @code{x86-64-v3} has AVX2, and @code{x86-64-v4} has AVX-512. An executable built for a CPU stops at start on a CPU that does not have the instructions, so use @code{"native"} for a program that runs where it is built. This matters most to @code{(std array)} and @code{(std simd)}; see the section on vectors under performance. + +Only the project being built reads @code{aot} and @code{aot-cpu}, as with @code{runtime}: a dependency is not a program. A change to either rebuilds the program. The compiler itself takes them as @code{--aot} and @code{--aot-cpu}, for a file built without a project. + @subsubsection{Publishing a package} Publishing a Bjolang package requires three basic steps: @@ -2104,3 +2137,41 @@ Use a @code{Vec} for everything else. It knows its length and last item, it read Use a @code{Map} to look up, update, insert and remove by key. It wins at all of those at every size. The difference is largest for string keys, because a map hashes the key once and an ordered map compares it at every step down the tree. Use an @code{OrderedMap} when the order of the keys matters: the smallest or largest key, the entries between two keys, or the entries in key order. A map can only answer those by reading every entry. An ordered map is also faster to iterate in full, so a program that walks a large map more often than it looks things up in it may prefer one. + +@subsubsection{Vectors} +A CPU can do one operation on several numbers at once: 4 doubles, 8 ints or 32 bytes in one 256-bit register with AVX2. Bjolang does not do this for you, because the .NET JIT does not turn a loop into vector code. Two modules do it explicitly. + +Try @code{(std array)} first. @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-count}, @code{array-index-of}, @code{array-add!} and @code{array-scale!} are each one vector loop in the runtime, with the tail handled for you. On 4 million elements a sum of doubles took 0.8 ms instead of 2.6 ms for a loop, and a sum of ints 0.21 ms instead of 0.88 ms. + +For a loop that the kernels do not cover, write it with @code{(std simd)}. A loop over an array has two parts: whole vectors, and the elements after the last whole vector, one at a time: + +@codeblock[#:lang "bjolang"]|{ +(import (std simd)) + +(: dot (-> (Array double) (Array double) double)) +(defun (dot a b) + (def w (simd256-lanes a)) ; 4 doubles in a Float64x4 + (def whole (simd256-whole a)) ; where the tail starts + (def acc + (loop (:for i (up-from 0 #:to whole #:by w)) + (:acc v (folding (simd256-zero) + (simd-add v (simd-mul (simd256-load a i) (simd256-load b i))))) + => v)) + (+ (simd-sum acc) + (loop (:for i (up-from whole #:to (array-length a))) + (:acc s (folding 0.0 (+ s (* (array-ref a i) (array-ref b i))))) + => s))) +}| + +A comparison gives a mask instead of a bool: @code{(simd-lt v limit)} has every bit of a lane set where the lane is less. Use @code{simd-select} to choose lanes by a mask, @code{simd-count-true} to count them, and @code{simd-and} and @code{simd-or} to combine masks. A branch in a scalar loop is often a select in a vector loop. + +Some things to keep in mind: + +@read-list{ + - Use more than one accumulator in a sum. An addition waits for the one before it. On 4 million doubles the loop above took 2.9 ms, the same with four @code{Float64x4} accumulators 1.4 ms, and a scalar loop 9.9 ms. + - Put a margin of zeros round a grid. A load at a neighbour's offset then needs no bounds test, and the zeros match nothing. Make each row longer than the grid by a whole vector, so that the last vector of a row does not read the next row. + - A vector loop pays when the work is dense and does not stop early. On Advent of Code 2025 day 4, a neighbour count over a 3000x3000 grid, the vectors were 9 times faster than the same loop one byte at a time. On day 9, a search that usually stops after a few comparisons, they were 1.6 times faster. On a small input, reading the input takes almost all of the time, and vectors change nothing. The @code{-avx} solutions in @code{Examples/aoc} show all three. + - Use 128 bits for code that must run everywhere. A 256-bit vector is emulated on a CPU without AVX2, and on ARM64, and is then much slower than scalar code. @code{simd256-accelerated?} says which you have. + - A Native AOT build targets a baseline x86-64 CPU, without AVX2, unless the manifest says @code{(aot-cpu "native")} or a level such as @code{(aot-cpu "x86-64-v3")}. The executable then does not start on a CPU without those instructions. A program that runs on .NET uses what the CPU has. + - Shifts, conversions between element types and shuffles are not there yet. +} diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index eed722e..907937b 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -12,6 +12,11 @@ For more details, see @link["modules/std/prelude.html"]{@code{(std prelude)}}. Numbers beyond the operators. Integer division has Racket's names: @code{quotient} and @code{remainder} are what @code{/} and @code{%} do, and @code{modulo} takes the sign of the divisor, so @code{(modulo -1 10)} is 9. There are @code{gcd}, @code{lcm}, an integer @code{expt} and an exact @code{integer-sqrt} at every integer type, the @code{exact-} roundings from a double to an int, and the rest of @code{System.Math} over doubles. The prelude has @code{abs}, @code{min}, @code{max} and @code{clamp} from here without an import. For more details, see @link["modules/std/maths.html"]{@code{(std maths)}}. +@subsection{std/array and std/simd} + +Two modules for numbers in bulk, with the CPU's vector instructions. @code{(std array)} has whole-array operations: @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-add!}, @code{array-scale!}, @code{array-count}, @code{array-index-of} and @code{array-fill!}. Each is one loop in the runtime, which works on as many elements per instruction as a vector register holds when the element is a primitive number, and one at a time for a type such as @code{bigint}. At @code{double}, a sum adds lane by lane, so its last bits can differ from a left fold. @code{(std simd)} has the vectors themselves, for the loops that the kernels do not cover. @code{(Simd128 %a)} and @code{(Simd256 %a)} are .NET's @code{Vector128} and @code{Vector256}, with a name per element type: @code{Float64x4} is four doubles, @code{UInt8x32} is 32 bytes. The operations are the methods of one trait, @code{Simd}, at both widths: @code{simd-add}, @code{simd-mul}, the comparisons @code{simd-lt} and the others, which give masks, @code{simd-select}, @code{simd-count-true}, @code{simd-sum} and more. A vector is made with @code{simd256-load}, @code{simd128-splat} or @code{simd256-zero}, and @code{simd256-whole} says where the elements after the last whole vector start. A 128-bit vector is fast on every x86-64 and ARM64 CPU. A 256-bit vector needs AVX2, and @code{simd256-accelerated?} says whether the CPU has it. The section on performance in "Writing programs" says when vectors pay. +For more details, see @link["modules/std/array.html"]{@code{(std array)}} and @link["modules/std/simd.html"]{@code{(std simd)}}. + @subsection{std/fmt} Formatting in Bjolang is divided into two layers: interpolation and layout. diff --git a/modules/std/array.sz b/modules/std/array.sz new file mode 100644 index 0000000..5ea4468 --- /dev/null +++ b/modules/std/array.sz @@ -0,0 +1,3 @@ +(meta (title "(std array)")) + +@generate-docs[#:module "std array"] diff --git a/modules/std/simd.sz b/modules/std/simd.sz new file mode 100644 index 0000000..4f9ca61 --- /dev/null +++ b/modules/std/simd.sz @@ -0,0 +1,3 @@ +(meta (title "(std simd)")) + +@generate-docs[#:module "std simd"] From 21d0c9de06a0c9767fceacab8375c9910d35a43f253840fa52723c3d33f495ab Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 08:01:41 +0200 Subject: [PATCH 02/38] aot clarifications --- _02_.Writing-programs.sz | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index d6d4bd9..0fb0aea 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -1,4 +1,3 @@ - @section{Writing programs} While the first part focused on the core language constructs in single files, this part covers the wider ecosystem: breaking programs into modules, managing project dependencies, handling I/O, concurrency with bjoroutines, .NET interop, macros, and testing. @@ -393,8 +392,8 @@ The executable is beside the source, with no extension: @code{src/main}, or @cod - @strong{@code{bjo run} starts the native executable, and @code{bjo publish} copies it} into the output directory. The ordinary @code{.exe} is still built, for the debugger and for @code{-d}. - @strong{The build takes longer}, about 10 s more, and needs clang or gcc. The first build downloads .NET's ILCompiler package from NuGet. - @strong{The runtime settings are built into the executable.} The GC settings, such as @code{(runtime (gc-server #t))}, take effect. The settings of tiered compilation do nothing, because there is no JIT. - - @strong{A long-running program can be slower.} The JIT optimizes the code with what it sees the program do (dynamic PGO), and a native executable can not. A program that calls through many traits in a hot loop gains the most from that. Measured on one such program: 0.73 s on .NET and 0.89 s native. - - @strong{Reflection that is resolved at run time may fail.} Native AOT keeps only the code that it can see the program use. + - @strong{A long-running program can be slower.} The JIT optimizes the code with what it sees the program do (dynamic PGO), and a native executable can not. This matters for programs where the call graph stops the monomorphization or when trait calls cannot be determined at compile time. + - @strong{Reflection that is resolved at run time may fail.} Native AOT keeps only the code that it can see the program use. Bjolang does not use reflection by default, so this should be a small problem. } By default the executable runs on any x86-64 CPU, so it uses only the instructions that all of them have: no AVX, and 128-bit vectors. @code{aot-cpu} names a CPU to build for instead: @@ -2080,6 +2079,8 @@ Strings in bjolang are utf8, as such they do @em{not} support random access. Thi - Use slices when copying is the bottleneck: an operation ending with @code{/slice} will return a view into the string instead of copying. This will retain the whole string in memory, so use it wisely. } +When parsing strings with a known format, (std scan) will be a lot faster than (std rx) and the good old string-split. Often by an order of magnitude. The only cost is IL code size, since it builds the parser in the code, which will have other costs, such as no inlining. + @subsubsection{loops} Most of the time you should use the @code{loop} form. It will be the fastest, and is usually flexible enough to do what you need. The only case is for hot loops (ones that are run often) that are performance sensitive. For those, using the various @code{-fold}, @code{-for-each}, and @code{-iter} functions will be faster for the collections that are tree-based. Those collections are: @code{Vec}, @code{Map}, @code{HashSet}, @code{OrderedMap}, and @code{OrderedMap}. The overhead of loop vs. fold is often something like 3x, with fold being as fast or slightly faster thana @code{foreach} on a C# @code{List}. It does not, however, beat using indexed iteration over the @code{List} due to the @code{foreach} doing some safety checks to detect concurrent mutation. @@ -2138,7 +2139,7 @@ Use a @code{Map} to look up, update, insert and remove by key. It wins at all of Use an @code{OrderedMap} when the order of the keys matters: the smallest or largest key, the entries between two keys, or the entries in key order. A map can only answer those by reading every entry. An ordered map is also faster to iterate in full, so a program that walks a large map more often than it looks things up in it may prefer one. -@subsubsection{Vectors} +@subsubsection{Simd vectors} A CPU can do one operation on several numbers at once: 4 doubles, 8 ints or 32 bytes in one 256-bit register with AVX2. Bjolang does not do this for you, because the .NET JIT does not turn a loop into vector code. Two modules do it explicitly. Try @code{(std array)} first. @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-count}, @code{array-index-of}, @code{array-add!} and @code{array-scale!} are each one vector loop in the runtime, with the tail handled for you. On 4 million elements a sum of doubles took 0.8 ms instead of 2.6 ms for a loop, and a sum of ints 0.21 ms instead of 0.88 ms. From aaf92b38c18c928208593a3fef9a0dbba249a12e145d524385370851335bee89 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 10:48:12 +0200 Subject: [PATCH 03/38] Formatting text: a guide to std/fmt and its blocks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A section in "Writing programs" on (std fmt): what a block and a vec of blocks are, what goes in the brackets and how to convert the rest, blocks from data, padding, numbers, paragraphs, tables, settings, and what to write when it does not compile. The std/fmt entry in Part III gives the rule and points to it, and no longer names str*. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 393 +++++++++++++++++++++++++++++++++++++++ _03.Standard-library.sz | 4 +- 2 files changed, 394 insertions(+), 3 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 0fb0aea..b3b292e 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -599,6 +599,399 @@ Prefixing an identifier with a comma (e.g. @code{,who}) splices the variable's v All of these functions return a @code{Result} and execute synchronously, blocking the calling thread until completion. When running inside a bjoroutine, non-blocking asynchronous alternatives are available; refer to the @code{(std run)} module documentation for details. +@subsection{Formatting text} + +For one line of output, an interpolated string or @code{println*} is enough. @code{println*} is in @code{(std fmt)}, and takes strings, ints, doubles, chars and bools in one vec: + +@codeblock[#:lang "bjolang"]|{ +(import (std fmt)) + +(println #"${who} has ${points} points") ;; ada has 31 points +(println* [who " has " points " points"]) ;; ada has 31 points +}| + +When the shape of the output matters, such as columns that line up, padding, a paragraph wrapped to a width or a table, use the layout part of @code{(std fmt)}. This section is a guide to it. The module's page has the reference. + +@subsubsection{A block and a vec of blocks} + +Layout is made of two types, and nearly every type error in layout code is one of them where the other is wanted: + +@read-list{ + - A @code{Block} is one piece of output: a string, an int, a char, or a combinator such as @code{(padded/left 5 ["x"])}. + - A @code{(Vec Block)} is pieces of output in order, written as a vec literal: @code{["x = " 7 nl]}. +} + +There is one rule. Every combinator takes its content as a @code{(Vec Block)} and gives back one @code{Block}. @code{show}, @code{render} and @code{render->string}, which write the output, take a @code{(Vec Block)}. So the brackets go around content, and a combinator call is one element in them: + +@codeblock[#:lang "bjolang"]|{ +(show [(padded/left 5 ["x"])]) ;; " x" +(show [(padded/left 5 ["x"]) (padded/left 5 ["y"]) nl]) ;; " x y", then a newline +(show ["Total:" (padded/left 8 [42]) nl]) ;; "Total: 42" +}| + +Read @code{(padded/left 5 ["x"])} as "pad this content to 5 columns". The content is a vec even when it is one thing, as the body of a @code{begin} is a body even when it is one form. These are refused: + +@codeblock[#:lang "bjolang"]|{ +(show (padded/left 5 "x")) ;; "x" is not content, and show wants a vec +(show (padded/left 5 ["x"])) ;; show wants a vec, and this is one block +(show [["x"]]) ;; a vec in a vec: a vec is not a block +}| + +More brackets do not help. A vec of vecs is right in one place only: a table, which is a vec of rows or of columns, each of them a @code{(Vec Block)}. + +Nothing is written until @code{show} runs the blocks. A block is a value: you can bind it, pass it to a function and return it. + +@subsubsection{What goes in the brackets} + +A vec literal is read element by element when it stands where a @code{(Vec Block)} is wanted: as an argument of a function that takes one, as the return value of a function declared to return one, or as the value of a binding whose type is written. Each element is one of these: + +@read-table{ +| Element | Is read as | +|--------------------------------------------------------------------------------+---------------------------------------------| +| A string, an int or a char: a literal, a variable or a call | @code{(BStr s)}, @code{(BInt n)}, @code{(BChar c)} | +| A @code{Block}: @code{nl}, @code{nothing}, a combinator call, a variable of type @code{Block} | itself | +| An @code{if}, @code{cond}, @code{match} or @code{let} whose results are any of these | the result it gives | +} + +So variables and calls go in as they are: + +@codeblock[#:lang "bjolang"]|{ +(def name "Ada") +(: n int) +(def n 3) +(show ["Hello, " name "! You have " n " new messages" #\. nl]) +;; Hello, Ada! You have 3 new messages. + +(show [(string-upcase name) " " (vec-length [1 2 3]) nl]) ;; ADA 3 +(show ["Paid: " (if paid? "yes" "no") nl]) ;; Paid: yes +(show ["Title: " (match title ((Some t) t) (None "-")) nl]) ;; Title: Dr +}| + +Anything else is converted first: + +@read-table{ +| You have | Write | +|-----------------------------------+---------------------------------------------------------------| +| A double @code{x} | @code{(numeric x)}, or @code{(numeric x #:precision 2)} | +| A bool @code{b} | @code{(if b "yes" "no")} | +| A long @code{n} | @code{(long->string n)}, or @code{(bytes n)} for a size | +| A @code{(Vec Block)} @code{bs} | @code{(each bs)} | +| A vec literal | @code{(each [...])}, or its elements in the outer vec | +| A @code{(Vec string)} @code{xs} | @code{(each (vec-map BStr xs))} | +| A @code{(List %a)} @code{xs} | @code{(joined f xs [", "])}, where @code{f} makes the block of one element | +} + +@strong{A number bound by @code{def}.} The @code{(: n int)} above is needed. @code{(def n 3)} alone does not say which kind of number @code{n} is, as it could still become a long or a double, so @code{[n]} cannot read it as a block, and the compiler says so at @code{n}. The same holds for @code{(def xs [1 2 3])} and @code{(def total (+ 1 2))}. A number that comes from a typed parameter, a record field or a function such as @code{vec-length} has its type already, and a number written in the vec itself, as in @code{["n = " 3]}, is an int. + +@strong{A vec bound by @code{def}.} @code{(def parts ["a" "b"])} is a @code{(Vec string)}, which is not a @code{(Vec Block)}: write its type, and it is read as blocks. + +@codeblock[#:lang "bjolang"]|{ +(: parts (Vec Block)) +(def parts ["a" 1 #\b]) +(show parts) ;; a1b +(show ["<" (each parts) ">" nl]) ;; +}| + +@subsubsection{From one type to the other} + +@read-table{ +| You have | You want a @code{Block} | You want a @code{(Vec Block)} | +|-------------------------+-------------------------------------+-------------------------------| +| a string @code{s} | @code{(BStr s)} | @code{[s]} | +| an int @code{n} | @code{(BInt n)} | @code{[n]} | +| a block @code{b} | @code{b} | @code{[b]} | +| blocks @code{bs} | @code{(each bs)} | @code{bs} | +| pieces | @code{(each [name ": " n])} | @code{[name ": " n]} | +} + +Outside a vec literal nothing is converted: a function declared to return a @code{Block} returns @code{(BStr s)} or @code{(each [s])}, not @code{s}. @code{BStr}, @code{BInt} and @code{BChar} are the cases of @code{Block}, and like any constructor they are functions, so @code{(vec-map BStr names)} is a @code{(Vec Block)}. + +@subsubsection{Blocks from your data} + +Most layout code is a function from a value of your own to a block. Give it a signature: the signature is what tells the compiler that the literal inside is blocks. + +@codeblock[#:lang "bjolang"]|{ +(type (: Item (Record (: name string) (: price double) (: stock int)))) + +(: stock (Vec Item)) +(def stock + [(Item (name "Green tea") (price 3.5) (stock 12)) + (Item (name "Gaiwan") (price 1240.0) (stock 2)) + (Item (name "Kettle") (price 89.9) (stock 0))]) + +(: item-line (-> Item Block)) +(defun (item-line i) + (each [(padded/right 10 [(record-ref i name)]) + (padded/left 9 [(numeric (record-ref i price) #:precision 2 #:comma 3)]) + (if (= (record-ref i stock) 0) " sold out" nothing) + nl])) + +(show [(each (vec-map item-line stock))]) +}| + +@codeblock[#:lang "text"]|{ +Green tea 3.50 +Gaiwan 1,240.00 +Kettle 89.90 sold out +}| + +@code{item-line} returns one block, so its pieces are under @code{each}. @code{vec-map} gives a vec of blocks, and @code{each} makes that one block too. A function can return content instead, a @code{(Vec Block)}, and the caller decides where it goes: + +@codeblock[#:lang "bjolang"]|{ +(: labelled (-> string int (Vec Block))) +(defun (labelled label n) + [label ": " n nl]) + +(show (labelled "cups" 3)) ;; cups: 3 +(show [">> " (each (labelled "pots" 1))]) ;; >> pots: 1 +}| + +A lambda has no signature, so a vec literal returned from one is not read as blocks: in @code{(vec-map (fun (i) [(record-ref i name) (record-ref i stock)]) stock)} the vec has a string and an int in it, and is refused. Put the literal under @code{each}, which says that it is content, or write a function with a signature: + +@codeblock[#:lang "bjolang"]|{ +(show [(each (vec-map (fun (i) (each [(record-ref i name) ": " (record-ref i stock) nl])) + stock))]) +;; Green tea: 12 +;; Gaiwan: 2 +;; Kettle: 0 +}| + +@code{nothing} is the block that writes nothing, for the branch that has nothing to say. In a vec literal, what an @code{if}, a @code{match} or a @code{cond} gives is read as any element is, so an arm can be a string. In a function declared to return a @code{Block}, nothing is converted, so there the arms are blocks: + +@codeblock[#:lang "bjolang"]|{ +(: greeting (-> string (Option string) Block)) +(defun (greeting name title) + (each ["Hello, " + (match title + ((Some t) (each [t " "])) + (None nothing)) + name "!"])) + +(: medal (-> int Block)) +(defun (medal place) + (cond ((= place 1) (BStr "gold")) + ((= place 2) (BStr "silver")) + (else nothing))) + +(show [(greeting "Ada" (Some "Dr")) nl (greeting "Bob" None) nl]) +;; Hello, Dr Ada! +;; Hello, Bob! +}| + +A list is written with @code{joined}, which takes a function that makes each element's block and a separator, which is content too: + +@codeblock[#:lang "bjolang"]|{ +(show [(joined BStr (list "tea" "milk" "honey") [", "]) nl]) ;; tea, milk, honey +(show [(joined/last BStr (fun (s) (each ["and " s])) + (list "tea" "milk" "honey") [", "]) nl]) ;; tea, milk, and honey +(show [(joined (fun (s) (each ["* " s])) (list "tea" "milk") [nl]) nl]) +;; * tea +;; * milk +}| + +@subsubsection{Padding, cutting and leaders} + +@code{padded/left}, @code{padded/right} and @code{padded/both} pad content to a width: aligned right, left, or centred. Padding never cuts. @code{trimmed/left}, @code{trimmed/right} and @code{trimmed/both} cut content to a width, and keep its end, its start or its middle. @code{fitted/left}, @code{fitted/right} and @code{fitted/both} do both, so the content is exactly the width. + +@codeblock[#:lang "bjolang"]|{ +(show [(padded/right 8 [who]) (padded/left 4 [points]) nl]) ;; ada 31 +(show [(with-pad #\0 [(padded/left 4 [42])]) nl]) ;; 0042 +(show [(with-pad #\= [(padded/both 20 [" Receipt "])]) nl]) ;; ===== Receipt ====== +(show [(trimmed/left 6 ["/home/ada/scores.txt"]) nl]) ;; es.txt +(show [(with-fmt current-ellipsis "โ€ฆ" + [(trimmed/left 12 ["/home/ada/scores.txt"])]) nl]) ;; โ€ฆ/scores.txt +(show ["[" (fitted/both 6 ["ab"]) "]" nl]) ;; [ ab ] +}| + +@code{space-to} pads to a column, whatever was written before it, and @code{with-pad} changes what the padding is made of. Together they make a leader: + +@codeblock[#:lang "bjolang"]|{ +(show [(with-pad #\. ["Tea" (space-to 12) "3.50"]) nl + (with-pad #\. ["Gaiwan" (space-to 12) "1240.00"]) nl]) +;; Tea.........3.50 +;; Gaiwan......1240.00 +}| + +@code{nl} is a newline, and @code{fl} a newline unless the cursor is already at the start of a line. + +@subsubsection{Numbers} + +An int is a block. A double is not, as there is no one way to write one: @code{numeric} says how, with a precision, digits in groups, and the separators. @code{bytes} and @code{percent} are blocks for two common kinds of number. + +@codeblock[#:lang "bjolang"]|{ +(show [(numeric 1234567.891 #:precision 2 #:comma 3) " " + (numeric 0.5 #:precision 3) " " + (numeric 42.0 #:signed #t) " " + (percent 0.125) " " + (bytes 1500L #:si #t) nl]) +;; 1,234,567.89 0.500 +42 12.5% 1.5 kB +}| + +The digits are always the invariant culture's, so the machine's locale never shows. The separators are settings, described below. + +@subsubsection{Paragraphs} + +@code{wrapped} fills content to @code{current-width}, and @code{justified} makes it flush against both margins. Every run of whitespace in the content is a break between words. + +@codeblock[#:lang "bjolang"]|{ +(show [(with-fmt current-width 20 + [(justified ["Bjolang lays text out with blocks, and blocks nest."])]) + nl]) +;; Bjolang lays text +;; out with blocks, and +;; blocks nest. +}| + +The first line starts where the cursor is, so after a label it has less room. The lines after it start at column 0. + +@subsubsection{Tables} + +A table is the one place where a vec of vecs is right. @code{tabular} takes a vec of columns, and @code{tabular/rows} a vec of rows. Each column or row is a @code{(Vec Block)}, so the inner vecs are read as blocks as any other content is: + +@codeblock[#:lang "bjolang"]|{ +(show [(tabular [["tea" nl "gaiwan" nl "kettle"] + [3 nl 1240 nl 90]] + #:sep " " #:align [AlignLeft AlignRight])]) + +(show [(tabular/rows [["tea" 3] ["gaiwan" 1240] ["kettle" 90]] + #:sep " " #:align [AlignLeft AlignRight])]) +}| + +Both write the same table: + +@codeblock[#:lang "text"]|{ +tea 3 +gaiwan 1240 +kettle 90 +}| + +In @code{tabular} a column's cells are separated by @code{nl}. In @code{tabular/rows} each element of a row is one cell, so a cell of several pieces is under @code{each}, and a newline in a cell gives the row another line: + +@codeblock[#:lang "bjolang"]|{ +(show [(tabular/rows [["tea" (each [3 " cups"])] + ["kettle" (each ["boils in" nl "4 minutes"])]] + #:style StyleLight)]) +}| + +@codeblock[#:lang "text"]|{ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ tea โ”‚ 3 cups โ”‚ +โ”‚ kettle โ”‚ boils in โ”‚ +โ”‚ โ”‚ 4 minutes โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +}| + +Rows from data come from a function with a signature, for the reason given above. Here is the high-score ledger from "Files and I/O" as a table: + +@codeblock[#:lang "bjolang"]|{ +(: score-row (-> (Tuple string int) (Vec Block))) +(defun (score-row s) + (def (Tuple name points) s) + [name points]) + +(show [(tabular/rows (vec-map score-row (read-scores)) + #:titles ["Player" "Score"] + #:align [AlignLeft AlignRight] + #:style StyleRounded)]) +}| + +@codeblock[#:lang "text"]|{ +โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ +โ”‚ Player โ”‚ Score โ”‚ +โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค +โ”‚ ada โ”‚ 31 โ”‚ +โ”‚ bo โ”‚ 27 โ”‚ +โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ +}| + +@code{#:style} takes a @code{TableStyle}: @code{StyleNone}, the default, @code{StyleSimple}, @code{StyleAscii}, @code{StyleLight}, @code{StyleHeavy}, @code{StyleDouble}, @code{StyleRounded}, @code{StyleMarkdown}, or @code{StyleCustom} with pieces of your own. @code{#:align} takes an @code{Align} for each column, from the first: @code{AlignLeft}, @code{AlignRight}, @code{AlignCentre}, or @code{(AlignOn #\.)}, which puts every decimal point of the column in one place. @code{#:widths} and @code{#:overflow} make a table fit @code{current-width}, by cutting or wrapping its cells. + +A table whose rows are not all known yet, such as results that come one at a time, is laid out a row at a time with @code{table-layout} and @code{table-row}. The widths are fixed beforehand, so each row is written when it is known: + +@codeblock[#:lang "bjolang"]|{ +(def layout (table-layout [8 6] #:sep " " #:align [AlignLeft AlignRight])) +(show [(table-row layout ["Player" "Score"])]) +(loop (:for (Tuple name points) (read-scores)) + (:do (show [(table-row layout [name points])]))) +}| + +@codeblock[#:lang "text"]|{ +Player Score +ada 31 +bo 27 +}| + +A table can also be a quoted literal, which the compiler checks where it is written. A column can say what kind of number it holds: + +@codeblock[#:lang "bjolang"]|{ +(show-table '(table (style light) header + (column "Player") + (column "Score" (align right)) + (column "Share" (percent (precision 0))) + (rows (row "ada" 31 0.46) (row "bo" 27 0.40) (row "cy" 9 0.13)))) +}| + +@codeblock[#:lang "text"]|{ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Player โ”‚ Score โ”‚ Share โ”‚ +โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค +โ”‚ ada โ”‚ 31 โ”‚ 46% โ”‚ +โ”‚ bo โ”‚ 27 โ”‚ 40% โ”‚ +โ”‚ cy โ”‚ 9 โ”‚ 13% โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +}| + +The rows can be spliced, @code{(rows ,scores)}, from a @code{(Vec (Vec TableCell))}. See @code{Table} on the module's page. + +@subsubsection{Settings} + +The pad character, the right margin, the ellipsis and the number formats are parameters. @code{with-fmt} binds one around some blocks while they run, and @code{with-pad} is @code{with-fmt} for @code{current-pad}. Settings nest: + +@codeblock[#:lang "bjolang"]|{ +(show [(with-fmt current-comma 3 + [(with-fmt current-comma-sep #\space + [(with-fmt current-decimal-sep #\, + [(numeric 1234567.5)])])]) + nl]) +;; 1 234 567,5 +}| + +The settings are @code{current-pad}, @code{current-width}, @code{current-ellipsis}, @code{current-string-width}, @code{current-precision}, @code{current-comma}, @code{current-comma-sep}, @code{current-decimal-sep} and @code{current-signed}. For text with wide characters or colours in it, which a terminal shows in more or fewer columns than it has characters, set @code{current-string-width} to @code{terminal-width}. + +@subsubsection{When it does not compile} + +The messages for the common mistakes, and what to write instead: + +@read-table{ +| The compiler says | Write | +|----------------------------------------------------------------------+-----------------------------------------------------| +| expected @code{(Vec fmt/Block)}, found @code{string} | @code{["x"]} for @code{"x"} | +| expected @code{(Vec fmt/Block)}, found @code{fmt/Block} | @code{[b]} for @code{b} | +| expected @code{fmt/Block}, found @code{(Vec ...)} | @code{(each bs)} for @code{bs} | +| They differ inside the Vec: @code{fmt/Block} against @code{string} | @code{(: bs (Vec Block))} above @code{(def bs ...)} | +| Block: expected a string, an int, a char ...; found a vec | @code{(each [...])}, or no inner brackets | +| Block: ... found the number 1.5 | @code{(numeric 1.5)} | +| Block: ... found a boolean | @code{(if b "yes" "no")} | +| Block: 'n' is a number, and nothing says yet whether it is an int ... | @code{(: n int)} above @code{(def n 3)} | +| what @code{f} returns has the wrong type: expected @code{fmt/Block}, found @code{string} | @code{(BStr s)} or @code{(each [s])} | +} + +When none of these fits, find the place where a @code{Block} and a @code{(Vec Block)} meet, and check it against the rule: content in brackets, one block out. + +@subsubsection{A block of your own} + +@code{captured} renders content to a string and gives it to a function, which returns the block to write in its place. It is how a block that needs the whole of its content is made, and how padding and trimming are written: + +@codeblock[#:lang "bjolang"]|{ +(: boxed (-> (Vec Block) Block)) +(defun (boxed bs) + (captured (fun (s) (each ["[" s "]"])) bs)) + +(show [(boxed ["x = " 7]) nl]) ;; [x = 7] +}| + +A block that writes as it goes is a @code{BRun}: a function given the sink to write to and the column the cursor is at, which returns the column it ends at. @code{run-blocks} runs content inside one. The module's page describes both. + @subsection{Concurrency} Bjolang's concurrency model rests on three foundations: lightweight fibers called *bjoroutines*, communication channels, and structured concurrency scopes ensuring background work is cleanly contained. In addition, Bjolang adopts the Concurrent ML (CML) paradigm, where communication operationsโ€”such as sending or receiving on a channelโ€”are first-class values that can be composed and combined before being synchronized. diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 907937b..811a032 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -19,9 +19,7 @@ For more details, see @link["modules/std/array.html"]{@code{(std array)}} and @l @subsection{std/fmt} -Formatting in Bjolang is divided into two layers: interpolation and layout. -Interpolation provides a simple way to combine mixed scalar types into strings or print them directly using @code{println*} and @code{str*}. -The layout layer provides a composable block algebra for more complex text formatting, such as padding, aligning, wrapping paragraphs, and laying out tables. +Formatting in two layers: interpolation and layout. @code{println*} prints strings, ints, doubles, chars and bools written in one vec. The layout layer is made of blocks: a @code{Block} is one piece of output, and every combinator takes its content as a @code{(Vec Block)}, a vec literal such as @code{["x = " 7 nl]}, and gives back one @code{Block}. With them you pad, cut and align text, wrap and justify paragraphs, format numbers, and lay out tables with borders, a header, and columns that fit the width. The section on formatting text in "Writing programs" is a guide, with what goes in the brackets and what to write when it does not compile. For more details, see @link["modules/std/fmt.html"]{@code{(std fmt)}}. @subsection{std/rx} From 21fa7af6dcd6f3b348b7df00406738e1099b5c1ce65486ec82de05a5ea66d47e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 11:24:16 +0200 Subject: [PATCH 04/38] Formatting text: tables drawn with StyleAscii MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guide's framed tables use +, - and |, which every font has, so they line up in a browser whatever font it draws code in. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 44 ++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index b3b292e..cb3f203 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -870,15 +870,15 @@ In @code{tabular} a column's cells are separated by @code{nl}. In @code{tabular/ @codeblock[#:lang "bjolang"]|{ (show [(tabular/rows [["tea" (each [3 " cups"])] ["kettle" (each ["boils in" nl "4 minutes"])]] - #:style StyleLight)]) + #:style StyleAscii)]) }| @codeblock[#:lang "text"]|{ -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ tea โ”‚ 3 cups โ”‚ -โ”‚ kettle โ”‚ boils in โ”‚ -โ”‚ โ”‚ 4 minutes โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ ++--------+-----------+ +| tea | 3 cups | +| kettle | boils in | +| | 4 minutes | ++--------+-----------+ }| Rows from data come from a function with a signature, for the reason given above. Here is the high-score ledger from "Files and I/O" as a table: @@ -892,19 +892,19 @@ Rows from data come from a function with a signature, for the reason given above (show [(tabular/rows (vec-map score-row (read-scores)) #:titles ["Player" "Score"] #:align [AlignLeft AlignRight] - #:style StyleRounded)]) + #:style StyleAscii)]) }| @codeblock[#:lang "text"]|{ -โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ -โ”‚ Player โ”‚ Score โ”‚ -โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค -โ”‚ ada โ”‚ 31 โ”‚ -โ”‚ bo โ”‚ 27 โ”‚ -โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ ++--------+-------+ +| Player | Score | ++--------+-------+ +| ada | 31 | +| bo | 27 | ++--------+-------+ }| -@code{#:style} takes a @code{TableStyle}: @code{StyleNone}, the default, @code{StyleSimple}, @code{StyleAscii}, @code{StyleLight}, @code{StyleHeavy}, @code{StyleDouble}, @code{StyleRounded}, @code{StyleMarkdown}, or @code{StyleCustom} with pieces of your own. @code{#:align} takes an @code{Align} for each column, from the first: @code{AlignLeft}, @code{AlignRight}, @code{AlignCentre}, or @code{(AlignOn #\.)}, which puts every decimal point of the column in one place. @code{#:widths} and @code{#:overflow} make a table fit @code{current-width}, by cutting or wrapping its cells. +The tables here are drawn with @code{StyleAscii}, whose characters every font has. @code{#:style} takes a @code{TableStyle}: @code{StyleNone}, the default, @code{StyleSimple}, @code{StyleAscii}, @code{StyleLight}, @code{StyleHeavy}, @code{StyleDouble}, @code{StyleRounded}, @code{StyleMarkdown}, or @code{StyleCustom} with pieces of your own. @code{#:align} takes an @code{Align} for each column, from the first: @code{AlignLeft}, @code{AlignRight}, @code{AlignCentre}, or @code{(AlignOn #\.)}, which puts every decimal point of the column in one place. @code{#:widths} and @code{#:overflow} make a table fit @code{current-width}, by cutting or wrapping its cells. A table whose rows are not all known yet, such as results that come one at a time, is laid out a row at a time with @code{table-layout} and @code{table-row}. The widths are fixed beforehand, so each row is written when it is known: @@ -924,7 +924,7 @@ bo 27 A table can also be a quoted literal, which the compiler checks where it is written. A column can say what kind of number it holds: @codeblock[#:lang "bjolang"]|{ -(show-table '(table (style light) header +(show-table '(table (style ascii) header (column "Player") (column "Score" (align right)) (column "Share" (percent (precision 0))) @@ -932,13 +932,13 @@ A table can also be a quoted literal, which the compiler checks where it is writ }| @codeblock[#:lang "text"]|{ -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ Player โ”‚ Score โ”‚ Share โ”‚ -โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค -โ”‚ ada โ”‚ 31 โ”‚ 46% โ”‚ -โ”‚ bo โ”‚ 27 โ”‚ 40% โ”‚ -โ”‚ cy โ”‚ 9 โ”‚ 13% โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ ++--------+-------+-------+ +| Player | Score | Share | ++--------+-------+-------+ +| ada | 31 | 46% | +| bo | 27 | 40% | +| cy | 9 | 13% | ++--------+-------+-------+ }| The rows can be spliced, @code{(rows ,scores)}, from a @code{(Vec (Vec TableCell))}. See @code{Table} on the module's page. From 5bc42069cd3e3d41e66cff88283944fd63fb0d1229e34a9f028e17e5ca5d43d9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 11:56:52 +0200 Subject: [PATCH 05/38] Formatting text: a string is a block, and so is a vec of blocks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guide is written for (std fmt) as it is now: one thing needs no brackets, several things are a vec, and each is needed only for a lambda given to a function such as vec-map. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 223 +++++++++++++++------------------------ 1 file changed, 83 insertions(+), 140 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index cb3f203..292041f 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -612,61 +612,38 @@ For one line of output, an interpolated string or @code{println*} is enough. @co When the shape of the output matters, such as columns that line up, padding, a paragraph wrapped to a width or a table, use the layout part of @code{(std fmt)}. This section is a guide to it. The module's page has the reference. -@subsubsection{A block and a vec of blocks} +@subsubsection{Blocks} -Layout is made of two types, and nearly every type error in layout code is one of them where the other is wanted: +Layout is made of blocks. A @code{Block} is a piece of output, and every function that lays something out takes its content as a block and gives back a block. These are blocks: @read-list{ - - A @code{Block} is one piece of output: a string, an int, a char, or a combinator such as @code{(padded/left 5 ["x"])}. - - A @code{(Vec Block)} is pieces of output in order, written as a vec literal: @code{["x = " 7 nl]}. + - A string, an int or a char. Written out, or the value of a variable or a call: @code{"tea"}, @code{name}, @code{(string-upcase name)}. + - A vec of blocks: the blocks one after another. @code{["x = " n nl]} is a string, an int and a newline. + - What a function in @code{(std fmt)} gives back: @code{(padded/left 5 "x")}, @code{nl}, @code{nothing}. + - An @code{if}, @code{cond}, @code{match} or @code{let} whose results are blocks. } -There is one rule. Every combinator takes its content as a @code{(Vec Block)} and gives back one @code{Block}. @code{show}, @code{render} and @code{render->string}, which write the output, take a @code{(Vec Block)}. So the brackets go around content, and a combinator call is one element in them: - -@codeblock[#:lang "bjolang"]|{ -(show [(padded/left 5 ["x"])]) ;; " x" -(show [(padded/left 5 ["x"]) (padded/left 5 ["y"]) nl]) ;; " x y", then a newline -(show ["Total:" (padded/left 8 [42]) nl]) ;; "Total: 42" -}| - -Read @code{(padded/left 5 ["x"])} as "pad this content to 5 columns". The content is a vec even when it is one thing, as the body of a @code{begin} is a body even when it is one form. These are refused: - -@codeblock[#:lang "bjolang"]|{ -(show (padded/left 5 "x")) ;; "x" is not content, and show wants a vec -(show (padded/left 5 ["x"])) ;; show wants a vec, and this is one block -(show [["x"]]) ;; a vec in a vec: a vec is not a block -}| - -More brackets do not help. A vec of vecs is right in one place only: a table, which is a vec of rows or of columns, each of them a @code{(Vec Block)}. - -Nothing is written until @code{show} runs the blocks. A block is a value: you can bind it, pass it to a function and return it. - -@subsubsection{What goes in the brackets} - -A vec literal is read element by element when it stands where a @code{(Vec Block)} is wanted: as an argument of a function that takes one, as the return value of a function declared to return one, or as the value of a binding whose type is written. Each element is one of these: - -@read-table{ -| Element | Is read as | -|--------------------------------------------------------------------------------+---------------------------------------------| -| A string, an int or a char: a literal, a variable or a call | @code{(BStr s)}, @code{(BInt n)}, @code{(BChar c)} | -| A @code{Block}: @code{nl}, @code{nothing}, a combinator call, a variable of type @code{Block} | itself | -| An @code{if}, @code{cond}, @code{match} or @code{let} whose results are any of these | the result it gives | -} - -So variables and calls go in as they are: +So one thing needs no brackets, and several things are a vec: @codeblock[#:lang "bjolang"]|{ (def name "Ada") (: n int) (def n 3) + +(show "Hello\n") ;; Hello +(show (padded/left 6 name)) ;; " Ada" +(show (padded/left 8 ["x = " n])) ;; " x = 3" +(show [(padded/left 5 "x") (padded/left 5 "y") nl]) ;; " x y", then a newline (show ["Hello, " name "! You have " n " new messages" #\. nl]) ;; Hello, Ada! You have 3 new messages. - -(show [(string-upcase name) " " (vec-length [1 2 3]) nl]) ;; ADA 3 -(show ["Paid: " (if paid? "yes" "no") nl]) ;; Paid: yes -(show ["Title: " (match title ((Some t) t) (None "-")) nl]) ;; Title: Dr }| +Read @code{(padded/left 6 name)} as "pad @code{name} to 6 columns", and @code{(padded/left 8 ["x = " n])} as "pad these pieces, together, to 8 columns". A vec in a vec is a block in a block, so extra brackets change nothing. + +Nothing is written until @code{show} runs the block. A block is a value: you can bind it, pass it to a function and return it. + +@subsubsection{What is not a block} + Anything else is converted first: @read-table{ @@ -675,40 +652,19 @@ Anything else is converted first: | A double @code{x} | @code{(numeric x)}, or @code{(numeric x #:precision 2)} | | A bool @code{b} | @code{(if b "yes" "no")} | | A long @code{n} | @code{(long->string n)}, or @code{(bytes n)} for a size | -| A @code{(Vec Block)} @code{bs} | @code{(each bs)} | -| A vec literal | @code{(each [...])}, or its elements in the outer vec | -| A @code{(Vec string)} @code{xs} | @code{(each (vec-map BStr xs))} | -| A @code{(List %a)} @code{xs} | @code{(joined f xs [", "])}, where @code{f} makes the block of one element | +| A @code{(Vec string)} @code{xs} | @code{(vec-map BStr xs)} | +| A @code{(List %a)} @code{xs} | @code{(joined f xs ", ")}, where @code{f} makes the block of one element | } -@strong{A number bound by @code{def}.} The @code{(: n int)} above is needed. @code{(def n 3)} alone does not say which kind of number @code{n} is, as it could still become a long or a double, so @code{[n]} cannot read it as a block, and the compiler says so at @code{n}. The same holds for @code{(def xs [1 2 3])} and @code{(def total (+ 1 2))}. A number that comes from a typed parameter, a record field or a function such as @code{vec-length} has its type already, and a number written in the vec itself, as in @code{["n = " 3]}, is an int. +@code{BStr}, @code{BInt}, @code{BChar} and @code{BMany} are the cases of @code{Block}. You rarely write them, but like any constructor they are functions, so @code{(vec-map BStr names)} is a vec of blocks. -@strong{A vec bound by @code{def}.} @code{(def parts ["a" "b"])} is a @code{(Vec string)}, which is not a @code{(Vec Block)}: write its type, and it is read as blocks. +@strong{A number bound by @code{def}.} The @code{(: n int)} above is needed. @code{(def n 3)} alone does not say which kind of number @code{n} is, as it could still become a long or a double, so it cannot be read as a block, and the compiler says so at @code{n}. The same holds for @code{(def xs [1 2 3])} and @code{(def total (+ 1 2))}. A number that comes from a typed parameter, a record field or a function such as @code{vec-length} has its type already, and a number written in the vec itself, as in @code{["n = " 3]}, is an int. -@codeblock[#:lang "bjolang"]|{ -(: parts (Vec Block)) -(def parts ["a" 1 #\b]) -(show parts) ;; a1b -(show ["<" (each parts) ">" nl]) ;; -}| - -@subsubsection{From one type to the other} - -@read-table{ -| You have | You want a @code{Block} | You want a @code{(Vec Block)} | -|-------------------------+-------------------------------------+-------------------------------| -| a string @code{s} | @code{(BStr s)} | @code{[s]} | -| an int @code{n} | @code{(BInt n)} | @code{[n]} | -| a block @code{b} | @code{b} | @code{[b]} | -| blocks @code{bs} | @code{(each bs)} | @code{bs} | -| pieces | @code{(each [name ": " n])} | @code{[name ": " n]} | -} - -Outside a vec literal nothing is converted: a function declared to return a @code{Block} returns @code{(BStr s)} or @code{(each [s])}, not @code{s}. @code{BStr}, @code{BInt} and @code{BChar} are the cases of @code{Block}, and like any constructor they are functions, so @code{(vec-map BStr names)} is a @code{(Vec Block)}. +@strong{Where a block is wanted.} A string or a vec is read as a block where the compiler knows that a block is wanted: as an argument, as what a function declared to return a @code{Block} returns, and as the value of a binding whose type is written, such as @code{(: parts Block)}. A @code{def} without a type is what its value is: @code{(def parts ["a" "b"])} is a @code{(Vec string)}, and @code{(show parts)} is refused. Write @code{(: parts Block)} above it. @subsubsection{Blocks from your data} -Most layout code is a function from a value of your own to a block. Give it a signature: the signature is what tells the compiler that the literal inside is blocks. +Most layout code is a function from a value of your own to a block. Give it a signature: the signature is what tells the compiler that what it returns is a block. @codeblock[#:lang "bjolang"]|{ (type (: Item (Record (: name string) (: price double) (: stock int)))) @@ -721,12 +677,12 @@ Most layout code is a function from a value of your own to a block. Give it a si (: item-line (-> Item Block)) (defun (item-line i) - (each [(padded/right 10 [(record-ref i name)]) - (padded/left 9 [(numeric (record-ref i price) #:precision 2 #:comma 3)]) - (if (= (record-ref i stock) 0) " sold out" nothing) - nl])) + [(padded/right 10 (record-ref i name)) + (padded/left 9 (numeric (record-ref i price) #:precision 2 #:comma 3)) + (if (= (record-ref i stock) 0) " sold out" nothing) + nl]) -(show [(each (vec-map item-line stock))]) +(show (vec-map item-line stock)) }| @codeblock[#:lang "text"]|{ @@ -735,56 +691,48 @@ Gaiwan 1,240.00 Kettle 89.90 sold out }| -@code{item-line} returns one block, so its pieces are under @code{each}. @code{vec-map} gives a vec of blocks, and @code{each} makes that one block too. A function can return content instead, a @code{(Vec Block)}, and the caller decides where it goes: +@code{vec-map} gives a vec of blocks, which is a block. A lambda given to @code{vec-map} is different from a function with a signature: @code{vec-map} does not say what its function returns, so a vec of mixed pieces that the lambda returns, such as a string and an int, is not read as blocks and is refused. Give it to @code{each}, which says that it is one: @codeblock[#:lang "bjolang"]|{ -(: labelled (-> string int (Vec Block))) -(defun (labelled label n) - [label ": " n nl]) - -(show (labelled "cups" 3)) ;; cups: 3 -(show [">> " (each (labelled "pots" 1))]) ;; >> pots: 1 -}| - -A lambda has no signature, so a vec literal returned from one is not read as blocks: in @code{(vec-map (fun (i) [(record-ref i name) (record-ref i stock)]) stock)} the vec has a string and an int in it, and is refused. Put the literal under @code{each}, which says that it is content, or write a function with a signature: - -@codeblock[#:lang "bjolang"]|{ -(show [(each (vec-map (fun (i) (each [(record-ref i name) ": " (record-ref i stock) nl])) - stock))]) +(show (vec-map (fun (i) (each [(record-ref i name) ": " (record-ref i stock) nl])) + stock)) ;; Green tea: 12 ;; Gaiwan: 2 ;; Kettle: 0 }| -@code{nothing} is the block that writes nothing, for the branch that has nothing to say. In a vec literal, what an @code{if}, a @code{match} or a @code{cond} gives is read as any element is, so an arm can be a string. In a function declared to return a @code{Block}, nothing is converted, so there the arms are blocks: +A lambda given to a function in @code{(std fmt)}, such as @code{joined}, needs no @code{each}: their signatures say that the lambda returns a block. + +@code{nothing} is the block that writes nothing, for the branch that has nothing to say: @codeblock[#:lang "bjolang"]|{ (: greeting (-> string (Option string) Block)) (defun (greeting name title) - (each ["Hello, " - (match title - ((Some t) (each [t " "])) - (None nothing)) - name "!"])) + ["Hello, " + (match title + ((Some t) [t " "]) + (None nothing)) + name "!"]) (: medal (-> int Block)) (defun (medal place) - (cond ((= place 1) (BStr "gold")) - ((= place 2) (BStr "silver")) + (cond ((= place 1) "gold") + ((= place 2) "silver") (else nothing))) (show [(greeting "Ada" (Some "Dr")) nl (greeting "Bob" None) nl]) ;; Hello, Dr Ada! ;; Hello, Bob! +(show [(medal 1) "/" (medal 2) "/" (medal 3) nl]) ;; gold/silver/ }| -A list is written with @code{joined}, which takes a function that makes each element's block and a separator, which is content too: +A list is written with @code{joined}, which takes a function that makes each element's block, and a separator: @codeblock[#:lang "bjolang"]|{ -(show [(joined BStr (list "tea" "milk" "honey") [", "]) nl]) ;; tea, milk, honey -(show [(joined/last BStr (fun (s) (each ["and " s])) - (list "tea" "milk" "honey") [", "]) nl]) ;; tea, milk, and honey -(show [(joined (fun (s) (each ["* " s])) (list "tea" "milk") [nl]) nl]) +(show [(joined BStr (list "tea" "milk" "honey") ", ") nl]) ;; tea, milk, honey +(show [(joined/last BStr (fun (s) ["and " s]) + (list "tea" "milk" "honey") ", ") nl]) ;; tea, milk, and honey +(show [(joined (fun (s) ["* " s]) (list "tea" "milk") nl) nl]) ;; * tea ;; * milk }| @@ -794,13 +742,13 @@ A list is written with @code{joined}, which takes a function that makes each ele @code{padded/left}, @code{padded/right} and @code{padded/both} pad content to a width: aligned right, left, or centred. Padding never cuts. @code{trimmed/left}, @code{trimmed/right} and @code{trimmed/both} cut content to a width, and keep its end, its start or its middle. @code{fitted/left}, @code{fitted/right} and @code{fitted/both} do both, so the content is exactly the width. @codeblock[#:lang "bjolang"]|{ -(show [(padded/right 8 [who]) (padded/left 4 [points]) nl]) ;; ada 31 -(show [(with-pad #\0 [(padded/left 4 [42])]) nl]) ;; 0042 -(show [(with-pad #\= [(padded/both 20 [" Receipt "])]) nl]) ;; ===== Receipt ====== -(show [(trimmed/left 6 ["/home/ada/scores.txt"]) nl]) ;; es.txt +(show [(padded/right 8 who) (padded/left 4 points) nl]) ;; ada 31 +(show [(with-pad #\0 (padded/left 4 42)) nl]) ;; 0042 +(show [(with-pad #\= (padded/both 20 " Receipt ")) nl]) ;; ===== Receipt ====== +(show [(trimmed/left 6 "/home/ada/scores.txt") nl]) ;; es.txt (show [(with-fmt current-ellipsis "โ€ฆ" - [(trimmed/left 12 ["/home/ada/scores.txt"])]) nl]) ;; โ€ฆ/scores.txt -(show ["[" (fitted/both 6 ["ab"]) "]" nl]) ;; [ ab ] + (trimmed/left 12 "/home/ada/scores.txt")) nl]) ;; โ€ฆ/scores.txt +(show ["[" (fitted/both 6 "ab") "]" nl]) ;; [ ab ] }| @code{space-to} pads to a column, whatever was written before it, and @code{with-pad} changes what the padding is made of. Together they make a leader: @@ -835,7 +783,7 @@ The digits are always the invariant culture's, so the machine's locale never sho @codeblock[#:lang "bjolang"]|{ (show [(with-fmt current-width 20 - [(justified ["Bjolang lays text out with blocks, and blocks nest."])]) + (justified "Bjolang lays text out with blocks, and blocks nest.")) nl]) ;; Bjolang lays text ;; out with blocks, and @@ -846,15 +794,15 @@ The first line starts where the cursor is, so after a label it has less room. Th @subsubsection{Tables} -A table is the one place where a vec of vecs is right. @code{tabular} takes a vec of columns, and @code{tabular/rows} a vec of rows. Each column or row is a @code{(Vec Block)}, so the inner vecs are read as blocks as any other content is: +A table is the place where the brackets matter. @code{tabular} takes a vec of columns, and @code{tabular/rows} a vec of rows. Each column or row is a @code{(Vec Block)}, with a block for each cell: @codeblock[#:lang "bjolang"]|{ -(show [(tabular [["tea" nl "gaiwan" nl "kettle"] - [3 nl 1240 nl 90]] - #:sep " " #:align [AlignLeft AlignRight])]) +(show (tabular [["tea" nl "gaiwan" nl "kettle"] + [3 nl 1240 nl 90]] + #:sep " " #:align [AlignLeft AlignRight])) -(show [(tabular/rows [["tea" 3] ["gaiwan" 1240] ["kettle" 90]] - #:sep " " #:align [AlignLeft AlignRight])]) +(show (tabular/rows [["tea" 3] ["gaiwan" 1240] ["kettle" 90]] + #:sep " " #:align [AlignLeft AlignRight])) }| Both write the same table: @@ -865,12 +813,12 @@ gaiwan 1240 kettle 90 }| -In @code{tabular} a column's cells are separated by @code{nl}. In @code{tabular/rows} each element of a row is one cell, so a cell of several pieces is under @code{each}, and a newline in a cell gives the row another line: +In @code{tabular} a column's cells are separated by @code{nl}. In @code{tabular/rows} each element of a row is one cell, so a cell of several pieces is a vec in the row, and a newline in a cell gives the row another line: @codeblock[#:lang "bjolang"]|{ -(show [(tabular/rows [["tea" (each [3 " cups"])] - ["kettle" (each ["boils in" nl "4 minutes"])]] - #:style StyleAscii)]) +(show (tabular/rows [["tea" [3 " cups"]] + ["kettle" ["boils in" nl "4 minutes"]]] + #:style StyleAscii)) }| @codeblock[#:lang "text"]|{ @@ -881,7 +829,7 @@ In @code{tabular} a column's cells are separated by @code{nl}. In @code{tabular/ +--------+-----------+ }| -Rows from data come from a function with a signature, for the reason given above. Here is the high-score ledger from "Files and I/O" as a table: +Rows from data come from a function with a signature, for the reason given above: a row of a string and an int has no element type of its own. Here is the high-score ledger from "Files and I/O" as a table: @codeblock[#:lang "bjolang"]|{ (: score-row (-> (Tuple string int) (Vec Block))) @@ -889,10 +837,10 @@ Rows from data come from a function with a signature, for the reason given above (def (Tuple name points) s) [name points]) -(show [(tabular/rows (vec-map score-row (read-scores)) - #:titles ["Player" "Score"] - #:align [AlignLeft AlignRight] - #:style StyleAscii)]) +(show (tabular/rows (vec-map score-row (read-scores)) + #:titles ["Player" "Score"] + #:align [AlignLeft AlignRight] + #:style StyleAscii)) }| @codeblock[#:lang "text"]|{ @@ -910,9 +858,9 @@ A table whose rows are not all known yet, such as results that come one at a tim @codeblock[#:lang "bjolang"]|{ (def layout (table-layout [8 6] #:sep " " #:align [AlignLeft AlignRight])) -(show [(table-row layout ["Player" "Score"])]) +(show (table-row layout ["Player" "Score"])) (loop (:for (Tuple name points) (read-scores)) - (:do (show [(table-row layout [name points])]))) + (:do (show (table-row layout [name points])))) }| @codeblock[#:lang "text"]|{ @@ -949,9 +897,9 @@ The pad character, the right margin, the ellipsis and the number formats are par @codeblock[#:lang "bjolang"]|{ (show [(with-fmt current-comma 3 - [(with-fmt current-comma-sep #\space - [(with-fmt current-decimal-sep #\, - [(numeric 1234567.5)])])]) + (with-fmt current-comma-sep #\space + (with-fmt current-decimal-sep #\, + (numeric 1234567.5)))) nl]) ;; 1 234 567,5 }| @@ -965,32 +913,27 @@ The messages for the common mistakes, and what to write instead: @read-table{ | The compiler says | Write | |----------------------------------------------------------------------+-----------------------------------------------------| -| expected @code{(Vec fmt/Block)}, found @code{string} | @code{["x"]} for @code{"x"} | -| expected @code{(Vec fmt/Block)}, found @code{fmt/Block} | @code{[b]} for @code{b} | -| expected @code{fmt/Block}, found @code{(Vec ...)} | @code{(each bs)} for @code{bs} | -| They differ inside the Vec: @code{fmt/Block} against @code{string} | @code{(: bs (Vec Block))} above @code{(def bs ...)} | -| Block: expected a string, an int, a char ...; found a vec | @code{(each [...])}, or no inner brackets | +| Block: ... found a double | @code{(numeric x)} | | Block: ... found the number 1.5 | @code{(numeric 1.5)} | -| Block: ... found a boolean | @code{(if b "yes" "no")} | +| Block: ... found a boolean, or found a bool | @code{(if b "yes" "no")} | +| Block: ... found a (Vec string) | @code{(vec-map BStr xs)}, or @code{(: xs Block)} above @code{(def xs ...)} | | Block: 'n' is a number, and nothing says yet whether it is an int ... | @code{(: n int)} above @code{(def n 3)} | -| what @code{f} returns has the wrong type: expected @code{fmt/Block}, found @code{string} | @code{(BStr s)} or @code{(each [s])} | +| Block: ... found a (Vec (Vec ...)), at a lambda that returns a vec | @code{(each [...])} around the vec the lambda returns | } -When none of these fits, find the place where a @code{Block} and a @code{(Vec Block)} meet, and check it against the rule: content in brackets, one block out. - @subsubsection{A block of your own} @code{captured} renders content to a string and gives it to a function, which returns the block to write in its place. It is how a block that needs the whole of its content is made, and how padding and trimming are written: @codeblock[#:lang "bjolang"]|{ -(: boxed (-> (Vec Block) Block)) -(defun (boxed bs) - (captured (fun (s) (each ["[" s "]"])) bs)) +(: boxed (-> Block Block)) +(defun (boxed b) + (captured (fun (s) ["[" s "]"]) b)) (show [(boxed ["x = " 7]) nl]) ;; [x = 7] }| -A block that writes as it goes is a @code{BRun}: a function given the sink to write to and the column the cursor is at, which returns the column it ends at. @code{run-blocks} runs content inside one. The module's page describes both. +A block that writes as it goes is a @code{BRun}: a function given the sink to write to and the column the cursor is at, which returns the column it ends at. @code{run-block} runs a block inside one. The module's page describes both. @subsection{Concurrency} From 4f8da92dc22e4d9f47bb94eb1231a29dbba685c7a79ad653225c36562ebb57a5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 12:20:48 +0200 Subject: [PATCH 06/38] Formatting text: settings by parameterize around show, or with-fmt* MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Settings section says where a parameterize of a setting works: around show, not around the code that builds a block. For part of an output it shows with-fmt*, which binds several settings as parameterize does. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 292041f..9677a5d 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -893,14 +893,25 @@ The rows can be spliced, @code{(rows ,scores)}, from a @code{(Vec (Vec TableCell @subsubsection{Settings} -The pad character, the right margin, the ellipsis and the number formats are parameters. @code{with-fmt} binds one around some blocks while they run, and @code{with-pad} is @code{with-fmt} for @code{current-pad}. Settings nest: +The pad character, the right margin, the ellipsis and the number formats are parameters. For a whole output, set them with @code{parameterize} around the @code{show}: @codeblock[#:lang "bjolang"]|{ -(show [(with-fmt current-comma 3 - (with-fmt current-comma-sep #\space - (with-fmt current-decimal-sep #\, - (numeric 1234567.5)))) - nl]) +(parameterize ((current-width 20)) + (show (wrapped "Bjolang lays text out with blocks, and blocks nest."))) +;; Bjolang lays text +;; out with blocks, and +;; blocks nest. +}| + +A block reads its settings when it runs, which is in @code{show}, not when it is built. So the @code{parameterize} must be around the @code{show}: around the code that builds the block, it has ended before the block runs, and does nothing. @code{(show (parameterize ((current-width 20)) (wrapped text)))} wraps at the default width, 78. + +For part of an output, such as one paragraph, one cell or the numbers of one column, put the setting in the layout. @code{with-fmt} binds one setting around a block while it runs, @code{with-fmt*} binds several, written as @code{parameterize} is, and @code{with-pad} is @code{with-fmt} for @code{current-pad}: + +@codeblock[#:lang "bjolang"]|{ +(show (with-fmt* ((current-comma 3) + (current-comma-sep #\space) + (current-decimal-sep #\,)) + (numeric 1234567.5) nl)) ;; 1 234 567,5 }| From 84000bf3844c9e263b2cc6e0b9d4aadd85c1e0ecb7f443d423f666f41b570d5c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 13:24:25 +0200 Subject: [PATCH 07/38] A main that does not return an int exits with 0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first program no longer ends in a 0, and the appendix says what the compiler knows about main. The unit type is spelled void in the signatures that wrote unit, which is its value, and print-n-times has a loop clause for its body. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 19 +++++++++---------- _04.Appendices.sz | 4 ++-- 2 files changed, 11 insertions(+), 12 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 3cad378..1844606 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -24,21 +24,20 @@ git pull @subsection{Your first bjolang program} -Bjolang compiles to c#. Many of the design decisions behind bjolang stems from this limitation. The first one you will encounter is that a program needs a main function. like the one in c#, that returns an int. If execution ended correctly it should return 0 if the program executed successfully. +Bjolang compiles to c#. Many of the design decisions behind bjolang stems from this limitation. The first one you will encounter is that a program needs a main function, like the one in c#. An int it returns is the program's exit code, 0 when it ran correctly; a main that returns anything else exits with 0. @codeblock[#:lang "bjolang"]{ -(: print-hello (-> string unit)) +(: print-hello (-> string void)) (defun (print-hello name) (println (str "Hello, " name "!"))) (defun (main args) - (print-hello "User") - 0) + (print-hello "User")) } This can then be run using @code{bjo run filename}, and it should print @code{Hello, User!}. -What you can see here is that defun is the form for defining a function. main is special, it needs no type declaration. print-hello is a regular function, and thus has to declare its type. Everything at toplevel needs to do that. The form is (: name type). A function type is declared as (-> ArgType ... Return-Type). print-hello takes a string argument and returns unit. unit is the name for nothing. +What you can see here is that defun is the form for defining a function. main is special, it needs no type declaration. print-hello is a regular function, and thus has to declare its type. Everything at toplevel needs to do that. The form is (: name type). A function type is declared as (-> ArgType ... Return-Type). print-hello takes a string argument and returns void. void is the name for nothing: the unit type, whose one value is what println gives back. @subsection{Declaring types} For most of your bjolang code, you will only declare types in the top level. Types within functions are inferred, so declaring types there is almost never necessary. A value type is declared just like a function, but without the function type declaration: @@ -62,7 +61,7 @@ You can also declare your own types. Either aliases of other types, records of y This can then be destructed using pattern matching. @codeblock[#:lang "bjolang"]{ -(: print-user (-> User unit)) +(: print-user (-> User void)) (defun (print-user u) (match u ;; Subpatterns: here we match name, age, the Card rank @@ -119,14 +118,14 @@ A function takes any number of arguments and produces a value. The signature of ;; the last expression in the body is the return value (* x 2)) -(:print-n-times (-> string int unit)) +(: print-n-times (-> string int void)) (defun (print-n-times what n) (loop (:for i (up-from 0 #:to n)) - (println what))) + (:do (println what)))) ;; call twice takes a function take takes an argument -;; of type %a and returns unit. -(: call-twice (-> (-> %a unit) %a unit)) +;; of type %a and returns void. +(: call-twice (-> (-> %a void) %a void)) (defun (call-twice f val) (f val) (f val)) diff --git a/_04.Appendices.sz b/_04.Appendices.sz index 5a2cd06..26c6767 100644 --- a/_04.Appendices.sz +++ b/_04.Appendices.sz @@ -67,12 +67,12 @@ When compiled, the frontend rejects this with: Type Error: Function 'greet' requires a type signature (: greet ...) at hello.bjo:1:1 }| -@strong{Why it happens:} Unlike local helper functions defined inside another function, every top-level definition in a module must carry an explicit type signature. The compiler requires this so that module interfaces are self-documenting and can be emitted into the compiled assembly's public metadata without requiring whole-program inference across files. The only exception is @code{main}, which has a fixed signature known to the compiler. +@strong{Why it happens:} Unlike local helper functions defined inside another function, every top-level definition in a module must carry an explicit type signature. The compiler requires this so that module interfaces are self-documenting and can be emitted into the compiled assembly's public metadata without requiring whole-program inference across files. The only exception is @code{main}: the compiler knows what it takes, the arguments as a @code{(Vec string)}, and what it returns is the exit code when it is an int. @strong{How to fix it:} Add an explicit signature right before the definition: @codeblock[#:lang "bjolang"]|{ -(: greet (-> string unit)) +(: greet (-> string void)) (defun (greet name) (println (str "Hello, " name))) }| From 3f8c13cc2f49da3f2f644f9e1839aa774bd33c6bded015b6bf88820d5aa3a8e5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 13:24:31 +0200 Subject: [PATCH 08/38] Formatting text: a number bound by def needs no type MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A number bound by def is read as the number written there would be, so the guide no longer asks for (: n int), and says when (: n double) is needed. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 9677a5d..ffb9fff 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -627,7 +627,6 @@ So one thing needs no brackets, and several things are a vec: @codeblock[#:lang "bjolang"]|{ (def name "Ada") -(: n int) (def n 3) (show "Hello\n") ;; Hello @@ -658,7 +657,7 @@ Anything else is converted first: @code{BStr}, @code{BInt}, @code{BChar} and @code{BMany} are the cases of @code{Block}. You rarely write them, but like any constructor they are functions, so @code{(vec-map BStr names)} is a vec of blocks. -@strong{A number bound by @code{def}.} The @code{(: n int)} above is needed. @code{(def n 3)} alone does not say which kind of number @code{n} is, as it could still become a long or a double, so it cannot be read as a block, and the compiler says so at @code{n}. The same holds for @code{(def xs [1 2 3])} and @code{(def total (+ 1 2))}. A number that comes from a typed parameter, a record field or a function such as @code{vec-length} has its type already, and a number written in the vec itself, as in @code{["n = " 3]}, is an int. +@strong{A number bound by @code{def}.} @code{(def n 3)} does not say which kind of number @code{n} is: it could still become a long or a double. Where a block is wanted, it is read as the number written there would be, so @code{["n = " n]} is what @code{["n = " 3]} is, and @code{n} is an int from then on. A number that is used as a double later, as in @code{(+ n 0.5)}, needs @code{(: n double)} above its @code{def}. @strong{Where a block is wanted.} A string or a vec is read as a block where the compiler knows that a block is wanted: as an argument, as what a function declared to return a @code{Block} returns, and as the value of a binding whose type is written, such as @code{(: parts Block)}. A @code{def} without a type is what its value is: @code{(def parts ["a" "b"])} is a @code{(Vec string)}, and @code{(show parts)} is refused. Write @code{(: parts Block)} above it. @@ -928,7 +927,6 @@ The messages for the common mistakes, and what to write instead: | Block: ... found the number 1.5 | @code{(numeric 1.5)} | | Block: ... found a boolean, or found a bool | @code{(if b "yes" "no")} | | Block: ... found a (Vec string) | @code{(vec-map BStr xs)}, or @code{(: xs Block)} above @code{(def xs ...)} | -| Block: 'n' is a number, and nothing says yet whether it is an int ... | @code{(: n int)} above @code{(def n 3)} | | Block: ... found a (Vec (Vec ...)), at a lambda that returns a vec | @code{(each [...])} around the vec the lambda returns | } From cefa2313add8eeac492ae7e60a215583394f494c25ac3b1e1df312a3d34581f4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 18:07:05 +0200 Subject: [PATCH 09/38] Timers take a Duration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit timeout and with-deadline take a Duration, made with milliseconds, seconds, minutes or hours; within takes one too. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index ffb9fff..39bce26 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -955,7 +955,7 @@ Functions that may suspend executionโ€”whether waiting on a channel, sleeping, o @codeblock[#:lang "bjolang"]|{ (: think (-bjo-> int int)) (defbjo (think n) - (sync (timeout 50)) ;; wait 50 ms, without holding up a thread + (sync (timeout (milliseconds 50))) ;; wait 50 ms, without holding up a thread (* n n)) }| @@ -1035,29 +1035,31 @@ Crucially, calling @code{(chan-recv seat)} does not perform the receive immediat |---------------------------+-------------------------------------------------------------| | @code{(chan-recv ch)} | Receives a value from channel @code{ch} | | @code{(chan-send ch v)} | Sends @code{v} on channel @code{ch} | -| @code{(timeout ms)} | Fires after @code{ms} milliseconds | +| @code{(timeout d)} | Fires after the @code{Duration} @code{d} | | @code{(choose e1 e2 ...)} | Fires when the first of the supplied events occurs | | @code{(wrap e f)} | Fires when @code{e} occurs, applying @code{f} to its value | } +A @code{Duration} is a length of time, made with @code{milliseconds}, @code{seconds}, @code{minutes} or @code{hours}: @code{(timeout (seconds 5))}. It has its unit in its type, so a timer cannot be given a bare number, and one too long for its range raises rather than wrapping round. + With these combinators, deadlines and timeouts become simple functions that compose over any event. For instance, we can enforce a response timeout on our card players: @codeblock[#:lang "bjolang"]|{ -(: within (-> int (Event %a) (Event (Option %a)))) -(defun (within ms ev) +(: within (-> Duration (Event %a) (Event (Option %a)))) +(defun (within d ev) (choose (wrap ev #(Some &)) - (wrap (timeout ms) (fun (u) None)))) + (wrap (timeout d) (fun (u) None)))) (: slow-player (-bjo-> (Chan string) void)) (defbjo (slow-player seat) - (sync (timeout 200)) + (sync (timeout (milliseconds 200))) (sync (chan-send seat "7โ™ฃ"))) (defbjo (main) (def seat (make-chan)) (spawn (slow-player seat)) - (println (sync (within 50 (chan-recv seat)))) ;; None - (println (sync (within 500 (chan-recv seat)))) ;; (Some 7โ™ฃ) + (println (sync (within (milliseconds 50) (chan-recv seat)))) ;; None + (println (sync (within (milliseconds 500) (chan-recv seat)))) ;; (Some 7โ™ฃ) 0) }| @@ -1097,14 +1099,14 @@ The top-level @code{main} function acts as an implicit root scope, which is why |---------------------------------------+------------------------------------------------------------| | @code{(with-scope body ...)} | Waits for child fibers and finalizes open file ports | | @code{(with-cancel (cancel) body ...)} | Binds a @code{cancel} procedure to signal child fibers | -| @code{(with-deadline ms body ...)} | Cancels the scope automatically after @code{ms} ms | +| @code{(with-deadline d body ...)} | Cancels the scope automatically after the duration @code{d} | | @code{(with-shield body ...)} | Shields cleanup operations from parent cancellation | } @codeblock[#:lang "bjolang"]|{ (: fetch (-bjo-> string void)) (defbjo (fetch name) - (sync (timeout 30)) + (sync (timeout (milliseconds 30))) (println #"fetched ${name}")) (: count-up (-bjo-> (Chan int) void)) @@ -1115,7 +1117,7 @@ The top-level @code{main} function acts as an implicit root scope, which is why (defbjo (main) ;; Both are fetched when this returns, or the deadline raises. - (with-deadline 1000 + (with-deadline (seconds 1) (spawn (fetch "a")) (spawn (fetch "b"))) (println "both done") @@ -1249,7 +1251,7 @@ If a transaction raises an exception, it writes nothing, and @code{atomically} r @codeblock[#:lang "bjolang"]|{ ;; A card from the pile, or None after 100 ms. (sync (choose (wrap (atomically-evt (fun (tx) (take-top tx pile))) Some) - (wrap (timeout 100) (fun (_) None)))) + (wrap (timeout (milliseconds 100)) (fun (_) None)))) }| Inside a transaction, @code{(tx-or-else tx a b)} runs @code{a}, and if @code{a} retries, undoes its writes and runs @code{b}. If both retry, the whole transaction waits for a change to anything either of them read. From 8729cb3ef7c864a48f35f146daae865f305c6e8ef2449a821dd31078b72e027a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 20 Oct 2026 22:15:21 +0200 Subject: [PATCH 10/38] Traits: implementations for particular type arguments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One type may have an implementation at (List int) and another at (List string). The section says how the compiler picks one, and that overlapping implementations and calls at a type it cannot decide are errors. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 1844606..14b9eff 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -2101,6 +2101,28 @@ A trait has exactly one implementor type. When a method needs another type that This is how the prelude's own @code{Iterable}, @code{Collection} and @code{Refable} work, and why @code{length} and @code{fold} work on so many types. +@subsubsection{Implementations for particular type arguments} + +An implementation may be written for a generic type at particular type arguments, and one type may have several, one for each: + +@codeblock[#:lang "bjolang"]|{ +(impl (Describe (List int)) + (defun (describe xs) "some numbers")) + +(impl (Describe (List string)) + (defun (describe xs) "some words")) + +(describe (list 1 2)) ;; "some numbers" +(describe (list "a" "b")) ;; "some words" +}| + +The compiler picks the one whose arguments match. Two rules keep that choice unambiguous: + +@itemlist[ + @item{Two implementations may not overlap. With @code{(List int)} already there, @code{(impl (Describe (List %a)) ...)} is refused, since a @code{(List int)} would then have two.} + @item{The type has to be known where the call is. @code{(describe (list))} is an error, because an empty list could be either, and so is a call at @code{(List %a)} in a generic function: neither implementation is written for every element type. Give such a value an annotation, or write one implementation for @code{(List %a)} with a @code{where}, as in the next section.} +] + @subsubsection{Conditional and blanket implementations} An implementation for a generic type can require something of its type variables, with a @code{where} after the head. A list can be described if its elements can: @@ -2114,7 +2136,7 @@ An implementation for a generic type can require something of its type variables (describe (list Clubs Hearts)) ;; "clubs, hearts" }| -A blanket implementation is one for a bare type variable. It applies to every type that has no implementation of its own, and a more specific implementation always wins over it. Only the module that declares a trait may write its blanket: +A blanket implementation is one for a bare type variable. It applies to every type that has no implementation of its own, and a more specific implementation always wins over it. A type whose head has implementations only for other arguments, such as a @code{(List bool)} beside the @code{(List int)} above, is an error rather than the blanket. Only the module that declares a trait may write its blanket: @codeblock[#:lang "bjolang"]|{ (def/trait (Weight %a) From 6d9c9bf8a0ae4947242a791bbda9985f0f0bcf0153f71ef29a8fe0581b664d66 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Wed, 21 Oct 2026 06:33:00 +0200 Subject: [PATCH 11/38] Traits: traits that require other traits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A trait may require another of every implementor, with a where clause in the trait. The section shows a function that asks for one trait and uses the other, and notes that Ord now requires Eq. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 14b9eff..87d6a59 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -2152,6 +2152,26 @@ A blanket implementation is one for a bare type variable. It applies to every ty (weight "hello") ;; 5 }| +@subsubsection{Traits that require other traits} + +A trait may say that every type implementing it implements another trait too, with a @code{where} in the trait itself. Something that greets must have a name: + +@codeblock[#:lang "bjolang"]|{ +(def/trait (Named %a) + (: name-of (-> %a string))) + +(def/trait (Greet %a) + (where (Named %a)) + (: greeting (-> %a string))) + +(: introduce (-> %a string) (where (Greet %a))) +(defun (introduce x) (str (name-of x) " says " (greeting x))) +}| + +@code{introduce} asks only for @code{Greet}, and may still call @code{name-of}: a type that can greet is known to have a name. An implementation of @code{Greet} for a type with no @code{Named} is a compile error at the implementation. + +The prelude's @code{Ord} requires @code{Eq} this way, so a function with @code{(where (Ord %a))} can use @code{=} as well as @code{compare}. + @subsubsection{Eq, Ord and ->str} Three traits from the prelude are worth knowing by heart, since the rest of the library leans on them. From bd71102c19b6d08ec634a75994d2afe8bd6921470d3ca3765994d7f346b64943 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Wed, 21 Oct 2026 08:25:02 +0200 Subject: [PATCH 12/38] Newtypes, and giving one the traits of the type it wraps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A newtype beside aliases in "Your own types", and #:via and type/derive for one in the traits chapter. ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-5-5) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 44 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 87d6a59..8bd22cb 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -354,6 +354,25 @@ A type alias is a different name for a type that already exists. It is declared (type (: Stringalias string)) } +@subsubsection{Newtypes} + +A newtype is the opposite of an alias: a type of its own, which the compiler will not mix up with the type it wraps, but which is that type when the program runs, so it costs nothing. + +@codeblock[#:lang "bjolang"]|{ +(type (: UserId (Newtype int))) +(type (: OrderId (Newtype int))) + +(def u (UserId 7)) ;; built by its name +(def (UserId n) u) ;; taken apart by a pattern: n is 7 + +(: (-> UserId string)) +(defun (lookup-user id) ...) + +(lookup-user (OrderId 7)) ;; a compile error: an OrderId is not a UserId +}| + +A newtype does not get the traits of the type it wraps: @code{(+ u u)} is an error, though @code{(+ 7 7)} is not. It gets those you give it, as the section on traits shows. + @subsubsection{Records and structs} Records and structs are direct mappings to c# records and immutable structs. The constructor is the type name, together with parenthesised fields. As follows. @@ -2172,6 +2191,31 @@ A trait may say that every type implementing it implements another trait too, wi The prelude's @code{Ord} requires @code{Eq} this way, so a function with @code{(where (Ord %a))} can use @code{=} as well as @code{compare}. +@subsubsection{Giving a newtype the traits of the type it wraps} + +A newtype is the type it wraps when the program runs, so it can borrow that type's implementation of a trait as it is, with @code{#:via}: + +@codeblock[#:lang "bjolang"]|{ +(type (: Meters (Newtype double))) + +(impl (Eq Meters) #:via double) +(impl (Ord Meters) #:via double) +(impl (Num Meters) #:via double) ;; + - * / +(impl (Ordered Meters) #:via double) ;; < > <= >= + +(+ (Meters 1.5) (Meters 2.0)) ;; (Meters 3.5) +(+ (Meters 1.5) 2) ;; a number literal is a Meters too +}| + +@code{type/derive} writes these for you, for any trait, when the type is a newtype: + +@codeblock[#:lang "bjolang"]|{ +(type/derive (Eq Ord Num Ordered) + (: Meters (Newtype double))) +}| + +A trait that should behave differently is written by hand instead. That is the other use of a newtype: the same data with a different behaviour, such as an @code{Eq} that ignores case on a @code{(Newtype string)}. Sets and maps of the newtype use that implementation, not the wrapped type's. + @subsubsection{Eq, Ord and ->str} Three traits from the prelude are worth knowing by heart, since the rest of the library leans on them. From 481538a1cbed74c04dc1f7b3772d2a3ca14538b68bfb5456727e6787999db46f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Wed, 21 Oct 2026 13:05:45 +0200 Subject: [PATCH 13/38] Numbers: the full set of types, and real literals that take their type MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The numeric types table lists bigint, int128, uint128, float and decimal. A real literal without a suffix is a double, float or decimal as the context says. A boolean prints as #t. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 33 +++++++++++++++++++++------------ _02_.Writing-programs.sz | 2 +- 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 8bd22cb..ddbdef5 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -2315,22 +2315,25 @@ Nothing about @code{list-sort} knows about cards. It asks for @code{(Ord %a)}, a @subsubsection{The types} -There are eight numeric types, each a .NET primitive: +The numeric types are the .NET ones: @read-table{ -| Type | Range | Type | Range | -|----------------+------------------------------+----------------+------------------------| -| @code{byte} | 0 โ€ฆ 255 | @code{uint} | 0 โ€ฆ 2^32โˆ’1 | -| @code{short} | โˆ’32 768 โ€ฆ 32 767 | @code{long} | โˆ’2^63 โ€ฆ 2^63โˆ’1 | -| @code{ushort} | 0 โ€ฆ 65 535 | @code{ulong} | 0 โ€ฆ 2^64โˆ’1 | -| @code{int} | about ยฑ2.1 billion | @code{double} | 64-bit floating point | +| Type | Range | Type | Range | +|----------------+------------------------------+----------------+---------------------------| +| @code{byte} | 0 โ€ฆ 255 | @code{uint} | 0 โ€ฆ 2^32โˆ’1 | +| @code{short} | โˆ’32 768 โ€ฆ 32 767 | @code{long} | โˆ’2^63 โ€ฆ 2^63โˆ’1 | +| @code{ushort} | 0 โ€ฆ 65 535 | @code{ulong} | 0 โ€ฆ 2^64โˆ’1 | +| @code{int} | about ยฑ2.1 billion | @code{double} | 64-bit floating point | +| @code{bigint} | any integer | @code{float} | 32-bit floating point | +| @code{int128} | โˆ’2^127 โ€ฆ 2^127โˆ’1 | @code{decimal} | 28 digits, exact decimals | +| @code{uint128} | 0 โ€ฆ 2^128โˆ’1 | | | } -There is no @code{float} and no @code{decimal}. A @code{char} is not a number: it can be compared and converted, but not added. +A @code{char} is not a number: it can be compared and converted, but not added. @subsubsection{Writing numbers} -A number with a suffix is of that type. A decimal point or an exponent makes it a @code{double}: +A number with a suffix is of that type. A decimal point or an exponent makes it a real number, which is a @code{double}, a @code{float} or a @code{decimal}, and never an integer: @read-table{ | Written | Type | @@ -2341,11 +2344,14 @@ A number with a suffix is of that type. A decimal point or an exponent makes it | @code{21u} | @code{uint} | | @code{21L} | @code{long} | | @code{21UL} | @code{ulong} | -| @code{21d} @code{2.5} @code{1e3} | @code{double} | +| @code{21d} @code{2.5d} | @code{double} | +| @code{21M} @code{2.5M} | @code{decimal} | +| @code{21I} | @code{bigint} | +| @code{2.5} @code{1e3} | a real type, as the context says | | @code{0x2A} @code{0b101010} | 42, of whatever type the context says | } -A number without a suffix has no type of its own. It takes the type of wherever it is used, and is an @code{int} only when nothing says otherwise: +A number without a suffix has no type of its own. It takes the type of wherever it is used, and is an @code{int}, or a @code{double} if it is real, only when nothing says otherwise: @codeblock[#:lang "bjolang"]|{ (: doubled (-> ushort ushort)) @@ -2355,9 +2361,12 @@ A number without a suffix has no type of its own. It takes the type of wherever (let ((start 5)) (+ start 3L)) ;; 8L: start was a long all along + +(: tenth decimal) +(def tenth 0.1) ;; a decimal, so (+ tenth 0.2) is exactly 0.3 }| -A literal that does not fit the type it ends up with is a compile error. @code{(doubled 99999)} says that 99999 does not fit in a @code{ushort}. +A literal that does not fit the type it ends up with is a compile error. @code{(doubled 99999)} says that 99999 does not fit in a @code{ushort}, and @code{(doubled 2.5)} that 2.5 has a fraction. @subsubsection{Arithmetic} diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 39bce26..5d2d9cf 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -1718,7 +1718,7 @@ Literal prefixes like @code{#map(...)} are reader-level hash macros. You can reg (defun (main) (println #card(Q hearts)) ;; Qโ™ฅ (println #card[10 spades]) ;; 10โ™ , either bracket works - (println (court? #card(K clubs))) ;; True + (println (court? #card(K clubs))) ;; #t (with-each-card (c [#card(A spades) #card(2 hearts)]) (println #"a card: ${c}")) 0) From e448ee0d9336cb2cd3e4843607581dacf9366ffa539ddcc93e0ad486bda39b9e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Wed, 21 Oct 2026 15:38:01 +0200 Subject: [PATCH 14/38] Printing: written and displayed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ->str gives way to two traits, as Scheme's write and display: a value as the reader reads it, and the value for a person. Every type of your own is written in constructor syntax. The examples, the REPL session and the expect output follow. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 112 ++++++++++++++++++++++++++++----------- _02_.Writing-programs.sz | 44 ++++++++------- 2 files changed, 107 insertions(+), 49 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index ddbdef5..c7df994 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -104,9 +104,11 @@ Then there are some special cases, like string interpolation: (a-string "hej")) (println $"This is interpolated: ${an-int} ${a-string}") ;; Which is literally the same as: - (println (str "This is interpolated: " (->str an-int) " " (->str a-string)))) + (println (str "This is interpolated: " (displayed an-int) " " (displayed a-string)))) } +Each hole goes through @code{displayed}, so a string hole gives its text without quotes. See @em{Converting to and from strings} below. + @subsubsection{Function types} A function takes any number of arguments and produces a value. The signature of a function in bjolang is denoted using the arrow @code{->}: @@ -456,7 +458,7 @@ A few things to know about unions: @read-list{ - Case names share a namespace with everything else in the module, so two unions in the same module cannot both have a case called @code{Red}. - A union case cannot have a @code{#:mutable} field. -- @code{println} on a union value without a @code{->str} implementation prints the generated C# name, something like @code{Card { rank = Queen { }, suit = Hearts { } }}. Implement @code{->str} (see the traits section) for anything you intend to print. +- @code{println} on a value of your own type prints it as you would build it, for example @code{(Card (rank Queen) (suit Hearts))} or @code{(Pip 7)}. The compiler writes this for each type you declare. To print a type another way, implement @code{Displayed} (see the traits section). } @subsubsection{Types that refer to themselves and each other} @@ -543,23 +545,31 @@ The rest of this chapter uses a card game as a running example, so here are its (: tricks int)))) ;; How a card prints. Traits are explained later; for now it is enough -;; to know that this is what println and #"...${...}" use. -(impl (->str Suit) - (defun (->str s) - (match s (Clubs "โ™ฃ") (Diamonds "โ™ฆ") (Hearts "โ™ฅ") (Spades "โ™ ")))) +;; to know that Displayed is what println and #"...${...}" use, and that +;; display-into adds the text to the string builder b. +(impl (Displayed Suit) + (defun (display-into s b) + (stringbuilder-add-string! b + (match s (Clubs "โ™ฃ") (Diamonds "โ™ฆ") (Hearts "โ™ฅ") (Spades "โ™ "))))) -(impl (->str Rank) - (defun (->str r) - (match r - ((Pip n) (int->string n)) - (Jack "J") (Queen "Q") (King "K") (Ace "A")))) +(impl (Displayed Rank) + (defun (display-into r b) + (stringbuilder-add-string! b + (match r + ((Pip n) (int->string n)) + (Jack "J") (Queen "Q") (King "K") (Ace "A"))))) -(impl (->str Card) - (defun (->str c) #"${(record-ref c rank)}${(record-ref c suit)}")) +(impl (Displayed Card) + (defun (display-into c b) + (display-into (record-ref c rank) b) + (display-into (record-ref c suit) b))) (println (Card (rank Queen) (suit Hearts))) ;; Qโ™ฅ +(written (Card (rank Queen) (suit Hearts))) ;; "(Card (rank Queen) (suit Hearts))" }| +Without the three implementations, @code{println} prints the card as it is written: @code{(Card (rank Queen) (suit Hearts))}. + @code{Card} is a struct because it is two small values and is copied around a lot. @code{Player} is a record because it holds a whole hand. That is the usual rule of thumb: a struct for something small that behaves like a value, a record for everything else. A struct also cannot contain itself and cannot have mutable fields. Records are updated by copying, with @code{record-set}: @@ -1926,12 +1936,34 @@ Do not build strings by writing to a string port. Writing to a port is I/O as fa @subsubsection{Converting to and from strings} -@code{->str} turns anything into a string, and @code{println} and interpolation use it. It is a trait, so your own types can decide what they look like (the card game did that for @code{Card}). For the basic types there are also named conversions: +Two functions turn any value into a string: + +@read-list{ +- @code{written} gives the value as the reader reads it, as Scheme's @code{write} does. A string is in quotes, a char is @code{#\a}, a number keeps the suffix of its type (@code{21L}), and a double always has a point (@code{3.0}). The REPL shows a result this way. +- @code{displayed} gives the value for a person to read, as Scheme's @code{display} does. A string is its text, and a number has no suffix. @code{println}, @code{print} and interpolation use it. +} + +@codeblock[#:lang "bjolang"]|{ +(written "hej") ;; "\"hej\"" +(displayed "hej") ;; "hej" +(written 21L) ;; "21L" +(displayed 21L) ;; "21" +(written (list "a" "b")) ;; "(\"a\" \"b\")" +(displayed (list "a" "b")) ;; "(a b)" +(written (Some (Tuple 1 "x"))) ;; "(Some (Tuple 1 \"x\"))" +(println (Fruit (name "banana") (amount 5) (ripeness Rotten))) +;; prints (Fruit (name "banana") (amount 5) (ripeness Rotten)) +}| + +Every type you declare is written as you would build it. A type that prints differently for a person implements @code{Displayed}, as the card game did for @code{Card}; one that is written differently implements @code{Written}. The traits section has more. @code{->str} is deprecated: use @code{displayed}. + +For the basic types there are also named conversions: @codeblock[#:lang "bjolang"]|{ (int->string 42) ;; "42" (double->string 2.5) ;; "2.5" -(->str 3.0) ;; "3" +(displayed 3.0) ;; "3" +(written 3.0) ;; "3.0" (string->int "42") ;; 42, and throws on anything that is not a number (string->double "1.5") ;; 1.5 (char->string #\a) ;; "a" @@ -2018,7 +2050,7 @@ Define patterns at the top level, as above, rather than writing @code{#rx(...)} @subsection{Traits} -A trait is a set of functions that a type can implement. It is how bjolang does what other languages do with interfaces, type classes or overloading: @code{=} is a trait method, so is @code{->str}, and so are @code{length}, @code{fold} and @code{compare}. +A trait is a set of functions that a type can implement. It is how bjolang does what other languages do with interfaces, type classes or overloading: @code{=} is a trait method, so is @code{displayed}, and so are @code{length}, @code{fold} and @code{compare}. @subsubsection{Declaring and implementing a trait} @@ -2089,7 +2121,7 @@ A generic function that calls a trait method has to say that its type variable i @code{describe-all} can now be called with a vec of anything that has a @code{Describe}, and calling it with anything else is a compile error at the call. -For @code{Eq} and @code{->str} you may leave the @code{where} out: the compiler adds them by itself when a function uses @code{=} or @code{->str} on a type variable. +For @code{Eq}, @code{Written} and @code{Displayed} you may leave the @code{where} out: the compiler adds them by itself when a function uses @code{=}, @code{written} or @code{displayed} on a type variable. @subsubsection{Associated types} @@ -2189,7 +2221,7 @@ A trait may say that every type implementing it implements another trait too, wi @code{introduce} asks only for @code{Greet}, and may still call @code{name-of}: a type that can greet is known to have a name. An implementation of @code{Greet} for a type with no @code{Named} is a compile error at the implementation. -The prelude's @code{Ord} requires @code{Eq} this way, so a function with @code{(where (Ord %a))} can use @code{=} as well as @code{compare}. +The prelude's @code{Ord} requires @code{Eq} this way, so a function with @code{(where (Ord %a))} can use @code{=} as well as @code{compare}. @code{Displayed} requires @code{Written} in the same way. @subsubsection{Giving a newtype the traits of the type it wraps} @@ -2216,16 +2248,17 @@ A newtype is the type it wraps when the program runs, so it can borrow that type A trait that should behave differently is written by hand instead. That is the other use of a newtype: the same data with a different behaviour, such as an @code{Eq} that ignores case on a @code{(Newtype string)}. Sets and maps of the newtype use that implementation, not the wrapped type's. -@subsubsection{Eq, Ord and ->str} +@subsubsection{Eq, Ord, Written and Displayed} -Three traits from the prelude are worth knowing by heart, since the rest of the library leans on them. +Four traits from the prelude are worth knowing by heart, since the rest of the library leans on them. @read-table{ -| Trait | Methods | Gives you | -|----------------+---------------------------------+------------------------------------------------------------------| -| @code{Eq} | @code{=}, @code{eq-hash} | equality, and use as a key in maps and sets | -| @code{Ord} | @code{compare} | @code{less?}, @code{least}, @code{list-sort}, @code{list-max}, ordered collections | -| @code{->str} | @code{->str} | @code{println}, @code{print}, string interpolation | +| Trait | Methods | Gives you | +|-------------------+-----------------------------------------+------------------------------------------------------------------| +| @code{Eq} | @code{=}, @code{eq-hash} | equality, and use as a key in maps and sets | +| @code{Ord} | @code{compare} | @code{less?}, @code{least}, @code{list-sort}, @code{list-max}, ordered collections | +| @code{Written} | @code{write-into}, @code{written} | the REPL's output, the failure text of @code{expect} | +| @code{Displayed} | @code{display-into}, @code{displayed} | @code{println}, @code{print}, string interpolation | } @code{Eq} and @code{Ord} can be derived with @code{type/derive}, as shown earlier. When you write them by hand, they must be written in the module that declares the type, because they are compiled into the type itself: they become its .NET @code{Equals}, @code{GetHashCode} and @code{CompareTo}, which is why a .NET dictionary or a sorted collection agrees with @code{=} and @code{compare}. @code{=} and @code{eq-hash} must agree: two values that are equal must have the same hash. @code{compare} returns a negative number, zero or a positive number. @@ -2239,19 +2272,36 @@ Three traits from the prelude are worth knowing by heart, since the rest of the (hash-combine (eq-hash (record-ref c rank)) (eq-hash (record-ref c suit))))) }| -@code{->str} has a blanket implementation that uses .NET's @code{ToString}, which is why printing one of your own types shows its C# name until you implement it. +The compiler writes a @code{Written} for every type you declare, so a value prints as you would build it: @code{(Card (rank Queen) (suit Hearts))}, @code{(Pip 7)}, @code{Hearts}. A generic type is written when its type arguments are. An @code{#:opaque} type is written as @code{#}, and a field that holds a function as @code{#}. @code{Displayed} has a blanket implementation that displays a value as it is written. + +To print a type your own way, implement @code{Displayed} for the text a person reads, as the card game did, or @code{Written} to change the written form too. The method to give is @code{display-into} (or @code{write-into}), which adds the text to a string builder; @code{displayed} and @code{written} then come with the trait: + +@codeblock[#:lang "bjolang"]|{ +(type (: Money (Union (: Kr int)))) + +(impl (Displayed Money) + (defun (display-into m b) + (match m ((Kr n) (stringbuilder-add-string! b #"${n} kr"))))) + +(println (Kr 42)) ;; 42 kr +(written (Kr 42)) ;; "(Kr 42)" +}| + +A container that holds other values calls @code{write-into} or @code{display-into} on them with the same builder. @code{elements-print-into} and @code{entries-print-into} from the prelude lay out the elements of a collection, or the entries of a map, for you. + +@code{->str} is deprecated. It still works, through @code{displayed}, but each use gives a warning. @subsubsection{Values of different types in one collection: dyn} A list holds values of one type. When you need a collection of different types that all implement the same trait, pack each value as a @code{(dyn Trait)}: @codeblock[#:lang "bjolang"]|{ -(: render-all (-> (List (dyn ->str)) (List string))) -(defun (render-all xs) (list-map ->str xs)) +(: render-all (-> (List (dyn Displayed)) (List string))) +(defun (render-all xs) (list-map displayed xs)) -(render-all (list (dyn ->str 1) - (dyn ->str "two") - (dyn ->str (Card (rank Queen) (suit Hearts))))) +(render-all (list (dyn Displayed 1) + (dyn Displayed "two") + (dyn Displayed (Card (rank Queen) (suit Hearts))))) ;; '("1" "two" "Qโ™ฅ") }| diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 5d2d9cf..ae452b5 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -22,18 +22,22 @@ Here is the deck from the first part as a module of its own. Put it in @code{dec (: Rank (Union (: Pip int) Jack Queen King Ace)) (: Card (Struct (: rank Rank) (: suit Suit)))) -(impl (->str Suit) - (defun (->str s) - (match s (Clubs "โ™ฃ") (Diamonds "โ™ฆ") (Hearts "โ™ฅ") (Spades "โ™ ")))) +(impl (Displayed Suit) + (defun (display-into s b) + (stringbuilder-add-string! b + (match s (Clubs "โ™ฃ") (Diamonds "โ™ฆ") (Hearts "โ™ฅ") (Spades "โ™ "))))) -(impl (->str Rank) - (defun (->str r) - (match r - ((Pip n) (int->string n)) - (Jack "J") (Queen "Q") (King "K") (Ace "A")))) +(impl (Displayed Rank) + (defun (display-into r b) + (stringbuilder-add-string! b + (match r + ((Pip n) (int->string n)) + (Jack "J") (Queen "Q") (King "K") (Ace "A"))))) -(impl (->str Card) - (defun (->str c) #"${(record-ref c rank)}${(record-ref c suit)}")) +(impl (Displayed Card) + (defun (display-into c b) + (display-into (record-ref c rank) b) + (display-into (record-ref c suit) b))) ;; Not exported: only this module needs them. (def all-suits [Clubs Diamonds Hearts Spades]) @@ -78,7 +82,7 @@ Notice how the imports and exports interact here: @read-list{ - Relative imports: @code{(import "deck.bjo")} specifies a file path relative to the importing file. Bjolang compiles @code{deck.bjo} into @code{deck.dll} ahead of time, automatically rebuilding it whenever it (or any of its dependencies) changes. - Type exports are complete: Exporting a type exposes all its constructors, union cases, and fields. Because @code{Card} is exported, @code{game.bjo} can freely construct @code{(Card (rank Ace) (suit Spades))} and pattern-match on its fields. -- Trait implementations are automatic: Although @code{->str} was not explicitly listed in the export list, its implementations still work in @code{game.bjo}. Because @code{impl} blocks define anonymous implementations rather than named bindings, they cannot be exported individually; they are automatically carried along whenever the module defining them is loaded. +- Trait implementations are automatic: Although @code{Displayed} was not explicitly listed in the export list, its implementations still work in @code{game.bjo}. Because @code{impl} blocks define anonymous implementations rather than named bindings, they cannot be exported individually; they are automatically carried along whenever the module defining them is loaded. - Private definitions: Unexported bindings like @code{all-suits} remain private to @code{deck.bjo} and are inaccessible from outside. } @@ -148,7 +152,7 @@ so its representation is visible only to the code of pile. A value of it can be passed on here, and built and taken apart through the functions that module exports. }| -Even though the type's representation is private, any trait implementations defined in @code{pile.bjo} remain available to consumers. For instance, an @code{(impl (->str Pile) ...)} inside @code{pile.bjo} allows any importing module to convert a @code{Pile} to a string. +Even though the type's representation is private, any trait implementations defined in @code{pile.bjo} remain available to consumers. For instance, an @code{(impl (Displayed Pile) ...)} inside @code{pile.bjo} allows any importing module to print a @code{Pile}. Without one, a @code{Pile} is written as @code{#}, so printing it shows nothing of its representation. @subsubsection{Import modifiers} @@ -221,7 +225,7 @@ While @code{export} publishes definitions authored in the current module, @code{ (re-export Card Rank Suit full-deck shuffle Pile new-pile draw) }| -Re-exporting a type exposes the exact original type, not a distinct copy; a @code{Card} referenced via @code{cards.bjo} is identical to one from @code{deck.bjo}. However, trait implementations (@code{impl}) do not transfer through re-exports. They remain bound to the module where they were defined, meaning any file that wants to format cards with @code{->str} must still import @code{deck.bjo}. +Re-exporting a type exposes the exact original type, not a distinct copy; a @code{Card} referenced via @code{cards.bjo} is identical to one from @code{deck.bjo}. However, trait implementations (@code{impl}) do not transfer through re-exports. They remain bound to the module where they were defined, meaning any file that wants to print cards with their @code{Displayed} implementation must still import @code{deck.bjo}. You can provide alternative names for bindings or macros using @code{(:alias new old)}. Combining @code{:alias} with @code{export} or @code{re-export} allows libraries to offer cleaner or more idiomatic public APIs: @@ -471,7 +475,7 @@ To illustrate, we can add a persistent high-score ledger to our card game. Each (defun (main) (save-score "ada" 31) (save-score "bo" 27) - (println (read-scores)) ;; [(ada, 31) (bo, 27)] + (println (read-scores)) ;; [(Tuple ada 31) (Tuple bo 27)] 0) }| @@ -1730,7 +1734,7 @@ Invalid literal syntax is flagged with a compile-time error at the point of use: The hash macro 'card' failed at bad.bjo:3: a suit is clubs, diamonds, hearts or spades โ€” in hurts }| -Notice the @code{re-export} in @code{cardmacros.bjo}: because the macro expansion emits constructor symbols like @code{Card} and @code{Queen}, those types must be resolvable in the module invoking the macro. Re-exporting them ensures that importing @code{cardmacros.bjo} alone brings the required types into scope. Importing @code{deck.bjo} alongside it remains recommended so that the corresponding @code{->str} implementations are also present. +Notice the @code{re-export} in @code{cardmacros.bjo}: because the macro expansion emits constructor symbols like @code{Card} and @code{Queen}, those types must be resolvable in the module invoking the macro. Re-exporting them ensures that importing @code{cardmacros.bjo} alone brings the required types into scope. Importing @code{deck.bjo} alongside it remains recommended so that the corresponding @code{Displayed} implementations are also present. @subsubsection{Literals read by a grammar} @@ -1850,7 +1854,7 @@ ok: an ace is worth 11 ok: shuffling keeps the cards }| -Whenever an assertion fails, the output reports both the computed value and the expected value: @code{FAILURE: deliberately wrong gave 2 but wanted 3}. +Whenever an assertion fails, the output reports both the computed value and the expected value, each as @code{written} gives it: @code{FAILURE: deliberately wrong gave 2 but wanted 3}, or @code{gave "4" but wanted "5"} for strings. So @code{expect} needs a @code{Written} for the type, which every type has. For asynchronous or resource-intensive testing, @code{(with-test "name" body ...)} executes the test body within an isolated structured concurrency scope configured with a default deadline. Any background fibers spawned during the test are automatically awaited, and the test fails if unclosed file ports remain open when the scope exits. Because @code{with-test} forms a concurrency scope, it must be invoked from within a bjoroutine. @@ -1885,14 +1889,18 @@ You can launch an interactive Read-Eval-Print Loop by running @code{bjo repl}. W bjo> (+ 1 2) 3 bjo> (import "deck.bjo") -bjo> (vec-slice (full-deck) 0 3) -[2โ™ฃ 3โ™ฃ 4โ™ฃ] +bjo> (vec-slice (full-deck) 0 2) +[(Card (rank (Pip 2)) (suit Clubs)) (Card (rank (Pip 3)) (suit Clubs))] +bjo> (displayed (vec-slice (full-deck) 0 3)) +"[2โ™ฃ 3โ™ฃ 4โ™ฃ]" bjo> (defun (double (: x int)) : int (* x 2)) double bjo> (double 21) 42 }| +The REPL shows a result with @code{written}, as the reader reads it: a string in quotes, a card as it is built. @code{println} and @code{displayed} give the text a person reads. + Unlike many Lisp REPLs, Bjolang's REPL does not interpret forms. Instead, each entered interaction is compiled on the fly into an in-memory assembly using the exact same compiler pipeline applied to files on disk, ensuring identical semantics between interactive sessions and compiled production builds. Top-level functions declared in the REPL require explicit type signatures, mirroring the rules for modules. You can supply parameter and return types inline directly on the @code{defun}, or submit a standalone signature on its own line immediately preceding the definition: From 25720e4fb28b7337b4ef54900c02fb95adb69c045e3f71c71a915c41444301a2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 07:15:28 +0200 Subject: [PATCH 15/38] Traits: write the rules' itemlist with a brace body MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @itemlist[...] gave the items as arguments, which samizdat reads as bjodat, and the comma in the prose stopped it. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index c7df994..f021c08 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -2169,10 +2169,10 @@ An implementation may be written for a generic type at particular type arguments The compiler picks the one whose arguments match. Two rules keep that choice unambiguous: -@itemlist[ +@itemlist{ @item{Two implementations may not overlap. With @code{(List int)} already there, @code{(impl (Describe (List %a)) ...)} is refused, since a @code{(List int)} would then have two.} @item{The type has to be known where the call is. @code{(describe (list))} is an error, because an empty list could be either, and so is a call at @code{(List %a)} in a generic function: neither implementation is written for every element type. Give such a value an annotation, or write one implementation for @code{(List %a)} with a @code{where}, as in the next section.} -] +} @subsubsection{Conditional and blanket implementations} From 5a9f65accfb1617cce1f58bed37f54f1ba8ffffe4d5aa49e90f65454c1258340 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 07:59:36 +0200 Subject: [PATCH 16/38] Traits and fmt brought up to the compiler of 21 October MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Where an implementation is written: with its trait or its type, a newtype for anything else, and traits, types and impls usable above where they are written. - One implementation that fits decides the type of a call. - A function is written as # wherever it is; Procedure. - A (dyn T) implements the traits T requires. - Formatting: the lambda that returns a vec of mixed pieces, in the table of errors; the short fmt tour in Language basics in the style of a string being a block. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 41 ++++++++++++++++++++++++++++------------ _02_.Writing-programs.sz | 3 +++ 2 files changed, 32 insertions(+), 12 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index f021c08..2be4d4b 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1983,19 +1983,17 @@ For output where the layout matters, @code{(std fmt)} has two layers. The simple (println* ["Hello, " 42 ", ok? " #t]) ;; Hello, 42, ok? #t }| -The other one is a small layout language. A layout is a @code{(Vec Block)}, where a block is a string, a number, a character or one of the combinators, and @code{show} prints it: +The other one is a small layout language, made of blocks. A string, an int or a character is a block, so is a vec of blocks, and so is what each of the combinators gives back. @code{show} prints a block: @codeblock[#:lang "bjolang"]|{ -(show ["[" (padded/left 6 ["foo"]) "]" nl]) ;; [ foo] -(show ["[" (padded/right 6 ["foo"]) "]" nl]) ;; [foo ] -(show [(numeric/comma 1234567.891) nl]) ;; 1,234,567.891 +(show ["[" (padded/left 6 "foo") "]" nl]) ;; [ foo] +(show ["[" (padded/right 6 "foo") "]" nl]) ;; [foo ] +(show [(numeric 1234567.891 #:comma 3) nl]) ;; 1,234,567.891 (show [(numeric 3.14159 #:precision 2) nl]) ;; 3.14 -(render->string [(trimmed/right 5 ["abcdefgh"])]) ;; "abcde" +(render->string (trimmed/right 5 "abcdefgh")) ;; "abcde" -(show [(joined/last (fun (s) (each [s])) - (fun (s) (each ["and " s])) - (list "a" "b" "c") - [", "]) +(show [(joined/last BStr (fun (s) ["and " s]) + (list "a" "b" "c") ", ") nl]) ;; a, b, and c ;; tabular takes columns, and sizes each to its content. @@ -2007,7 +2005,7 @@ The other one is a small layout language. A layout is a @code{(Vec Block)}, wher ;; bo 7 }| -The full set of combinators, paragraphs and settings is described in the reference for @code{(std fmt)}. +The section on formatting text in "Writing programs" is a guide to the rest: paragraphs, tables, settings, and what to write when a layout does not compile. Every combinator is in the reference for @code{(std fmt)}. @subsubsection{Regular expressions with (std rx)} @@ -2174,6 +2172,8 @@ The compiler picks the one whose arguments match. Two rules keep that choice una @item{The type has to be known where the call is. @code{(describe (list))} is an error, because an empty list could be either, and so is a call at @code{(List %a)} in a generic function: neither implementation is written for every element type. Give such a value an annotation, or write one implementation for @code{(List %a)} with a @code{where}, as in the next section.} } +Where only one implementation fits, it decides the type instead. With @code{(impl (Describe (List int)) ...)} and no other implementation for a list, @code{(describe (list))} is a call at @code{(List int)}, and a function with @code{(where (Describe %a))} called with @code{(list)} is too. + @subsubsection{Conditional and blanket implementations} An implementation for a generic type can require something of its type variables, with a @code{where} after the head. A list can be described if its elements can: @@ -2187,7 +2187,7 @@ An implementation for a generic type can require something of its type variables (describe (list Clubs Hearts)) ;; "clubs, hearts" }| -A blanket implementation is one for a bare type variable. It applies to every type that has no implementation of its own, and a more specific implementation always wins over it. A type whose head has implementations only for other arguments, such as a @code{(List bool)} beside the @code{(List int)} above, is an error rather than the blanket. Only the module that declares a trait may write its blanket: +A blanket implementation is one for a bare type variable. It applies to every type that has no implementation of its own, and a more specific implementation always wins over it. A type whose head has implementations only for other arguments, such as a @code{(List bool)} beside the @code{(List int)} above, is an error rather than the blanket. Only the module that declares a trait may write its blanket, since a blanket is for every type and so for types the module did not declare: @codeblock[#:lang "bjolang"]|{ (def/trait (Weight %a) @@ -2203,6 +2203,21 @@ A blanket implementation is one for a bare type variable. It applies to every ty (weight "hello") ;; 5 }| +@subsubsection{Where an implementation is written} + +An implementation is written in the module that declares its trait, or in the one that declares its type. Every module that can name both the trait and the type then sees the implementation, so a call at a type means the same in every module. A module may implement its own traits for any type, and any trait for its own types: + +@codeblock[#:lang "bjolang"]|{ +(impl (Describe (List int)) ...) ;; fine: Describe is declared here +(impl (Displayed Card) ...) ;; fine: Card is declared here +(impl (Displayed (Tuple int string)) ;; refused: neither is declared here + ...) +}| + +A tuple, a type of the runtime such as @code{List}, and a .NET type are declared by no module, so only the module of a trait may implement it for one of them. To give such a type a trait from somewhere else, wrap it in a newtype of your own, as described below, and implement the trait for that. @code{#:via} gives the newtype the implementations the wrapped type already has. + +Traits, types and implementations may be used above where they are written. A function may name a type that is declared further down, and call a method of a trait that is declared below it, and an implementation may stand above its trait. + @subsubsection{Traits that require other traits} A trait may say that every type implementing it implements another trait too, with a @code{where} in the trait itself. Something that greets must have a name: @@ -2272,7 +2287,7 @@ Four traits from the prelude are worth knowing by heart, since the rest of the l (hash-combine (eq-hash (record-ref c rank)) (eq-hash (record-ref c suit))))) }| -The compiler writes a @code{Written} for every type you declare, so a value prints as you would build it: @code{(Card (rank Queen) (suit Hearts))}, @code{(Pip 7)}, @code{Hearts}. A generic type is written when its type arguments are. An @code{#:opaque} type is written as @code{#}, and a field that holds a function as @code{#}. @code{Displayed} has a blanket implementation that displays a value as it is written. +The compiler writes a @code{Written} for every type you declare, so a value prints as you would build it: @code{(Card (rank Queen) (suit Hearts))}, @code{(Pip 7)}, @code{Hearts}. A generic type is written when its type arguments are. An @code{#:opaque} type is written as @code{#}, and a function as @code{#}, wherever it is: @code{(println (Some (fun (x) (+ x 1))))} prints @code{(Some #)}, and so does a field that holds one, or a function at the REPL's prompt. Every function type has the head @code{Procedure}, so a trait of your own can have an implementation for functions, @code{(impl (Describe Procedure) ...)}, which applies to all of them. A blanket does not reach them, which is why @code{(= f g)} on two functions is still refused. @code{Displayed} has a blanket implementation that displays a value as it is written. To print a type your own way, implement @code{Displayed} for the text a person reads, as the card game did, or @code{Written} to change the written form too. The method to give is @code{display-into} (or @code{write-into}), which adds the text to a string builder; @code{displayed} and @code{written} then come with the trait: @@ -2305,6 +2320,8 @@ A list holds values of one type. When you need a collection of different types t ;; '("1" "two" "Qโ™ฅ") }| +A @code{(dyn T)} implements the traits that @code{T} requires as well, and the ones those require, through the value it holds. With @code{Greet} and @code{Named} from above, a @code{(dyn Greet)} can be given to @code{introduce}, which calls @code{name-of}, and to any function that asks for @code{(where (Named %a))}. So a trait cannot be packed if a trait it requires cannot be packed, or has associated types. + Packing is always explicit, and a @code{dyn} cannot be turned back into the value it came from. It is the exception rather than the rule: most code is better served by a union. @subsubsection{Traits that are .NET interfaces} diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index ae452b5..f4d5701 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -932,8 +932,11 @@ The messages for the common mistakes, and what to write instead: | Block: ... found a boolean, or found a bool | @code{(if b "yes" "no")} | | Block: ... found a (Vec string) | @code{(vec-map BStr xs)}, or @code{(: xs Block)} above @code{(def xs ...)} | | Block: ... found a (Vec (Vec ...)), at a lambda that returns a vec | @code{(each [...])} around the vec the lambda returns | +| these types do not match, naming two of int, string and char, at a lambda that returns a vec | @code{(each [...])} around the vec the lambda returns | } +The last two are one mistake, a lambda given to a function that does not say it returns a block. Its vec is a vec of strings if all its pieces are strings, and then @code{show} gets a vec of vecs. If the pieces are of different kinds, such as a name and a count, the vec has no type at all, and the compiler says so inside the lambda. + @subsubsection{A block of your own} @code{captured} renders content to a string and gives it to a function, which returns the block to write in its place. It is how a block that needs the whole of its content is made, and how padding and trimming are written: From 684745d6c028653ccadad4873455f14306a3f8ab840ab00d0a19399974986eff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 07:59:36 +0200 Subject: [PATCH 17/38] Pages and tour entries for the modules that had none MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reference pages for (std uuid), (std scan), (std priority-queue), (std crypto hash) and (text encodings), and entries for them in the tour of the standard library, with (text xml), (text json-codec) and (text bjodat-core). The text/json entry links to its page instead of prelude.org. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _03.Standard-library.sz | 42 ++++++++++++++++++++++++++++++----- modules/std/crypto/hash.sz | 3 +++ modules/std/crypto/index.sz | 2 ++ modules/std/priority-queue.sz | 3 +++ modules/std/scan.sz | 3 +++ modules/std/uuid.sz | 3 +++ modules/text/encodings.sz | 3 +++ 7 files changed, 53 insertions(+), 6 deletions(-) create mode 100644 modules/std/crypto/hash.sz create mode 100644 modules/std/crypto/index.sz create mode 100644 modules/std/priority-queue.sz create mode 100644 modules/std/scan.sz create mode 100644 modules/std/uuid.sz create mode 100644 modules/text/encodings.sz diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 811a032..03f0c6a 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -19,7 +19,7 @@ For more details, see @link["modules/std/array.html"]{@code{(std array)}} and @l @subsection{std/fmt} -Formatting in two layers: interpolation and layout. @code{println*} prints strings, ints, doubles, chars and bools written in one vec. The layout layer is made of blocks: a @code{Block} is one piece of output, and every combinator takes its content as a @code{(Vec Block)}, a vec literal such as @code{["x = " 7 nl]}, and gives back one @code{Block}. With them you pad, cut and align text, wrap and justify paragraphs, format numbers, and lay out tables with borders, a header, and columns that fit the width. The section on formatting text in "Writing programs" is a guide, with what goes in the brackets and what to write when it does not compile. +Formatting in two layers: interpolation and layout. @code{println*} prints strings, ints, doubles, chars and bools written in one vec. The layout layer is made of blocks: a @code{Block} is one piece of output, and every combinator takes its content as a block and gives back a block. A string, an int and a character are blocks, and so is a vec of blocks, such as @code{["x = " 7 nl]}. With them you pad, cut and align text, wrap and justify paragraphs, format numbers, and lay out tables with borders, a header, and columns that fit the width. The section on formatting text in "Writing programs" is a guide, with what goes in the brackets and what to write when it does not compile. For more details, see @link["modules/std/fmt.html"]{@code{(std fmt)}}. @subsection{std/rx} @@ -27,6 +27,11 @@ For more details, see @link["modules/std/fmt.html"]{@code{(std fmt)}}. Bjolang provides a built-in regular expression engine. Unlike languages that use strings for regex (like @code{"[a-z]+"}), Bjolang uses structural S-expressions. This means you never have to double-escape characters, and the compiler validates your patterns at compile time. Built-in character sets and anchors are written as keywords (e.g., @code{:bos}, @code{:digit}), and operations that group or repeat patterns are function-like lists (e.g., @code{(seq ...)}). For more details, see @link["modules/std/rx.html"]{@code{(std rx)}}. +@subsection{std/scan} + +Reading a line by a pattern, after OCaml's @code{Scanf}. The pattern is a quoted list of what the line holds, in order: @code{(scan "3-17" '(long "-" long))} is 3 and 17, and @code{'(int "x" int ": " int ... #:sep " ")} reads @code{"12x5: 1 0 3"} as two ints and a vec of them. There are items for numbers, a character, a word, the rest of the line, a run of given characters and the text up to one, with a width, a skip, a repetition, an option and a choice. The type of the result comes from the pattern, so @code{'(int "," int)} gives a @code{(Tuple int int)}. The macro compiles the pattern into straight code over the line's bytes, and a pattern of numbers and strings allocates nothing. The whole line must match. @code{try-scan} gives an @code{Err} that says at which column it failed, instead of raising. A scan does not search and does not backtrack: use @code{(std rx)} to find text in a line. +For more details, see @link["modules/std/scan.html"]{@code{(std scan)}}. + @subsection{std/http} The @code{std/http} module allows you to build and send HTTP requests. Sending a request three ways โ€” blocking, suspending, or as an event โ€” is the only thing that differs between the entry points. There is no C# in this module: @code{System.Net.Http} is reached with @code{import/extern}, and the response body is an ordinary @code{TextInputPort}. @@ -47,6 +52,16 @@ For more details, see @link["modules/std/run.html"]{@code{(std run)}}. The random module provides a ChaCha20 generator, seeded with 256 bits from the OS CSPRNG, with one generator per thread. The default path locks nothing and shares nothing, making it safe and fast on a bjoroutine. The ambient source has no state to race on, so drawing from multiple threads is efficient. You can generate random integers and more. For more details, see @link["modules/std/random.html"]{@code{(std random)}}. +@subsection{std/uuid} + +Uuids as RFC 9562 specifies them. @code{uuid-v4} is random, for an identifier that must tell nothing. @code{uuid-v7} starts with the time, for a database key: new keys go at the end of an index, and keys sort by when they were made, and the ones made by one process always increase. @code{uuid-v5} is made from a namespace and a name, so the same input gives the same uuid. A @code{Uuid} is 16 bytes and compares as its bytes do, which is also the order of its text and of Postgres' @code{uuid}. @code{uuid->string} writes the one text form, and @code{string->uuid} reads it in either case. The random bits come from @code{current-random} and the time from @code{current-clock}, so a test can make the same uuids in every run. +For more details, see @link["modules/std/uuid.html"]{@code{(std uuid)}}. + +@subsection{std/crypto/hash} + +Hashes, HMAC and password hashing, from .NET's @code{System.Security.Cryptography}. @code{hash-bytes}, @code{hash-string} and @code{hash-file} make a digest, SHA-256 unless @code{#:algorithm} says otherwise, and @code{make-hasher} makes one a piece at a time. @code{hmac-bytes} and @code{hmac-string} sign a message with a key. A digest is fast, so it is not a password hash: @code{password-hash} uses PBKDF2 with a random salt and gives a PHC string, which holds the algorithm and the iteration count, so @code{password-verify} needs only the password and the string, and @code{password-needs-rehash?} says which to replace after the count is raised. A password hash takes a few hundred milliseconds, and in a bjoroutine it runs on a thread of its own. Compare secrets with @code{constant-time-equal?}, not @code{=}. +For more details, see @link["modules/std/crypto/hash.html"]{@code{(std crypto hash)}}. + @subsection{std/atomic} An atomic is a cell that threads can read and write without a lock. @code{(make-atomic v)} makes one of any type, and @code{atomic-ref}, @code{atomic-set!}, @code{atomic-swap!}, @code{atomic-compare-and-swap!} and @code{atomic-update!} work on it. For @code{int}, @code{uint}, @code{long} and @code{ulong} there are also @code{atomic-increment!}, @code{atomic-decrement!}, @code{atomic-add!} and @code{atomic-sub!}, which compile to direct calls to @code{System.Threading.Interlocked} and wrap on overflow. Swap and compare-and-swap return the value from before the call; the other operations that change the atomic return the new value. The width of a counter comes from type inference, so write @code{(make-atomic 0L)} for a long counter: @code{(make-atomic 0)} is an int. @@ -82,6 +97,11 @@ For more details, see @link["modules/std/effect.html"]{@code{(std effect)}}. Persistent collections beside the prelude's @code{Map}: an update returns a new collection and does not change the old one. @code{(std hashset)} has the unordered @code{HashSet}, a CHAMP trie that compares elements with @code{Eq}. @code{(std orderedset)} and @code{(std orderedmap)} have @code{OrderedSet} and @code{OrderedMap}, B-trees kept in the order of @code{compare}. They add what the order gives: @code{-min}, @code{-max}, @code{-successor}, @code{-predecessor} and the range walks @code{-range}, @code{-from} and @code{-until}. Each collection has a builder, which makes it in one pass, and a transient, which edits it in place and gives it back as a persistent collection in O(1). For more details, see @link["modules/std/hashset.html"]{@code{(std hashset)}}, @link["modules/std/orderedset.html"]{@code{(std orderedset)}} and @link["modules/std/orderedmap.html"]{@code{(std orderedmap)}}. +@subsection{std/priority-queue} + +A persistent priority queue, least priority first. A @code{(PriorityQueue a p)} holds elements under priorities, and orders the priorities, so the elements need no order of their own. @code{priorityqueue-empty} orders priorities by their @code{Ord}, and @code{priorityqueue-by} by a comparison you give it. Adding or removing gives a new queue and leaves the old one as it was. It is a pairing heap: peeking, adding and @code{priorityqueue-merge} are O(1), and @code{priorityqueue-remove-min} is O(log n) amortised. That bound assumes each queue is used once. Call @code{priorityqueue-balance} on a queue you keep and remove from many times. Entries of equal priority come out in the order they went in. There is a transient and a builder, as for the other persistent collections. For a queue that is used in one place and never shared, @code{(std mutable heap)} is faster. +For more details, see @link["modules/std/priority-queue.html"]{@code{(std priority-queue)}}. + @subsection{std/mutable} Collections that change in place. Use them for a cache, an index, a worklist or a scratch buffer, where you do not need old versions. @code{(std mutable vec)}, @code{(std mutable map)}, @code{(std mutable set)} and @code{(std mutable heap)} are the .NET @code{List}, @code{Dictionary}, @code{HashSet} and @code{PriorityQueue} themselves, not wrappers, so .NET code that takes those types accepts them as they are. They compare with .NET's equality and order. @code{(std mutable deque)} is a double-ended queue with O(1) amortized push and pop at both ends. None of them has a lock. @@ -125,10 +145,20 @@ For more details, see @link["modules/std/syntax-match.html"]{@code{(std syntax-m @subsection{text/bjodat} -bjodat is Bjolang's shapes read as data: no evaluation, no macro expansion, no environment. It is what a package manifest, a lockfile, or a config file is written in. You can parse text straight into a record using @code{bjodat-parse-}, which avoids intermediate trees and allocations. -For more details, see @link["modules/text/bjodat.html"]{@code{(text bjodat)}}. +bjodat is Bjolang's shapes read as data: no evaluation, no macro expansion, no environment. It is what a package manifest, a lockfile, or a config file is written in. You can parse text straight into a record using @code{bjodat-parse-}, which avoids intermediate trees and allocations. @code{(text bjodat-core)} is the machinery underneath: the @code{Bjodat} tree, with @code{parse-bjodat}, @code{bjodat-ref} and the writer, for data whose shape is not known until the program runs. +For more details, see @link["modules/text/bjodat.html"]{@code{(text bjodat)}} and @link["modules/text/bjodat-core.html"]{@code{(text bjodat-core)}}. -@subsection{text/json} +@subsection{text/json and text/json-codec} -The @code{text/json} module provides JSON parsing and generation capabilities. Like @code{bjodat}, it allows for working with structured data but using the standard JSON format, which is essential for web communication. -For more details, see @code{prelude.org} and related standard library documentation. +@code{(text json)} reads JSON, exactly as RFC 8259 has it, into a @code{Json}: a union with a case for each kind of value. @code{parse-json} reads a string and @code{read-json} a port. A document that is not JSON is an @code{Err} that says the line and column where it went wrong. A number without a fraction or an exponent is a @code{JsonInt}, and an object is a @code{(Map string Json)}, so the order of its keys is not kept. @code{(text json-codec)} reads a record straight from the tree and back: @code{def/json-type} declares a record and the instances of @code{->json} and @code{json->} for it, with defaults, optional fields and other names for the keys. An @code{Option} is @code{null} when it is @code{None}. There is no writer yet, so a @code{Json} cannot be written out as text. +For more details, see @link["modules/text/json.html"]{@code{(text json)}} and @link["modules/text/json-codec.html"]{@code{(text json-codec)}}. + +@subsection{text/xml} + +XML, in three modules over @code{System.Xml}. @code{(text xml)} is the tree: names, elements, texts, comments and processing instructions, built by @code{xml-element} and the others, and taken apart by views. @code{(text xml read)} reads XML in four layers: a stream of signals, a fold after SSAX, a tree of your own type, and @code{parse-xml}, which gives a @code{Document}. @code{xml-for-each} reads a large document one element at a time. The defaults are the safe ones: a document with a DOCTYPE is refused unless asked for, and nothing external is ever resolved. @code{(text xml write)} writes a node or a document to a port or a string. It chooses the namespace prefixes, and content that XML cannot hold is an @code{XmlWriteError} rather than an exception. +For more details, see @link["modules/text/xml.html"]{@code{(text xml)}}, @link["modules/text/xml/read.html"]{@code{(text xml read)}} and @link["modules/text/xml/write.html"]{@code{(text xml write)}}. + +@subsection{text/encodings} + +Every character encoding .NET knows, as a value of the prelude's @code{Encoding}, to give to @code{string->bytes} and @code{bytes->string}, or to @code{reencode-input-port} and @code{reencode-output-port} from @code{(std ports)} for a port. There is one for each of the 139, named in lower case as .NET names them: @code{utf-16be}, @code{iso-8859-1}, @code{windows-1252}, @code{shift_jis}, @code{koi8-r} and the rest. .NET's little-endian @code{utf-16} and @code{utf-32} are @code{utf-16le} and @code{utf-32le} here. @code{encoding-named} finds one by name and @code{encoding-of-code-page} by number, both as an @code{Option}. No encoding writes or skips a byte order mark. Each encoding is made the first time it is used, so importing the module costs nothing. +For more details, see @link["modules/text/encodings.html"]{@code{(text encodings)}}. diff --git a/modules/std/crypto/hash.sz b/modules/std/crypto/hash.sz new file mode 100644 index 0000000..f4bbfcd --- /dev/null +++ b/modules/std/crypto/hash.sz @@ -0,0 +1,3 @@ +(meta (title "(std crypto hash)")) + +@generate-docs[#:module "std crypto hash"] diff --git a/modules/std/crypto/index.sz b/modules/std/crypto/index.sz new file mode 100644 index 0000000..a1e49d9 --- /dev/null +++ b/modules/std/crypto/index.sz @@ -0,0 +1,2 @@ +(meta (title "(std crypto)") + (sort title)) diff --git a/modules/std/priority-queue.sz b/modules/std/priority-queue.sz new file mode 100644 index 0000000..8c7b0ba --- /dev/null +++ b/modules/std/priority-queue.sz @@ -0,0 +1,3 @@ +(meta (title "(std priority-queue)")) + +@generate-docs[#:module "std priority-queue"] diff --git a/modules/std/scan.sz b/modules/std/scan.sz new file mode 100644 index 0000000..17bc420 --- /dev/null +++ b/modules/std/scan.sz @@ -0,0 +1,3 @@ +(meta (title "(std scan)")) + +@generate-docs[#:module "std scan"] diff --git a/modules/std/uuid.sz b/modules/std/uuid.sz new file mode 100644 index 0000000..4fa8678 --- /dev/null +++ b/modules/std/uuid.sz @@ -0,0 +1,3 @@ +(meta (title "(std uuid)")) + +@generate-docs[#:module "std uuid"] diff --git a/modules/text/encodings.sz b/modules/text/encodings.sz new file mode 100644 index 0000000..686306e --- /dev/null +++ b/modules/text/encodings.sz @@ -0,0 +1,3 @@ +(meta (title "(text encodings)")) + +@generate-docs[#:module "text encodings"] From 1a407720a9eca7f11ae396fd444c0134b7ee874fa685e63e2808800286117f79 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 11:27:57 +0200 Subject: [PATCH 18/38] Unions: two may have a case of one name; which name wins, as it is MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A case of one name in two unions is chosen by type, then by scope, and the manual says so in place of "cannot both have a case called Red". Writing programs said two imports offering one name, a type among them, resolved to the later import; a use is refused until the program says which, except for cases of different unions. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 24 +++++++++++++++++++++++- _02_.Writing-programs.sz | 4 ++-- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 2be4d4b..31942df 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -453,10 +453,32 @@ Unions can be generic. The type variables go after the type name, just like for You never have to write these two, since the prelude already has them under the names @code{(Option %a)}, with the cases @code{None} and @code{Some}, and @code{(Result %e %a)}, with the cases @code{Err} and @code{Ok}. They are ordinary unions, and they get a section of their own further down. +Two unions can each have a case of the same name, in one module or in two. Which one a use of the name means is decided by its type: the type wanted where it is written, the type of the value a @code{match} takes apart, or the number of values it is given, when only one of the cases takes that many. + +@codeblock[#:lang "bjolang"]|{ +(type (: Light (Union Red Amber Green))) +(type (: Colour (Union Red Blue (: Grey int)))) + +(: stop (-> Light)) +(defun (stop) Red) ;; a Light, because stop returns one + +(: colour-name (-> Colour string)) +(defun (colour-name c) + (match c + (Red "red") ;; Colour's, because c is a Colour + (Blue "blue") + ((Grey n) "grey"))) + +(list Red Amber) ;; Lights, because Amber is +(println (: Red Colour)) ;; written out +}| + +When nothing in the definition says which union is meant, a case your module declares wins over an imported one, and an imported one over the standard library's. If that does not decide it either, as for two @code{Red}s of your own in @code{(println Red)}, the compiler asks you to write the type: @code{(: Red Light)}. + A few things to know about unions: @read-list{ -- Case names share a namespace with everything else in the module, so two unions in the same module cannot both have a case called @code{Red}. +- A case cannot have the name of a record or a newtype of the same module, because those are built by their names too. - A union case cannot have a @code{#:mutable} field. - @code{println} on a value of your own type prints it as you would build it, for example @code{(Card (rank Queen) (suit Hearts))} or @code{(Pip 7)}. The compiler writes this for each type you declare. To print a type another way, implement @code{Displayed} (see the traits section). } diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index f4d5701..10ed9a8 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -195,7 +195,7 @@ Note that @code{only} and @code{except} apply exclusively to functions and macro 0) }| -Types are strictly scoped to the module that declared them. If two imported libraries each declare their own @code{Card} type, they remain distinct types; the unqualified name @code{Card} simply resolves to whichever import came last. To disambiguate between them, use @code{prefix-types}: +Types are strictly scoped to the module that declared them. If two imported libraries each declare their own @code{Card} type, they remain distinct types, and writing the unqualified name @code{Card} is an error until you say which one you mean. To tell them apart, use @code{prefix-types}: @codeblock[#:lang "bjolang"]|{ (import (prefix-types "poker.bjo" "P/") @@ -210,7 +210,7 @@ When multiple definitions share the same identifier, name resolution follows thr @read-list{ - Local definitions take precedence over imported names. -- For collisions between two imports, the later import shadows the earlier one. This shadowing behavior is how modules can freely override names from the prelude, which is loaded as the very first import. +- A name the prelude offers is overridden by any other import's, which is how a module can freely replace a prelude name. Between two other imports that offer one name as two different things, importing both is fine, but writing the name is an error until you say which one you mean, with @code{only}, @code{except} or a prefix. Two cases of different unions are the exception: which one a use means is decided by its type, as described for unions in Part 1. - Explicit renames or aliases created by import modifiers or @code{:alias} must be unique. If an alias collides with an existing name, the compiler raises an error to prevent ambiguity. } From ae58a1e0041d9982a5d3994135770c98d94f26d7378a621e03dfd5c36f9da5b8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 11:47:12 +0200 Subject: [PATCH 19/38] Newtypes: phantom type parameters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One Id with a tag for what it is an id of, in place of a newtype per kind of id, and a tag that records what has been checked about a value, kept honest by #:opaque. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 31942df..abc5ecd 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -375,6 +375,44 @@ A newtype is the opposite of an alias: a type of its own, which the compiler wil A newtype does not get the traits of the type it wraps: @code{(+ u u)} is an error, though @code{(+ 7 7)} is not. It gets those you give it, as the section on traits shows. +A newtype can have a type parameter that its value does not use. Such a parameter is called a @em{phantom}: it is only there to tell values apart. Instead of one newtype for each kind of id, there can be one @code{Id} that says what it is an id of. The tags are types with no values, written as unions with no cases: + +@codeblock[#:lang "bjolang"]|{ +(type (: User (Union))) ;; tags: types with no values +(type (: Order (Union))) +(type (: (Id %a) (Newtype int))) ;; %a is in no field + +(: lookup-user (-> (Id User) string)) +(defun (lookup-user id) ...) + +(: id-number (-> (Id %a) int)) ;; for an id of any kind +(defun (id-number id) + (def (Id n) id) + n) + +(lookup-user (: (Id 7) (Id Order))) ;; a compile error: an (Id Order) is not an (Id User) +}| + +An @code{(Id User)} is still an @code{int} when the program runs. A function can be written once for every kind of id, as @code{id-number} is, and a trait can have an implementation for each tag, such as one for @code{(Id User)} and another for @code{(Id Order)}. + +A tag can also record what has been checked about a value. Here only @code{validate} can make an @code{(Email Valid)}, so @code{send} never gets an address nobody checked: + +@codeblock[#:lang "bjolang"]|{ +(type (: Raw (Union))) +(type (: Valid (Union))) +(type (: (Email %s) #:opaque (Newtype string))) + +(: validate (-> (Email Raw) (Option (Email Valid)))) +(defun (validate e) + (def (Email s) e) + (if (string-contains? s "@") (Some (Email s)) None)) + +(: send (-> (Email Valid) void)) +(defun (send e) ...) +}| + +@code{#:opaque}, described under exporting types in "Writing programs", is what keeps another module from building an @code{(Email Valid)} itself, with @code{(Email s)}. + @subsubsection{Records and structs} Records and structs are direct mappings to c# records and immutable structs. The constructor is the type name, together with parenthesised fields. As follows. From 2a4e3b3fcf4ffe5d3306df9bef363365eace325c9030590c496a322b84d1dffe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 12:37:40 +0200 Subject: [PATCH 20/38] Typed holes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A subsection under Values, types and signatures: _ where the code is not written yet, what the compiler says about it, and a _name as a named hole. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index abc5ecd..f5d2fb7 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -164,6 +164,27 @@ Everything at the toplevel needs a signature, even values. In a function body, n The example above is of course facile, but it matters more for generic functions with inlined traits. They are pinned to the first use, and any subsequent usage will raise an error. More on this in the trait section. +@subsubsection{Typed holes} + +When you don't know what to write somewhere, write @code{_} there. That is a typed hole (not to be confused with the holes of string interpolation). A program with a typed hole does not compile. Instead the compiler tells you what type is wanted at that spot, and which bindings in scope have that type: + +@codeblock[#:lang "bjolang"]{ +(: lengths (-> (Vec string) (Vec int))) +(defun (lengths names) + (vec-map _ names)) +} + +@codeblock[#:lang "text"]|{ +Hole at lengths.bjo:3:12: '_' stands for a value of type (-?-> string int). + Bindings in scope that fit: string-count, unhashable, count, eq-hash, length, structural-hash, and 2 more. +}| + +@code{-?->} is a function that may or may not suspend, which is what @code{vec-map} accepts. More on that under Concurrency. + +The bindings closest to the hole are listed first: locals, then your own module's, then imported ones. A binding with a where clause is only listed if the clause holds. A name that starts with an underscore, like @code{_total}, is a hole too as long as nothing binds it, which is handy when you have several. If nothing around the hole decides its type, you get a @code{?a} and what it has to be able to do, like @code{(Displayed ?a)}. In the REPL, @code{(: _ string)} is a quick way to ask what in scope is a string. + +While a hole's type is still open, the other errors in the same definition are held back, since they are often caused by the hole. Once you fill it in, they show up. + @subsubsection{Naming conventions} Bjolang follows scheme's naming conventiens. Predicates end with @code{?}. Mutating functions end with @code{!}. Conversions are written using @code{->}. From c791af1593439b3050d34f625b2175c8a6d7c129b3ab252433be95d5bf56c1f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 15:06:15 +0200 Subject: [PATCH 21/38] Unused definitions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A subsection after typed holes: the compiler warns about a definition nothing uses, and a leading _ keeps one on purpose. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index f5d2fb7..532f789 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -185,6 +185,12 @@ The bindings closest to the hole are listed first: locals, then your own module' While a hole's type is still open, the other errors in the same definition are held back, since they are often caused by the hole. Once you fill it in, they show up. +@subsubsection{Unused definitions} + +The compiler warns about definitions nothing uses: a top-level @code{def} or @code{defun} that the module doesn't export and nothing in it reads, and a local binding (a @code{let}, a @code{def} in a body, a loop variable) that nothing in its scope reads. Dead code piles up quietly in a large program, and this keeps it from doing so. + +If you keep something on purpose, start its name with an underscore: @code{(def _registered (register!))}, or @code{(:for _ (range 0 n))} for a loop that just repeats. Parameters and pattern variables are not checked. + @subsubsection{Naming conventions} Bjolang follows scheme's naming conventiens. Predicates end with @code{?}. Mutating functions end with @code{!}. Conversions are written using @code{->}. From 3173c7176f2886659fd9f810f88b5122684e7e61e93db27f08add84e05bd5e7e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 15:36:57 +0200 Subject: [PATCH 22/38] Modules only the package may use MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A directory whose name starts with _ keeps its modules to the directory it is in, and to the project's tests/ and benches/. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 10ed9a8..8f1b896 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -313,6 +313,20 @@ Because @code{bjo} automatically detects the project root by walking upward unti In @code{bjo run . one two}, the dot specifies the project itself. Any arguments following the target are forwarded directly to the program's @code{main} function, binding @code{args} to @code{[one two]}. You can also point @code{bjo run} at a specific test file, such as @code{bjo run tests/deck-test.bjo}; the file will be compiled and linked against the project's declared packages. +@subsubsection{Modules only the package may use} + +An export list says what a module offers, but once a module is exported, any module can import it. As a package grows you want helpers that the package's own modules share without them becoming part of what the package offers to others. Put them in a directory whose name starts with an underscore: + +@codeblock[#:lang "text"]|{ +cards/ + src/ + _lib/shuffle.bjo (cards _lib shuffle) + deck.bjo may import (cards _lib shuffle) + tests/shuffle-test.bjo may import it too +}| + +A module under @code{src/_lib/} may be imported by the modules under @code{src/}, and by the project's own @code{tests/} and @code{benches/}. Any other package that tries is told the module is internal to @code{(cards)}, and should use a module outside @code{_lib/} instead. The rule works for any directory starting with @code{_}, at any depth: a module in @code{src/game/_rules/} is for the modules under @code{src/game/}. The REPL may import anything. + @subsubsection{Libraries} A package that lacks an executable entry point (@code{src/main.bjo}) is treated as a library. Running @code{bjo init --lib} initializes a library project, and running @code{bjo build} inside it compiles all modules found under @code{src/}. From 1e7c54fcc695b7aba6f33d5c1a15e082450ec6dac212489f607b30ea52319269 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Thu, 22 Oct 2026 20:35:00 +0200 Subject: [PATCH 23/38] Code actions for typed holes and incomplete matches MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The language server can fill a hole with what fits it and add the clauses a match lacks; the sections on typed holes and exhaustiveness say so. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 532f789..e71b7b8 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -185,6 +185,8 @@ The bindings closest to the hole are listed first: locals, then your own module' While a hole's type is still open, the other errors in the same definition are held back, since they are often caused by the hole. Once you fill it in, they show up. +In an editor that uses the Bjolang language server, a hole has code actions: fill it with one of the bindings that fit, with a constructor of its type such as @code{(Circle _)}, or, where a function is wanted, with @code{(fun (x y) _)}. What you fill in can have holes of its own, so you can build an expression a step at a time and let the compiler tell you what each part has to be. + @subsubsection{Unused definitions} The compiler warns about definitions nothing uses: a top-level @code{def} or @code{defun} that the module doesn't export and nothing in it reads, and a local binding (a @code{let}, a @code{def} in a body, a loop variable) that nothing in its scope reads. Dead code piles up quietly in a large program, and this keeps it from doing so. @@ -977,6 +979,8 @@ The counterexample is a real value, and for nested patterns it tells you exactly Pattern Error at cards.bjo:8: this match does not cover every value. (Card (rank Ace) (suit Clubs)) reaches no clause. }| +In an editor that uses the Bjolang language server, an incomplete match has a code action that adds the clauses it lacks, nested cases spelled out, each with a typed hole as its body: @code{((Square _) _)}. + A clause that can never run, because the clauses above it already match everything it would, is also an error: @codeblock[#:lang "text"]|{ From 0e6ff360760bea61f0f09915a6c3ffd9cf3610ca05af4a7c6e8de0c5011c4b72 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Fri, 23 Oct 2026 06:24:26 +0200 Subject: [PATCH 24/38] bjo fmt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The project commands table has bjo fmt, and a paragraph says what it does: indent as bjo-mode does, closing brackets up Lisp style, the whitespace tidied, --check for CI, - for an editor, and the lines it makes wider than 100 columns named. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 3 +++ 1 file changed, 3 insertions(+) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 8f1b896..2c40622 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -308,11 +308,14 @@ Because @code{bjo} automatically detects the project root by walking upward unti | @code{bjo build} | Fetches dependencies and compiles the program or library | | @code{bjo run . args ...} | Builds and runs the project with the specified arguments | | @code{bjo check} | Type-checks the entire project without emitting output binaries | +| @code{bjo fmt} | Lays out the project's source the way @code{bjo-mode} does | | @code{bjo repl} | Starts the REPL preloaded with access to the project's modules | } In @code{bjo run . one two}, the dot specifies the project itself. Any arguments following the target are forwarded directly to the program's @code{main} function, binding @code{args} to @code{[one two]}. You can also point @code{bjo run} at a specific test file, such as @code{bjo run tests/deck-test.bjo}; the file will be compiled and linked against the project's declared packages. +@code{bjo fmt} indents every line the way @code{indent-region} does in Emacs's @code{bjo-mode}, so the two always agree. It also moves a line of closing brackets up onto the line before it, Lisp style, removes trailing whitespace, and turns three or more blank lines into two. Line breaks are otherwise kept, and the lines of a string are never touched. Since it only moves text to the right, it names every line it makes wider than 100 columns, so that you can break it by hand. With no arguments it lays out every file under @code{src/}, @code{tests/} and @code{benches/}; you can also name files or directories. @code{bjo fmt --check} changes nothing, names the files that are not laid out, and fails if there are any, which is what a CI job wants. @code{bjo fmt -} reads standard input and writes the result to standard output, for an editor. A file whose brackets do not pair up, or that has a string that does not end, is refused with the line where it goes wrong. + @subsubsection{Modules only the package may use} An export list says what a module offers, but once a module is exported, any module can import it. As a package grows you want helpers that the package's own modules share without them becoming part of what the package offers to others. Put them in a directory whose name starts with an underscore: From 4d29d7b7b6a2bbda78acb669905a9b46bef88cfac3daac2f769482760a58711d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Fri, 23 Oct 2026 11:59:09 +0200 Subject: [PATCH 25/38] don't rememeber --- _02_.Writing-programs.sz | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 2c40622..ee04eb0 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -314,7 +314,7 @@ Because @code{bjo} automatically detects the project root by walking upward unti In @code{bjo run . one two}, the dot specifies the project itself. Any arguments following the target are forwarded directly to the program's @code{main} function, binding @code{args} to @code{[one two]}. You can also point @code{bjo run} at a specific test file, such as @code{bjo run tests/deck-test.bjo}; the file will be compiled and linked against the project's declared packages. -@code{bjo fmt} indents every line the way @code{indent-region} does in Emacs's @code{bjo-mode}, so the two always agree. It also moves a line of closing brackets up onto the line before it, Lisp style, removes trailing whitespace, and turns three or more blank lines into two. Line breaks are otherwise kept, and the lines of a string are never touched. Since it only moves text to the right, it names every line it makes wider than 100 columns, so that you can break it by hand. With no arguments it lays out every file under @code{src/}, @code{tests/} and @code{benches/}; you can also name files or directories. @code{bjo fmt --check} changes nothing, names the files that are not laid out, and fails if there are any, which is what a CI job wants. @code{bjo fmt -} reads standard input and writes the result to standard output, for an editor. A file whose brackets do not pair up, or that has a string that does not end, is refused with the line where it goes wrong. +@code{bjo fmt} indents every line the way @code{indent-region} does in Emacs's @code{bjo-mode}, so the two always agree. It also moves a line of closing brackets up onto the line before it, Lisp style, removes trailing whitespace, turns tabs into spaces (or @code{\t} in a string), and turns three or more blank lines into two. It adds a few line breaks: a @code{def}'s failure part (@code{:leave}, @code{:leave-with}, @code{:default}, @code{:propagate}) starts its own line, so does each form of a @code{cond}, @code{begin}, @code{seq} or @code{try} that spans lines, and so does each binding of a @code{let} whose bindings span lines. Line breaks are otherwise kept, and the lines of a string are never touched. Since it only moves text to the right, it names every line it makes wider than 102 columns, so that you can break it by hand. With no arguments it lays out every file under @code{src/}, @code{tests/} and @code{benches/}; you can also name files or directories. @code{bjo fmt --check} changes nothing, names the files that are not laid out, and fails if there are any, which is what a CI job wants. @code{bjo fmt --diff} changes nothing either, and prints what it would change as a unified diff, which @code{git apply -p0} or @code{patch -p0} takes. @code{bjo fmt -} reads standard input and writes the result to standard output, for an editor. A file whose brackets do not pair up, or that has a string that does not end, is refused with the line where it goes wrong. @subsubsection{Modules only the package may use} From 623b68513c59bd8456848a7435c90327461d21d527b61ac4613756fd14387fe0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Mon, 26 Oct 2026 09:27:53 +0100 Subject: [PATCH 26/38] When fibers hang: the deadline report and the fiber trace MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-opus-4-5) Co-Authored-By: eca-agent --- _02_.Writing-programs.sz | 93 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index ee04eb0..e3ec147 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -1199,6 +1199,99 @@ A scope also owns resources. @code{(own! thunk)} registers a release on the inne Do not park a bjoroutine's thread by accident. A scope in an ordinary @code{(fun ...)} that a bjoroutine calls parks the thread of that bjoroutine, and the compiler warns about it. Write the scope in the bjoroutine itself. Or, if the function the lambda is given to takes a @code{-?->} parameter, write the lambda as a @code{(bjoroutine ...)}. +@subsubsection{When fibers hang} + +A scope does not return until its fibers have ended, so a fiber that waits forever makes its scope wait forever too. Two things show you where the fibers wait. + +@strong{A deadline says where the others were.} When @code{with-deadline} stops the body, the @code{Cancelled} that the body raises tells where every other fiber of the scope was waiting. Fibers in nested scopes are counted too: + +@codeblock[#:lang "bjolang"]|{ +(: worker (-bjo-> (Chan int) void)) +(defbjo (worker jobs) + (let go () + (ignore (sync (chan-recv jobs))) + (go))) + +(defbjo (main) + (def jobs (make-chan)) + (match (try (with-deadline (milliseconds 100) + (spawn (worker jobs)) + (spawn (worker jobs)) + (sync (timeout (minutes 1)))) + #:catch (System.Exception)) + ((Ok v) (println "done")) + ((Err e) (println (.-Message e)))) + 0) +}| + +@codeblock[#:lang "text"]|{ +cancelled: (Deadline) +When the deadline passed, 2 other fibers were waiting: + 2 fibers at app.bjo:4 (ignore (sync (chan-recv jobs))) +}| + +This is always on and costs nothing until a deadline stops a fiber. A deadline that stops only children, after the body has returned, is not a failure, so then nothing is raised. + +@strong{The fiber trace.} Start the program with @code{BJO_FIBER_TRACE} set in the environment: + +@codeblock[#:lang "text"]|{ +BJO_FIBER_TRACE=1 bjo run +}| + +The program then keeps a record of every fiber that waits, and you get it as a report in three ways: + +@itemlist{ + @item{@code{(fiber-trace)} returns the report as a string.} + @item{SIGQUIT (Ctrl-\ in the terminal, or @code{kill -QUIT pid}) writes it to standard error. The program goes on.} + @item{When the program looks deadlocked, it writes the report once by itself: nothing has run for two seconds, and every fiber waits on a channel, on another fiber or on the close of a scope.} +} + +Here are two philosophers. Each takes one fork and then waits for the other one: + +@codeblock[#:lang "bjolang"]|{ +(: philosopher (-bjo-> (Chan int) (Chan int) void)) +(defbjo (philosopher left right) + (def l (sync (chan-recv left))) + (sync (timeout (milliseconds 50))) + (def r (sync (chan-recv right))) + (println "ate")) + +(: fork-keeper (-bjo-> (Chan int) void)) +(defbjo (fork-keeper fork) + (sync (chan-send fork 1))) + +(defbjo (main) + (def f1 (make-chan)) + (def f2 (make-chan)) + (with-scope + (spawn (fork-keeper f1)) + (spawn (fork-keeper f2)) + (spawn (philosopher f1 f2)) + (spawn (philosopher f2 f1))) + 0) +}| + +@codeblock[#:lang "text"]|{ +Bjolang fiber trace: nothing has run for 2 s, and every fiber waits on a channel or on another fiber. This looks like a deadlock. +Bjolang fiber trace: 3 fibers waiting. + +main + + scope, opened at app.bjo:15 (with-scope + 1 fiber closing this scope, waiting for its children, for 2.5 s + at app.bjo:15 (with-scope + 1 fiber waiting in chan-recv on channel #1, for 2.4 s + at app.bjo:5 (def r (sync (chan-recv right))) + 1 fiber waiting in chan-recv on channel #2, for 2.4 s + at app.bjo:5 (def r (sync (chan-recv right))) +}| + +Each fiber is under the scope it waits in, and the scopes are in a tree. The @code{at} line is the @code{sync}. For a fiber that waits inside a bjoroutine it called, @code{from} lines follow with the callers, innermost first. Fibers that wait at the same place on the same thing are counted together. Each channel has a number, so you can see which fibers wait on the same channel. + +The deadlock check reports only a program that cannot go on. A wait on a timer, a @code{choose}, an inbox or a .NET task can end without help from a fiber, so a program with such a wait is not reported. Use SIGQUIT to see where that program waits. + +With the trace off, the program runs the same code as without it: the JIT removes the tests. With the trace on, every wait costs more (a channel rendezvous takes about four times as long), so use it to find where a program is stuck, not in production. + @subsubsection{Work that is not a fiber} Performing long CPU-bound computations or calling legacy blocking .NET APIs directly inside a fiber can monopolize thread pool threads, preventing other fibers from running. To avoid blocking the pool, you can offload such operations onto dedicated background threads; both helpers return synchronization events: From d05f4c91dd3dc16eaa74fe8ba685e40a2da9cc32f695f731fd59380dd16a6874 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Mon, 26 Oct 2026 09:51:10 +0100 Subject: [PATCH 27/38] cleaned up appendix and 002 --- _02_.Writing-programs.sz | 67 ++++++++++++++++++---------------------- _04.Appendices.sz | 64 ++++++++++++-------------------------- 2 files changed, 49 insertions(+), 82 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index e3ec147..2853270 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -94,7 +94,7 @@ Export Error: Exported item 'helper' is missing a mandatory type signature at ba @subsubsection{The standard library, and the prelude} -A list-shaped module path such as @code{(std random)} refers to a standard library module. The compiler resolves these relative to the Bjolang installation rather than the current directory, so @code{(std random)} always points to the same library regardless of where you invoke @code{bjo}. A few notable modules include: +A list-shaped module path such as @code{(std random)} refers to a library module. The compiler resolves these relative to the Bjolang installation rather than the current directory, so @code{(std random)} always points to the same library regardless of where you invoke @code{bjo}. A few notable modules include: @read-table{ | Module | What it has | @@ -1477,9 +1477,9 @@ The tables in @code{BjolangRuntime/Cml/design.md} in the main repository have ev @subsection{Effects} -Our deck is shuffled using @code{shuffle-vec}, which is non-deterministic. While randomness is ideal during gameplay, it makes automated tests brittle when verifying deterministic outcomes like trick winners. The traditional object-oriented or functional workaround is dependency injectionโ€”passing an explicit shuffle function down through every layer of the call stack. +The deck is shuffled with @code{shuffle-vec}, so a hand dealt in a test is a different hand every run. A test for who wins a trick needs the same deck each time. The usual fix is to pass a shuffle function down through every function that deals, which puts a parameter in a lot of code that has nothing to do with shuffling. -Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. With effects, a function simply declares and performs an abstract operation, allowing an enclosing caller higher up the call stack to dynamically decide how that operation is handled. +Bjolang has *tail-resumptive algebraic effects* instead. A function declares an operation and performs it, and whoever is further up the call stack decides what it does. The handler runs once, and the value it returns is the value the operation returns. @subsubsection{Declaring an effect} @@ -1495,7 +1495,7 @@ Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. (vec-slice (shuffle-deck (full-deck)) 0 n)) }| -@code{shuffle-deck} is exposed as a regular function matching the declared signature. When called without an active handler in the dynamic scope, it falls back to the default implementationโ€”in this case, the real shuffle. In a test, you can override this behavior using @code{with-handler}: +@code{shuffle-deck} becomes an ordinary function with the signature it was declared with. Called with no handler in scope it runs the @code{#:default}, which here is the real shuffle. A test installs a handler with @code{with-handler}: @codeblock[#:lang "bjolang"]|{ (deal-hand 5) ;; five random cards @@ -1503,16 +1503,15 @@ Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. (deal-hand 5)) ;; [2โ™ฃ 3โ™ฃ 4โ™ฃ 5โ™ฃ 6โ™ฃ], every time }| -The handler intercepts operations executed anywhere within the dynamic extent of the @code{with-handler} block, including inside deeply nested helper calls and child fibers, without requiring @code{deal-hand} to pass any parameters around. +Every call to @code{shuffle-deck} inside the @code{with-handler} body runs the handler: in @code{deal-hand}, in helpers it calls, and in fibers it spawns. Nothing is passed down as a parameter. -Several key semantics govern effect handlers: +An effect declared without @code{#:default} and performed with no handler in scope raises an exception. Leave the default off for operations that only make sense with a handler installed. -@read-list{ -- Default implementations: If an effect omits @code{#:default} and is performed without an active handler, the runtime raises an exception. Omit defaults for operations that always require explicit environmental context. -- One-shot execution: Handlers in Bjolang behave like standard functions and return a single value. Full multi-shot delimited continuations are not supported: a handler cannot resume its caller multiple times or abort resumption except by throwing an exception. -- Effect coloring: Operations declared with @code{->} can be invoked from any standard function. Operations declared with @code{-bjo->} may only be invoked from within bjoroutines, and their handlers must likewise be bjoroutines. -- Handler composition: Invoking an operation from inside its own handler invokes that handler recursively. To delegate to an enclosing outer handler, retrieve it using @code{(handler-of op)}: -} +A handler is a function and returns one value. It cannot resume its caller twice, and the only ways out of it are to return or to raise. There are no multi-shot continuations. + +An operation declared with @code{->} can be performed from any function. One declared with @code{-bjo->} can only be performed from a bjoroutine, and its handler has to be a bjoroutine too. + +Performing an operation inside its own handler calls that handler again, so it recurses. To pass the work to the handler that was there before yours, ask for it with @code{(handler-of op)}: @codeblock[#:lang "bjolang"]|{ (with-handler ((log (let ((outer (handler-of log))) @@ -1522,7 +1521,7 @@ Several key semantics govern effect handlers: @subsubsection{The prelude's own effects} -Several core runtime facilities in the prelude are implemented as effects out of the box, allowing you to intercept or virtualize them in tests without modifying application code: +Part of the prelude already works this way, so a test can replace it without touching the code under test: @read-table{ | Operation | Default | @@ -1530,13 +1529,13 @@ Several core runtime facilities in the prelude are implemented as effects out of | @code{(log s)} | Writes a line to standard error | | @code{(warn s)} | Writes a warning line to standard error | | @code{(getenv name)} | Reads an environment variable as an @code{Option} | -| @code{(monotonic-ms)} | High-resolution monotonic clock for elapsed timing | -| The @code{FS} operations | Physical filesystem operations | +| @code{(monotonic-ms)} | A monotonic clock, in milliseconds | +| The @code{FS} operations | The real filesystem | } -Because all prelude file I/O routes through the built-in @code{FS} effect operations, tests can replace the entire filesystem with an in-memory mock using @code{with-fake-fs} (covered in the Testing section below). +All file I/O in the prelude goes through the @code{FS} operations, so a test can run against a fake filesystem with @code{with-fake-fs} instead of the disk. The testing section has the details. -Similarly, you can capture diagnostic output in memory instead of dumping it to the terminal by installing a custom handler for @code{log}: +Log output is easy to catch the same way. Install a handler for @code{log} and the lines end up wherever you put them: @codeblock[#:lang "bjolang"]|{ (def lines (make-box (list))) @@ -1546,15 +1545,15 @@ Similarly, you can capture diagnostic output in memory instead of dumping it to (box-ref lines) ;; '("two" "one") }| -Keep in mind that @code{with-handler} only establishes a dynamic binding; unlike a concurrency scope, it does not await background fibers. If code inside the handler spawns asynchronous fibers that emit logs, ensure the concurrency scope enclosing those fibers is placed *inside* the @code{with-handler} block so the handler remains active until all fibers finish writing. +A handler is a dynamic binding, not a scope. @code{with-handler} does not wait for fibers the body spawns. If the body spawns fibers that log, put the scope *inside* the handler, or the handler is gone and they write to stderr after all. @subsection{.NET interop} -Because Bjolang compiles directly to C#, you have full, seamless access to the entire .NET ecosystem. By declaring the CLR classes, methods, and properties you need along with their signatures, the compiler exposes them as ordinary Bjolang functions and types. +Bjolang compiles to C#, so the whole .NET library is within reach. You write the member you want and its signature, and the compiler turns it into an ordinary Bjolang function or type. @subsubsection{Methods: import/extern} -For example, we can render suit symbols in color in the terminal using .NET's @code{System.Console}: +Suit symbols in colour, with @code{System.Console}: @codeblock[#:lang "bjolang"]|{ (import "deck.bjo") @@ -1585,7 +1584,7 @@ For example, we can render suit symbols in color in the terminal using .NET's @c 0) }| -Each entry in @code{import/extern} specifies a local Bjolang identifier, the fully qualified CLR member, and its signature. For overloaded methods (such as @code{Console.Write}), the compiler determines which overload to bind based on the provided signature. Instance methods accept the target object as their first parameter. Properties and fields can be accessed via @code{#:get} and modified with @code{#:set}: +An entry in @code{import/extern} is three things: a local name, the CLR member it stands for, and a signature. Where the CLR has overloads, as @code{Console.Write} does, the signature picks one. An instance method takes its object as the first parameter. A field or property is read with @code{#:get} and, with @code{#:set}, written: @codeblock[#:lang "bjolang"]|{ (import/extern @@ -1598,11 +1597,11 @@ max-int ;; 2147483647 .NET's own string, @code{System.String}, is UTF-16 and a different type from @code{string}. A .NET method that takes or returns one, or an array of them, is converted where it is called, so @code{trim} above takes and gives a Bjolang string; a @code{null} from .NET arrives as @code{""}. A .NET string inside another type, such as a @code{(Seq System.String)}, is not converted: @code{string->clr-string} and @code{clr-string->string} do it by hand. -Avoid binding an external method to the name of an existing prelude function. Locally bound names and prelude symbols take precedence over @code{import/extern} declarations, which means an alias named @code{println} would simply be shadowed and ignored. +Do not give an external method the name of a prelude function. A local name and a prelude symbol both win over an @code{import/extern}, so an alias called @code{println} is ignored. @subsubsection{Classes: import/class} -The @code{import/class} form maps a .NET class to a Bjolang type, exposing its constructor as a function suffixed with a dot (e.g. @code{StringBuilder.}). CLR enums map similarly, with individual members accessed using dot notation like @code{Type.Member} (seen with @code{ConsoleColor.Red} above). +@code{import/class} gives a .NET class a Bjolang type, and its constructor a function with a trailing dot: @code{StringBuilder.} builds one. A CLR enum works the same way, and its members are written @code{Type.Member}, as @code{ConsoleColor.Red} above. @codeblock[#:lang "bjolang"]|{ (import/class @@ -1618,17 +1617,11 @@ The @code{import/class} form maps a .NET class to a Bjolang type, exposing its c A method that @code{string} does not have, like @code{ToUpper}, is called on the string converted to .NET's, so its indices and lengths are .NET's UTF-16 ones and not @code{string-length}'s bytes. -As shown in the example above, you do not need to declare every method up front with @code{import/extern}. Once an object's type is known to the type checker, you can invoke any public method using @code{(.Method obj args ...)} and inspect properties using @code{(.-Property obj)}. Because @code{Append} returns the builder itself and Bjolang enforces explicit handling of return values, discard unused returns intentionally using @code{ignore}. +You do not have to declare every method up front. Once the type checker knows an object's type, any public method is @code{(.Method obj args ...)} and any property is @code{(.-Property obj)}. @code{Append} returns the builder itself, and a return value has to be used, so the unused ones above go through @code{ignore}. @subsubsection{When .NET fails} -Idiomatic .NET code reports errors by throwing exceptions. Bjolang provides three distinct ways to translate these exceptions into safe value types: - -@read-list{ -- Generic @code{try} blocks: Wrap arbitrary code in @code{try}, as introduced in Part 1. -- Exception-mapped constructors: Annotating a constructor with @code{#:exceptions} in @code{import/class} causes the constructor to return a @code{Result} instead of throwing on specified exceptions. -- Output parameter conversion: Common .NET methods adhering to the @code{bool TryParse(..., out T result)} pattern automatically transform into Bjolang functions that return an @code{(Option T)}. -} +.NET reports errors by throwing. There are three ways to get a value instead: @code{try} around the call, as in Part 1; a constructor declared with @code{#:exceptions} in @code{import/class}, which returns a @code{Result} for those exceptions instead of throwing; and a method with the @code{bool TryParse(..., out T result)} shape, which becomes a function returning @code{(Option T)}. @codeblock[#:lang "bjolang"]|{ (import/class @@ -1648,7 +1641,7 @@ Idiomatic .NET code reports errors by throwing exceptions. Bjolang provides thre @subsubsection{Async methods, and blocking calls} -A .NET method returning a @code{Task} or @code{Task} can be imported using the @code{#:async} modifier. Calling an async method from within a bjoroutine suspends the fiber until the task completes without blocking an OS thread, with no explicit @code{await} required: +A method that returns a @code{Task} or @code{Task} is imported with @code{#:async}. Called from a bjoroutine it suspends the fiber until the task is done and blocks no thread. There is no @code{await} to write: @codeblock[#:lang "bjolang"]|{ (import/extern @@ -1659,11 +1652,11 @@ A .NET method returning a @code{Task} or @code{Task} can be imported using th 0) }| -Conversely, CLR methods known to perform synchronous, thread-blocking I/O should be tagged with @code{#:blocking}. Calling a blocking method from within a bjoroutine triggers a compiler warning; you can resolve this warning by offloading the call to a dedicated thread pool thread via @code{(sync (blocking (fun () ...)))}. +A CLR method that blocks its thread is tagged @code{#:blocking}. Calling one from a bjoroutine is a compiler warning, and the fix is to give it a thread of its own: @code{(sync (blocking (fun () ...)))}. @subsubsection{Generics} -Generic .NET types are imported by parameterizing them with type variables, and generic methods specify those type variables in their signature: +A generic .NET type is imported with type variables, and a generic method names them in its signature: @codeblock[#:lang "bjolang"]|{ (import/class @@ -1674,15 +1667,15 @@ Generic .NET types are imported by parameterizing them with type variables, and (-> (Dict %k %v) %k (out %v) (Option %v))))) }| -At the boundary between systems, Bjolang types map cleanly to standard .NET equivalents: a @code{Func} corresponds to @code{(-> %a %b)}, @code{IEnumerable} maps to @code{(Seq %a)}, C# tuples map to @code{Tuple}, and native arrays @code{T[]} correspond to @code{(Array %a)}. Note that constructing generic .NET classes dynamically and pattern matching on .NET objects via constructor patterns are not currently supported. +Bjolang types and .NET types line up: @code{(-> %a %b)} is a @code{Func}, @code{(Seq %a)} an @code{IEnumerable}, @code{(Array %a)} a @code{T[]}, and a C# tuple is a @code{Tuple}. Two things do not work yet: building a generic .NET class from a type only known at runtime, and constructor patterns on .NET objects. @subsubsection{Culture and formatting} -Bjolang programs execute using the invariant culture by default. For example, floating-point numbers format with a decimal point (@code{2.5}), avoiding regional discrepancies on machines where local culture settings would otherwise produce @code{2,5}. If your application requires localized formatting, you can configure @code{CultureInfo.CurrentCulture} explicitly. +A Bjolang program runs in the invariant culture, so a double formats with a decimal point (@code{2.5}) and not as @code{2,5} on a machine where it otherwise would. Set @code{CultureInfo.CurrentCulture} if you want local formatting. @subsubsection{Calling Bjolang from C#} -Every Bjolang module compiles to a public static class named @code{_Module}, exposing top-level functions defined with @code{defun} as static methods and @code{def} values as static fields. Identifiers with characters invalid in C# are automatically mangled (for example, hyphenated names like @code{full-deck} become @code{fullsubdeck}), while collections remain Bjolang's persistent data structures. You can inspect the emitted C# source code directly by passing the debug flag: @code{bjo build -d}, which generates @code{out.cs}. +Every module compiles to a public static class called @code{_Module}: the @code{defun}s in it are static methods and the @code{def}s are static fields. A name with characters C# will not take is mangled, so @code{full-deck} becomes @code{fullsubdeck}. Collections stay Bjolang's persistent ones. To read the C#, build with @code{-d} and open @code{out.cs}. @subsection{Macros} diff --git a/_04.Appendices.sz b/_04.Appendices.sz index 26c6767..0f5bdbf 100644 --- a/_04.Appendices.sz +++ b/_04.Appendices.sz @@ -1,54 +1,14 @@ @section{Appendices} -@subsection{Coming from Scheme or ML} +@subsection{Language inspiration} -Bjolang sits at the crossroads of two major functional programming lineages. From the Scheme and Lisp tradition, it draws its uniform S-expression syntax, prefix notation, code-as-data perspective, and hygienic macro system. From the ML familyโ€”specifically OCaml and F#โ€”it inherits static Hindley-Milner type inference, algebraic data types, compile-time exhaustiveness checking, traits, and high-performance immutable data structures. +Bjolang is firmly rooted in two large programming language families, and draws inspiration from both. I spent most of my life writing r6rs scheme. I think it is a beautiful language, and it is still often what I reach for to try out new things. The second is ML, mostly ocaml, with it's strong tradition of always managing to select the right abstractions. If you come from any of these languages you will feel at home in one way or the other. Some things I took from both -- like immutability -- will make adherents of both churches feel right at home and some things will feel weird. -Depending on which background you bring to Bjolang, parts of the language will feel like second nature while other parts will require shifting your mental habits. +When deciding on API for bjolang I mostly asked myself "What would ocaml/scheme do?". With ocaml there is a rich set of libraries solving real world problems, so inspiration was easy to find, and from scheme I mostly had a look at racket (which is kind of r6.1rs). -@subsubsection{Coming from Scheme} +A special nod goes to Alex Shinn of Chibi scheme. Both @code{(std fmt)}, @code{(std rx)} and @code{(std scan)} are heavily influenced by programs of his. In fact, the whole idea of the @code{loop} form comes from an earlier project of mine extending his foof-loop. -If you have written Scheme, Racket, or Common Lisp, the visual rhythm of Bjolang will feel immediately comfortable: - -@read-list{ -- S-expression syntax and prefix notation: Expressions are nested parenthesized lists, and operators are ordinary functions called in prefix position: @code{(+ 1 2 (* 3 4))}. -- Familiar control forms: Core branching and binding forms like @code{let}, @code{let*}, @code{cond}, @code{when}, @code{unless}, and @code{begin} behave as you expect. -- First-class closures: Functions are first-class values created with @code{fun} (or @code{defun} at top level), capturing lexical scope, and passing into or returning from higher-order functions. -- Persistent linked lists: The @code{List} type provides standard singly-linked functional lists with @code{Cons} and @code{Nil}, along with familiar combinators such as @code{list-head}, @code{list-tail}, @code{list-map}, and @code{list-filter}. -- Hygienic macros: Bjolang features a compile-time syntactic macro system based on pattern matching with @code{syntax-match}, syntax quotation with @code{#'(...)}, and explicit identifier injection with @code{inject}. -} - -However, several fundamental differences will quickly become apparent: - -@read-list{ -- Static typing and mandatory top-level signatures: Dynamic typing is absent. While types inside function bodies are inferred automatically using Hindley-Milner type inference, every top-level definition and exported value requires an explicit signature: @code{(: name type)}. Type errors are caught at compile time before any code runs. -- No truthiness tricks or @code{#f}-as-nil: In Scheme, functions like @code{memq} or @code{assoc} return either a value or @code{#f} to signal absence, and @code{#f} is often treated as a general falsey sentinel. In Bjolang, absent values are strictly represented by the @code{(Option %a)} union (@code{Some} or @code{None}), and operations that may fail return @code{(Result %e %a)} (@code{Ok} or @code{Err}). Booleans (@code{#t} and @code{#f}) are strictly booleans; conditionals do not coerce non-boolean values. -- Vectors and maps are persistent by default: In Scheme, vectors are mutable flat arrays. In Bjolang, @code{(Vec %a)} is an immutable Relaxed Radix Balanced (RRB) tree with efficient O(log n) random indexing, updates, and concatenation. Similarly, @code{(Map %k %v)} is an immutable Compressed Hash-Array Mapped Prefix-tree (CHAMP). In-place mutation is never the default: mutable state requires explicit declaration with @code{def/mutable} and modification with @code{set!}. -- Structural pattern matching over @code{car}/@code{cdr}: Destructuring lists, records, and unions is done using the @code{match} construct rather than long chains of @code{car}, @code{cdr}, or @code{cadr}. The compiler checks pattern matches for exhaustiveness and warns or errors if any potential value is unaccounted for. -- Concurrency without @code{call/cc}: Scheme systems often implement coroutines or thread libraries using first-class continuations (@code{call/cc}). Bjolang does not provide @code{call/cc}. Instead, it offers direct-style cooperative fibers (bjoroutines), typed channels, and Concurrent ML (CML) event combinators running on a dedicated fiber scheduler. -} - -@subsubsection{Coming from F# or OCaml} - -If you come from F#, OCaml, or Haskell, the conceptual model of Bjolang's type system and runtime behavior will feel very natural: - -@read-list{ -- Hindley-Milner type inference: Types within local expressions and function bodies are completely inferred. Polymorphic functions declare type variables with a leading percent sign, such as @code{%a} or @code{%elem}. -- Algebraic data types: Data structures are modeled as sum types (@code{Union}) and product types (@code{Record} on the heap, or value-type @code{Struct} on the stack). -- Exhaustive pattern matching: The compiler tracks which variants of a union or struct combination have been handled in a @code{match} or @code{def} binding, rejecting patterns that fail to cover every possible case. -- Immutability by default: Data structures, record fields, and bindings are immutable unless explicitly declared mutable. -- Traits and ad-hoc polymorphism: Bjolang's @code{def/trait} and @code{impl} system works much like Haskell typeclasses or Rust traits. Traits can define default method implementations, require super-traits, and specify associated types. -- Direct-style concurrency: Concurrency is written in direct style without callback pyramids, closely resembling F#'s async computations or OCaml 5 effect handlers. -} - -The main shifts you will encounter coming from ML are: - -@read-list{ -- Syntax and parentheses: There is no indentation-based syntax (like F#) or @code{let ... in} keyword framing (like OCaml). Everything is an S-expression with prefix notation. While this takes an afternoon to get used to, it makes code structure uniform and simplifies automated tooling. -- First-class compile-time macros: Rather than relying on external preprocessors like OCaml's PPX or compiler plugins, Bjolang allows writing hygienic macros in the language itself using @code{def/macro}. These transformers run inside the compiler process during the expansion phase. -- Concurrent ML events instead of raw tasks: Rather than manipulating raw asynchronous tasks or promises directly, Bjolang builds upon Concurrent ML. The primitive unit of asynchronous composition is the @code{(Event %a)}. Events can be combined with @code{choose}, wrapped with transformations via @code{wrap}, and selectively synchronized using @code{sync}. -- Automatic colour twins: In many typed asynchronous languages, asynchronous functions have distinct types (@code{Task<'T>} or @code{Async<'T>}) that cannot be passed to standard higher-order functions like @code{List.map} without creating asynchronous duplicates (@code{List.mapAsync}). Bjolang solves this "function colouring problem" at the compiler level: higher-order functions declaring @code{-?->} parameters automatically generate both synchronous and suspending twins, allowing the exact same @code{list-map} to be used whether your callback performs I/O or pure computation. -} +The concurrency story is that of ConcurrentML. A language almost as old as me, that has lived on in various smaller projects. It came to my attention through Andy Wingo, who in his turn got it from the sages of Racket. There are various resources online about it, but I would really recommend watching Andy's speeches about it. There is also a talk circulating online from Mike Sperber talking about it, which can probably be found on the large streaming site. @subsection{Reading compiler errors} @@ -414,3 +374,17 @@ Key editing features and keybindings include: - @code{C-c C-o}: Follow the reference at point, opening the target URL or jumping directly to an included file. - Document navigation: Seamless integration with @code{outline-minor-mode} and @code{imenu} allows folding sections and navigating complex documents through hierarchical headings. } + +@subsection{Why it is a sad world on dotnet} + +The reason bjolang runs on dotnet is mostly because of inertia. I had a lot of code already written for c#, and reimplementing RRB trees or CHAMPs in ocaml is nothing I felt like doing. Running on dotnet comes with tradeoffs, most notably function colouring. Had I gone with ocaml5, I would have gotten effect handlers, which means delimited control, which means everything can potentially be a bjoroutine. Now I have async, which means colouring and compiling functions in two ways and sad faces all the way down. + +The longer I have come with bjolang, the more I have started looking sideways at ocaml. When it comes to selecting abstractions, ocaml and racket are somehow almost always spot on, which means my main design guideline is "when in doubt, look at ocaml/racket/(guile)". A lot of the things that were not implemented in bjolang are things that simply were too expensive on dotnet, but more or less free on ocaml. + +Most of it is delimited control, but having the world's best STM and full effect handlers would make my life a breeze. + +@subsubsection{So... when will you switch?} + +Probably never. Interop is too nice to have. Just being able to Nuget a nice markdown parser and write some glue is just too damn nice. I did some playing with ocaml, and there are some things I really do not like. Minor gc collections stop all domains. Boxed generics. + +But the effect handlers, man... From 2334da2516ae5bcd8d264b04b238ac25852e67415e451649d38bdf2a2c4473c9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Mon, 26 Oct 2026 09:54:53 +0100 Subject: [PATCH 28/38] proper english --- _01.Language-basics.sz | 2 +- _02_.Writing-programs.sz | 10 +++++----- _04.Appendices.sz | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index e71b7b8..ca23c14 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -197,7 +197,7 @@ If you keep something on purpose, start its name with an underscore: @code{(def Bjolang follows scheme's naming conventiens. Predicates end with @code{?}. Mutating functions end with @code{!}. Conversions are written using @code{->}. -While the base types are lowercase (which is a design mishap), types should be capitalized. Record and struct fields are lower case. +While the base types are lowercase (which is a design mishap), types should be capitalised. Record and struct fields are lower case. @subsection{Functions} diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 2853270..4fc9eb7 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -256,7 +256,7 @@ As a general guideline: use @code{include} when you want to divide a single larg @subsection{Projects and packages} -Up to this point, our programs have consisted of standalone files importing neighboring modules directly by relative path. While that approach is convenient for quick scripts and small experiments, any software that pulls in external dependencies or is meant to be consumed by other developers should be structured as a project. +Up to this point, our programs have consisted of standalone files importing neighbouring modules directly by relative path. While that approach is convenient for quick scripts and small experiments, any software that pulls in external dependencies or is meant to be consumed by other developers should be structured as a project. A project is simply a directory containing a @code{manifest.bjodat} file. When operating inside a project directory, @code{bjo} understands the package's identity, resolves dependencies, and manages builds automatically. Outside of a project, the CLI falls back to single-file execution just as before. @@ -985,7 +985,7 @@ Functions that may suspend executionโ€”whether waiting on a channel, sleeping, o Bjoroutines execute as lightweight fibers scheduled across a managed thread pool. When a fiber suspends at a @code{sync} point, it yields its underlying thread back to the pool. As a result, maintaining thousands of waiting fibers incurs the memory overhead of small heap objects rather than expensive OS threads. -Because calling a suspending function means the caller may also have to wait, function coloring applies: bjoroutines can only be directly invoked from within other bjoroutines: +Because calling a suspending function means the caller may also have to wait, function colouring applies: bjoroutines can only be directly invoked from within other bjoroutines: @codeblock[#:lang "text"]|{ Type Error at colour.bjo:7: calling 'think' is a yield point, and a yield point is not allowed here. @@ -997,7 +997,7 @@ Type Error at colour.bjo:7: calling 'think' is a yield point, and a yield point To enter concurrent execution from the start, @code{main} can itself be defined with @code{defbjo}. -In practice, this coloring is less infectious than in many other languages. Most standard I/O functions (such as @code{file-read-text}) are automatically emitted with dual implementations: a synchronous blocking path and an asynchronous suspending path. An ordinary @code{defun} that calls standard I/O similarly gets both versions generated under the hood; when called from a bjoroutine, it transparently executes the non-blocking suspending version: +In practice, this colouring is less infectious than in many other languages. Most standard I/O functions (such as @code{file-read-text}) are automatically emitted with dual implementations: a synchronous blocking path and an asynchronous suspending path. An ordinary @code{defun} that calls standard I/O similarly gets both versions generated under the hood; when called from a bjoroutine, it transparently executes the non-blocking suspending version: @codeblock[#:lang "bjolang"]|{ (: size-of (-> string int)) @@ -1162,7 +1162,7 @@ Cancellation in Bjolang is cooperative rather than preemptive. Cancelling a scop Depending on the desired lifecycle and supervision strategy, fibers can be started using four distinct forms: @read-table{ -| Form | Scope behavior | +| Form | Scope behaviour | |------------------------------------+----------------------------------------------------------| | @code{(spawn (f x))} | Awaited by scope; propagates failure on error | | @code{(bjo (f x))} | Awaited by scope; yields a promise holding value/failure | @@ -1773,7 +1773,7 @@ In this expansion, the internal counter variable @code{count} is generated hygie @subsubsection{Pattern macros: def/pattern} -The @code{def/pattern} form defines macros intended for use within pattern-matching positions rather than ordinary expressions. Nullary pattern macros can be referenced simply by their bare capitalized name: +The @code{def/pattern} form defines macros intended for use within pattern-matching positions rather than ordinary expressions. Nullary pattern macros can be referenced simply by their bare capitalised name: @codeblock[#:lang "bjolang"]|{ ;; Face matches a jack, a queen or a king. diff --git a/_04.Appendices.sz b/_04.Appendices.sz index 0f5bdbf..5f200a9 100644 --- a/_04.Appendices.sz +++ b/_04.Appendices.sz @@ -377,7 +377,7 @@ Key editing features and keybindings include: @subsection{Why it is a sad world on dotnet} -The reason bjolang runs on dotnet is mostly because of inertia. I had a lot of code already written for c#, and reimplementing RRB trees or CHAMPs in ocaml is nothing I felt like doing. Running on dotnet comes with tradeoffs, most notably function colouring. Had I gone with ocaml5, I would have gotten effect handlers, which means delimited control, which means everything can potentially be a bjoroutine. Now I have async, which means colouring and compiling functions in two ways and sad faces all the way down. +The reason bjolang runs on dotnet is mostly because of inertia. I had a lot of code already written for c#, and reimplementing RRB trees or CHAMPs in ocaml is nothing I felt like doing. Running on dotnet comes with tradeoffs, most notably function colouring. Had I gone with ocaml5, I would have got effect handlers, which means delimited control, which means everything can potentially be a bjoroutine. Now I have async, which means colouring and compiling functions in two ways and sad faces all the way down. The longer I have come with bjolang, the more I have started looking sideways at ocaml. When it comes to selecting abstractions, ocaml and racket are somehow almost always spot on, which means my main design guideline is "when in doubt, look at ocaml/racket/(guile)". A lot of the things that were not implemented in bjolang are things that simply were too expensive on dotnet, but more or less free on ocaml. From d16bf477a1c596db71ea44b360815e0b3da4bed27562cdfc66afbc8ac1bdf64f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Tue, 3 Nov 2026 15:18:28 +0100 Subject: [PATCH 29/38] Document units of measure and rational numbers --- _01.Language-basics.sz | 130 +++++++++++++++++++++++++++++++++++++++- _03.Standard-library.sz | 5 ++ modules/std/rational.sz | 3 + modules/std/units.sz | 3 + 4 files changed, 139 insertions(+), 2 deletions(-) create mode 100644 modules/std/rational.sz create mode 100644 modules/std/units.sz diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index ca23c14..59128eb 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -2484,9 +2484,20 @@ The numeric types are the .NET ones: | @code{int} | about ยฑ2.1 billion | @code{double} | 64-bit floating point | | @code{bigint} | any integer | @code{float} | 32-bit floating point | | @code{int128} | โˆ’2^127 โ€ฆ 2^127โˆ’1 | @code{decimal} | 28 digits, exact decimals | -| @code{uint128} | 0 โ€ฆ 2^128โˆ’1 | | | +| @code{uint128} | 0 โ€ฆ 2^128โˆ’1 | @code{rational} | exact fractions | } +All but one are the .NET ones. @code{rational} is bjolang's own, an exact fraction over @code{bigint}, and a number like the others: @code{+ - * /} and the comparisons work on it, and a literal at it is exact, so @code{0.1} is one tenth and not the double nearest to it. It prints as @code{1/3}. @code{(std rational)} has its operations by name, as @code{rational+} and @code{rational->double}. + +@codeblock[#:lang "bjolang"]|{ +(: third rational) +(def third (/ 1 3)) + +(* third 0.3) ;; 1/10, exactly +(= (* third 3) 1) ;; #t +(cast double third) ;; 0.3333333333333333 +}| + A @code{char} is not a number: it can be compared and converted, but not added. @subsubsection{Writing numbers} @@ -2642,7 +2653,7 @@ If a function only compares values, use @code{(where (Ord %a))} instead. It work @codeblock[#:lang "bjolang"]|{ (import (std maths)) -(sqrt 9) ;; 3.0: the int 9 widens to a double at the call +(sqrt 9) ;; 3.0: the 9 is a double, since sqrt takes one (pow 2.0 10.0) ;; 1024.0 (floor -2.5) ;; -3.0 (truncate -2.5) ;; -2.0 @@ -2655,3 +2666,118 @@ If a function only compares values, use @code{(where (Ord %a))} instead. It work The logarithms are @code{loge} (natural), @code{log2}, @code{log10} and @code{(log-base x b)}. The name @code{log} is taken by the logging function of the prelude. + +@subsection{Units of measure} + +A number can carry a dimension in its type. A @code{(double m/s)} is a speed, a @code{(double m)} is a length, and the compiler refuses to add the two. The checking is all at compile time: a @code{(double m/s)} is a plain @code{double} at run time, and a conversion between units is a constant the compiler works out. + +@subsubsection{Quantities} + +A quantity is written in angle brackets, a number and then a unit: + +@codeblock[#:lang "bjolang"]|{ +(import (std units)) + +(def g <9.80665 m/s^2>) ;; a (double m/s^2) +(def limit <120 km/h>) ;; a (double m/s): 33.33โ€ฆ +(def width <3 ft>) ;; a (double m): 0.9144 +}| + +Each dimension has a base unit, and a quantity is kept in it: a length in metres, a time in seconds, a speed in metres per second. @code{<120 km/h>} is converted to metres per second when the program is compiled, and is one constant in the code that comes out. So every length is the same type, whatever unit it was written in, and @code{(+ <1 m> <1 ft>)} is 1.3048 metres. + +The unit is written as GNU @code{units} writes one: @code{kg m/s^2} is kilograms times metres per second squared, a space or a @code{*} multiplies, @code{/} divides, and @code{^} raises, with a negative or a fractional power as @code{s^-2} or @code{Hz^(-1/2)}. Everything after @code{/} up to the next @code{/} or @code{*} is below the line, so @code{m/s kg} is m/(sยทkg). Prefixes go on any unit: @code{km}, @code{mg}, @code{GHz}, @code{KiB}. The number is read exactly, and may be a fraction: @code{<1|3 h>} is twenty minutes, and @code{<1e-3 km>} is a metre. A @code{<} with a digit right after it starts a quantity, so @code{(< 1 2)} is still a comparison, but @code{(<1 2)} is not. + +@subsubsection{Types} + +A type with a unit is a number type and a unit: @code{(double m/s)}, @code{(long cent)}, @code{(double kg m^2/s^2)}. A type names base units, so a field of a record is @code{(double m)} even when the numbers in it are thought of as kilometres. A unit that is only a name for base units can be written too: @code{(double N)} is @code{(double kg m/s^2)}. One that has a factor cannot, because a @code{(double km)} would be a number in metres with a lie on it: + +@codeblock[#:lang "text"]|{ +Type Error at trip.bjo:4:10: 'km' is 1000 m, and a type names base units only: +write (double m), and (number-in x km) where a number in km is needed. +}| + +@code{} is the unit with the number type its dimension keeps it in, which is @code{double} for SI. Records, unions, signatures and @code{let} take these types as they take any other: + +@codeblock[#:lang "bjolang"]|{ +(type (: Leg (Record (: distance (double m)) + (: climb (double m)) + (: speed (double m/s))))) + +(def leg (Leg (distance <38 km>) (climb <800 ft>) (speed <15 mph>))) + +(: hours (-> Leg (double s))) +(defun (hours l) (/ (.distance l) (.speed l))) ;; m / (m/s) is s +}| + +@subsubsection{Arithmetic} + +@code{+}, @code{-} and the comparisons take two numbers of the same unit, and @code{*} and @code{/} multiply and divide the units, so @code{(* <2 m> <3 m>)} is a @code{(double m^2)}. @code{sqrt} takes the root of the unit too: the square root of an area is a length. A number with no unit scales: @code{(* 2 g)} is still an acceleration. + +@codeblock[#:lang "bjolang"]|{ +(+ (.distance leg) (.climb leg)) ;; a length: 38243.84 m +(+ (.distance leg) (.speed leg)) ;; a type error: a length and a speed +(* <2 m> <3 m>) ;; 6 m^2 +(sqrt (* <2 m> <8 m>)) ;; 4 m +}| + +A plain number has no unit, so it cannot be compared with a length. The error says what to write instead. Zero is the exception: it is a number of every unit, so @code{(> (.climb leg) 0)} works. + +@codeblock[#:lang "text"]|{ +Type Error at trip.bjo:9:22: '1' is a plain number, and it is being used where a +(double m) is wanted. Write <1 m>, or a unit of the same dimension. +}| + +A function that works at any unit says so with a unit variable, written as a type variable is: + +@codeblock[#:lang "bjolang"]|{ +(: larger (-> (double %u) (double %u) (double %u))) +(defun (larger a b) (if (> a b) a b)) + +(larger <1 km> <900 m>) ;; 1000 m: lengths are kept in metres +(larger <3 s> <2 s>) ;; 3 s + +(: rms (-> (double %u) (double %u) (double %u))) +(defun (rms a b) (sqrt (/ (+ (sqr a) (sqr b)) 2.0))) ;; sqr: %u to %u^2 +}| + +@code{(num %c %u)} is a number of any type at any unit, for a function that is generic in both. A function that is generic over @code{Num} does not take a number with a unit, because @code{Num} allows multiplication and its type promises that the unit stays: @code{(sqr x)} on metres is square metres. @code{abs}, @code{min}, @code{max}, @code{clamp}, @code{sign}, @code{zero?}, @code{sqr} and @code{positive?} take any unit. @code{summing} does not yet. + +@subsubsection{Getting the number out} + +@code{(number-in x unit)} is the number @code{x} is in a unit, as a plain number: what a quantity is when it is printed, written to a file or given to .NET. The unit must have @code{x}'s dimension. The other way, a plain number gets a unit by multiplication: + +@codeblock[#:lang "bjolang"]|{ +(number-in (hours leg) h) ;; 1.574: the leg takes an hour and a half +(number-in (.distance leg) km) ;; 38.0 +(number-in <45 deg> rad) ;; 0.785โ€ฆ +(* 2.5 <1 km>) ;; a length: 2500 m +(number-in (.distance leg) s) ;; a type error: a length, not a time +}| + +@code{number-in} is one multiplication by a constant, and none when the unit is the base. @code{cast} also takes the unit off, but it does not convert: it is right only for a number already in the base unit. + +@subsubsection{Dimensions and units of your own} + +Nothing about SI is built into the compiler. @code{(std units)} is a module of declarations, and a program can make its own the same way, with or without it: + +@codeblock[#:lang "bjolang"]|{ +(def/dimension money #:base cent #:carrier long) +(def/unit EUR <100 cent>) +(def/prefix k 1000) + +(: price (long cent)) +(def price <12.50 EUR>) ;; 1250L +(number-in price EUR) ;; a compile error: it would divide a long by 100 +}| + +@read-list{ +- @code{(def/dimension name #:base unit)} declares a dimension and its base unit. @code{#:carrier} says which number type its quantities are when nothing else says: @code{double} when it is left out. A type can always say otherwise, as @code{(double cent)} does. +- @code{(def/unit name quantity)} declares a unit as an amount of units already declared: @code{(def/unit mi <5280 ft>)}, @code{(def/unit N <1 kg m/s^2>)}. A unit with no dimension is a plain number: @code{(def/unit % <1|100>)}. +- @code{(def/prefix name number)} declares a prefix that goes on every unit. +} + +Units belong to the module that declares them and to the modules that import it. Two modules that each declare an @code{m} have two different units, as two types called @code{Point} are two types. + +A literal at an integer type has to come out whole: @code{<0.001 EUR>} at a @code{(long cent)} is a compile error, and so is @code{number-in} to a coarser unit, which would divide. At @code{rational} every conversion is exact, so a program that must not round declares its dimensions with @code{#:carrier rational}: there, @code{<1 km/h>} is exactly 5/18 metres per second, and back in kilometres per hour it is exactly 1. + +The module reference for @link["modules/std/units.html"]{@code{(std units)}} lists the units it declares, and has a library for units at run time, for a unit that is read as text. diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 03f0c6a..6b767f2 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -12,6 +12,11 @@ For more details, see @link["modules/std/prelude.html"]{@code{(std prelude)}}. Numbers beyond the operators. Integer division has Racket's names: @code{quotient} and @code{remainder} are what @code{/} and @code{%} do, and @code{modulo} takes the sign of the divisor, so @code{(modulo -1 10)} is 9. There are @code{gcd}, @code{lcm}, an integer @code{expt} and an exact @code{integer-sqrt} at every integer type, the @code{exact-} roundings from a double to an int, and the rest of @code{System.Math} over doubles. The prelude has @code{abs}, @code{min}, @code{max} and @code{clamp} from here without an import. For more details, see @link["modules/std/maths.html"]{@code{(std maths)}}. +@subsection{std/units and std/rational} + +Units of measure, and exact fractions. @code{(std units)} declares SI for the units in types and quantity literals that "Language basics" describes: the eight base dimensions, the SI and binary prefixes, and about 270 units from @code{in} and @code{mi} to @code{eV}, @code{Sv} and @code{rpm}. The constants @code{c}, @code{gn} and @code{planck} are units too, so @code{(number-in v c)} is a speed as a fraction of light's. It is also a library for units at run time, when a unit comes as text, as in a calculator: @code{(convert-query "6000 m + 50 ft in mi")} gives @code{(Ok "3.737697 mi")}, and a @code{Measurement} is a number with a dimension that is checked when the program runs. Absolute temperatures are the types @code{Celsius} and @code{Fahrenheit} rather than units, and the decibels are functions. @code{(std rational)} has the operations of the built-in @code{rational} by name, the roundings to a @code{bigint}, and conversions to and from text, rounded once to a @code{double} or a @code{decimal}. +For more details, see @link["modules/std/units.html"]{@code{(std units)}} and @link["modules/std/rational.html"]{@code{(std rational)}}. + @subsection{std/array and std/simd} Two modules for numbers in bulk, with the CPU's vector instructions. @code{(std array)} has whole-array operations: @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-add!}, @code{array-scale!}, @code{array-count}, @code{array-index-of} and @code{array-fill!}. Each is one loop in the runtime, which works on as many elements per instruction as a vector register holds when the element is a primitive number, and one at a time for a type such as @code{bigint}. At @code{double}, a sum adds lane by lane, so its last bits can differ from a left fold. @code{(std simd)} has the vectors themselves, for the loops that the kernels do not cover. @code{(Simd128 %a)} and @code{(Simd256 %a)} are .NET's @code{Vector128} and @code{Vector256}, with a name per element type: @code{Float64x4} is four doubles, @code{UInt8x32} is 32 bytes. The operations are the methods of one trait, @code{Simd}, at both widths: @code{simd-add}, @code{simd-mul}, the comparisons @code{simd-lt} and the others, which give masks, @code{simd-select}, @code{simd-count-true}, @code{simd-sum} and more. A vector is made with @code{simd256-load}, @code{simd128-splat} or @code{simd256-zero}, and @code{simd256-whole} says where the elements after the last whole vector start. A 128-bit vector is fast on every x86-64 and ARM64 CPU. A 256-bit vector needs AVX2, and @code{simd256-accelerated?} says whether the CPU has it. The section on performance in "Writing programs" says when vectors pay. diff --git a/modules/std/rational.sz b/modules/std/rational.sz new file mode 100644 index 0000000..51c6cef --- /dev/null +++ b/modules/std/rational.sz @@ -0,0 +1,3 @@ +(meta (title "(std rational)")) + +@generate-docs[#:module "std rational"] diff --git a/modules/std/units.sz b/modules/std/units.sz new file mode 100644 index 0000000..bcae12b --- /dev/null +++ b/modules/std/units.sz @@ -0,0 +1,3 @@ +(meta (title "(std units)")) + +@generate-docs[#:module "std units"] From 6ee8ca62b950109e34da0d8f7bf0f16fe916924d0474ed184f8e4a2006ae3a7b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Mon, 9 Nov 2026 16:11:17 +0100 Subject: [PATCH 30/38] Document spans, interop, csv, parallel, and combine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Language basics: add Span/RoSpan section with construction, operations and lifetime rules - Writing programs: add full Spans in .NET interop subsubsection (arrays-as-spans, #:copy for span returns, span values and their restrictions); update performance notes to remove "without spans" - Standard library: split std/units+rational into two subsections with fuller descriptions; add std/parallel+combine, std/interop, and text/csv+csv-codec subsections - index: mention CSV in the text module description ๐Ÿค– Generated with [ECA](https://eca.dev) (anthropic/claude-sonnet-4-6) Co-Authored-By: eca-agent --- _01.Language-basics.sz | 12 +++++++ _02_.Writing-programs.sz | 75 ++++++++++++++++++++++++++++++++++++++-- _03.Standard-library.sz | 26 ++++++++++++-- index.sz | 3 +- 4 files changed, 109 insertions(+), 7 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 59128eb..67bdb3d 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1332,6 +1332,18 @@ An @code{Array} is a plain .NET array: mutable, fixed in length, and exactly wha Every evaluation of an array literal makes a fresh array, so a function that returns @code{#[0 0]} returns a new one each time. +To work on part of an array without copying it, make a span: @code{(Span %a)} is a writable view (.NET's @code{Span}) and @code{(RoSpan %a)} is a read-only view (@code{ReadOnlySpan}). + +@codeblock[#:lang "bjolang"]|{ +(def s (array->span a #:from 1 #:to 3)) +(span-length s) ;; 2 +(span-ref s 0) ;; 2 +(span-set! s 0 99) ;; writes into a: a is now #[42 99 3] +(span->array s) ;; #[99 3], a fresh copy +}| + +A span is valid only while the call that made it runs, so it can be a local variable, a function argument or a return value, and the compiler refuses it anywhere it could outlive the call (such as inside another collection or across a yield point). The section on .NET interop in Part 2 covers the full rules and how spans work with .NET methods. + @subsubsection{Maps} @code{Map} is an immutable hash map. The @code{#map(...)} literal takes entries written as @code{(key value)}, @code{[key value]} or @code{(key . value)}. diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 4fc9eb7..0f16140 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -1669,6 +1669,77 @@ A generic .NET type is imported with type variables, and a generic method names Bjolang types and .NET types line up: @code{(-> %a %b)} is a @code{Func}, @code{(Seq %a)} an @code{IEnumerable}, @code{(Array %a)} a @code{T[]}, and a C# tuple is a @code{Tuple}. Two things do not work yet: building a generic .NET class from a type only known at runtime, and constructor patterns on .NET objects. +@subsubsection{Spans} + +Many .NET methods take a @code{Span} or a @code{ReadOnlySpan} instead of an array. You can work with them in two ways: pass an @code{(Array %a)} directly, or use the prelude's @code{(Span %a)} and @code{(RoSpan %a)} types. + +@strong{Arrays where .NET takes a span.} A @code{ReadOnlySpan} or @code{Span} parameter accepts an @code{(Array %a)}, both in an @code{import/extern} signature and in a @code{(.Method obj ...)} call. The span views the array in place, so a method that writes into a @code{Span} mutates the array. When a method is overloaded, an overload taking @code{T[]} wins over a span overload, and @code{ReadOnlySpan} wins over @code{Span}: + +@codeblock[#:lang "bjolang"]|{ +(import/extern + (ascii? (: System.Text.Ascii.IsValid (-> (Array byte) bool))) ;; ReadOnlySpan + (write-be! (: System.Buffers.Binary.BinaryPrimitives.WriteInt32BigEndian (-> (Array byte) int void))) ;; Span + (reverse! (: System.MemoryExtensions.Reverse (-> (Array %a) void)))) ;; Span +}| + +A span that a .NET member @em{returns} is not an array. To copy a returned span into a fresh @code{(Array %a)}, mark the @code{import/extern} clause with @code{#:copy}: + +@codeblock[#:lang "bjolang"]|{ +(import/class + (Encoding (: System.Text.Encoding))) + +(import/extern + (utf8 (: System.Text.Encoding.UTF8 #:get)) + (preamble (: System.Text.Encoding.Preamble (-> Encoding (Array byte)) #:get #:copy)) + (drop-span (: System.MemoryExtensions.AsSpan (-> (Array %a) int (Array %a)) #:copy))) + +(preamble utf8) ;; utf8.Preamble.ToArray() +}| + +@code{#:copy} emits @code{.ToArray()} on the call itself, so no span is kept in a local variable and the imported binding can even be passed as a first-class function value. Without @code{#:copy}, declaring a span return as an array is a compile error. @code{#:copy} cannot be combined with @code{#:async}, @code{#:set}, or @code{(out T)} parameters. + +@strong{Spans as values.} To slice an array or pass views around without allocating, use @code{(Span %a)} (.NET's @code{Span}) and @code{(RoSpan %a)} (@code{ReadOnlySpan}), which are in the prelude: + +@codeblock[#:lang "bjolang"]|{ +(: sum (-> (Span long) long)) +(defun (sum s) + (let loop ((i 0) (acc 0L)) + (if (< i (span-length s)) + (loop (+ i 1) (+ acc (span-ref s i))) + acc))) + +(def xs #[1L 2L 3L 4L]) +(sum (array->span xs #:from 1 #:to 3)) ;; 5, and nothing is copied +(span-set! (array->span xs) 0 10L) ;; xs[0] = 10 +}| + +@read-table{ +| @code{Span} | @code{RoSpan} | Does | +|--------------------------------------+-----------------------------------------+-----------------------------------------------------| +| @code{(array->span a #:from #:to)} | @code{(array->ro-span a #:from #:to)} | a view of @code{a} from @code{#:from} to @code{#:to}| +| @code{(span-length s)} | @code{(ro-span-length s)} | number of elements | +| @code{(span-ref s i)} | @code{(ro-span-ref s i)} | element at @code{i} | +| @code{(span-set! s i x)} | | writes @code{x} into the underlying storage | +| @code{(span-slice s #:from #:to)} | @code{(ro-span-slice s #:from #:to)} | a sub-view of @code{s}; nothing is copied | +| @code{(span->array s)} | @code{(ro-span->array s)} | a fresh array copy of the span's elements | +| @code{(span->ro-span s)} | | the same view, read-only | +} + +@code{#:from} defaults to @code{0} and @code{#:to} defaults to the length. Neither span type converts to the other implicitly; use @code{span->ro-span} when a @code{RoSpan} is needed. In @code{(std mutable vec)}, @code{(mutablevec-span v start n)} returns a @code{(Span %a)} over @code{n} elements of a @code{MutableVec}'s backing storage (valid until the vector's length changes), which pairs with @code{(mutablevec-grow! v n)} to fill a vector a block at a time. + +You can also call a span's .NET members directly: @code{(.-Length s)}, @code{(.Slice s 1)}, @code{(.Fill s x)}, @code{(.get_Item s i)} for @code{s[i]}, and @code{(.set_Item s i x)} for @code{s[i] = x} (an indexer returning @code{ref T} reads a copy of the element, and @code{.set_Item} writes through the reference). + +@strong{Where a span may not go.} A spanโ€”or any other .NET @code{ref struct}โ€”is a view of memory that is valid only while the call that made it runs. It can be a local variable, a parameter, or a return value of a function called by name, and the compiler refuses it everywhere it could outlive the call: + +@read-list{ +- Inside another value: a vec, list, tuple, record, union, @code{Option}, or @code{Result}. +- As a type argument of a generic function or trait (such as @code{(println s)} or @code{loop}'s @code{:for} over a collection); walk a span by index instead. +- As a parameter or result of a function @em{value} (a @code{(fun ...)} or a function passed as a value is a .NET delegate, which cannot take or return a span). Write a local @code{(defun ...)} and call it by name. +- Captured from an outer scope by a @code{(fun ...)}, a local @code{(defun ...)}, a non-tail-recursive @code{let} loop, a @code{(seq ...)}, or a hoisted form. Pass the span to the local function as an argument instead. +- In scope across a suspension point in a bjoroutine, or across a @code{(yield ...)} in a @code{(seq ...)}. Do the span work in a synchronous @code{(defun ...)} helper so the span ends before the fiber suspends or yields. +- As a parameter or return value of a bjoroutine, an argument to @code{(bjo ...)}, or a module-level @code{def}. +} + @subsubsection{Culture and formatting} A Bjolang program runs in the invariant culture, so a double formats with a decimal point (@code{2.5}) and not as @code{2,5} on a machine where it otherwise would. Set @code{CultureInfo.CurrentCulture} if you want local formatting. @@ -2528,7 +2599,7 @@ A percentage can be written @code{2} or @code{2%}. A benchmark file is an ordina @code{prepare} runs a benchmark's setup and warms it up and calibrates it, as a run would. @code{sample} takes one sample of @code{k} calls, and @code{:ns} is the whole sample's time. Anything wrong is answered with @code{(error "why")}, and the session goes on. A served program prints nothing else, so its output is the protocol and nothing more. A script that compares two builds made by two different compilers, which is not something @code{bjo bench} does by itself, can drive both of them this way, or use @code{--against PATH --against-build "CMD"}. @subsection{Performance} -Bjolang is compiled into pretty straight-forward c#, but without spans or ref structs. It performs pretty similar to c#, except for some small corners of the language that are different. +Bjolang is compiled into pretty straight-forward c#. It performs pretty similar to c#, except for some small corners of the language that are different. @subsubsection{Strings} Strings in bjolang are utf8, as such they do @em{not} support random access. This means we navigate them differently, using string-cursors, which is explained in the part about strings. Navigating a string using cursors is slower than blasting through integer indices, and if your code needs to be really fast, there are some things you can think about: @@ -2545,7 +2616,7 @@ When parsing strings with a known format, (std scan) will be a lot faster than ( @subsubsection{loops} Most of the time you should use the @code{loop} form. It will be the fastest, and is usually flexible enough to do what you need. The only case is for hot loops (ones that are run often) that are performance sensitive. For those, using the various @code{-fold}, @code{-for-each}, and @code{-iter} functions will be faster for the collections that are tree-based. Those collections are: @code{Vec}, @code{Map}, @code{HashSet}, @code{OrderedMap}, and @code{OrderedMap}. The overhead of loop vs. fold is often something like 3x, with fold being as fast or slightly faster thana @code{foreach} on a C# @code{List}. It does not, however, beat using indexed iteration over the @code{List} due to the @code{foreach} doing some safety checks to detect concurrent mutation. -The absolutely fastest collection to loop over is, unsurprisingly, the Array. In bjolang it should be just as fast as in c# (again, without spans). +The absolutely fastest collection to loop over is, unsurprisingly, the Array (or a @code{Span} over part of one). In bjolang it should be just as fast as in c#. The loop form gets turned into a kind of goto soup. For the single level case, it is just as efficient as a c# @code{while} loop, but for really complex cases with many levels and subloops with clauses the JIT in dotnet may struggle. In that case you either live with the clarity and simplicity it gives, or you reach for the glorified gotos of names lets. This is probably not going to be a problem though. I tried to rewrite the worst loop form in bjolang (a 16 line monstrosity with one large multi-level claused subloop and a regular subloop) and there was no difference. diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 6b767f2..9f721ab 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -12,16 +12,26 @@ For more details, see @link["modules/std/prelude.html"]{@code{(std prelude)}}. Numbers beyond the operators. Integer division has Racket's names: @code{quotient} and @code{remainder} are what @code{/} and @code{%} do, and @code{modulo} takes the sign of the divisor, so @code{(modulo -1 10)} is 9. There are @code{gcd}, @code{lcm}, an integer @code{expt} and an exact @code{integer-sqrt} at every integer type, the @code{exact-} roundings from a double to an int, and the rest of @code{System.Math} over doubles. The prelude has @code{abs}, @code{min}, @code{max} and @code{clamp} from here without an import. For more details, see @link["modules/std/maths.html"]{@code{(std maths)}}. -@subsection{std/units and std/rational} +@subsection{std/rational} -Units of measure, and exact fractions. @code{(std units)} declares SI for the units in types and quantity literals that "Language basics" describes: the eight base dimensions, the SI and binary prefixes, and about 270 units from @code{in} and @code{mi} to @code{eV}, @code{Sv} and @code{rpm}. The constants @code{c}, @code{gn} and @code{planck} are units too, so @code{(number-in v c)} is a speed as a fraction of light's. It is also a library for units at run time, when a unit comes as text, as in a calculator: @code{(convert-query "6000 m + 50 ft in mi")} gives @code{(Ok "3.737697 mi")}, and a @code{Measurement} is a number with a dimension that is checked when the program runs. Absolute temperatures are the types @code{Celsius} and @code{Fahrenheit} rather than units, and the decibels are functions. @code{(std rational)} has the operations of the built-in @code{rational} by name, the roundings to a @code{bigint}, and conversions to and from text, rounded once to a @code{double} or a @code{decimal}. -For more details, see @link["modules/std/units.html"]{@code{(std units)}} and @link["modules/std/rational.html"]{@code{(std rational)}}. +Exact fractions over @code{bigint}. @code{Rational} is the built-in @code{rational} type, kept in lowest terms with a positive denominator so equal fractions are equal values. @code{(rational n d)} constructs a fraction, @code{rational-num} and @code{rational-den} take it apart, and @code{rational+}, @code{rational*}, @code{rational-recip} and @code{rational-expt} work alongside the standard arithmetic operators. @code{string->rational} reads both @code{"1|60"} and decimal text such as @code{"0.3048"}, and @code{rational->double} and @code{rational->decimal} convert back. +For more details, see @link["modules/std/rational.html"]{@code{(std rational)}}. + +@subsection{std/units} + +Units of measure, both at compile time and at run time. Importing the module declares the SI base dimensions and @code{information} (@code{b}), the SI and binary prefixes, and about 270 units with @code{def/dimension}, @code{def/prefix} and @code{def/unit}โ€”the definitions that quantity literals like @code{<120 km/h>}, unit-annotated types like @code{(double m/s)}, and @code{number-in} use at compile time and erase at run time. For units that arrive as text at run time, @code{string->measurement}, @code{measurement-in} and @code{convert-query} (e.g. @code{(convert-query "6000 m + 50 ft in mi")}) parse, check dimensions and convert against a runtime unit table. +For more details, see @link["modules/std/units.html"]{@code{(std units)}}. @subsection{std/array and std/simd} Two modules for numbers in bulk, with the CPU's vector instructions. @code{(std array)} has whole-array operations: @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-add!}, @code{array-scale!}, @code{array-count}, @code{array-index-of} and @code{array-fill!}. Each is one loop in the runtime, which works on as many elements per instruction as a vector register holds when the element is a primitive number, and one at a time for a type such as @code{bigint}. At @code{double}, a sum adds lane by lane, so its last bits can differ from a left fold. @code{(std simd)} has the vectors themselves, for the loops that the kernels do not cover. @code{(Simd128 %a)} and @code{(Simd256 %a)} are .NET's @code{Vector128} and @code{Vector256}, with a name per element type: @code{Float64x4} is four doubles, @code{UInt8x32} is 32 bytes. The operations are the methods of one trait, @code{Simd}, at both widths: @code{simd-add}, @code{simd-mul}, the comparisons @code{simd-lt} and the others, which give masks, @code{simd-select}, @code{simd-count-true}, @code{simd-sum} and more. A vector is made with @code{simd256-load}, @code{simd128-splat} or @code{simd256-zero}, and @code{simd256-whole} says where the elements after the last whole vector start. A 128-bit vector is fast on every x86-64 and ARM64 CPU. A 256-bit vector needs AVX2, and @code{simd256-accelerated?} says whether the CPU has it. The section on performance in "Writing programs" says when vectors pay. For more details, see @link["modules/std/array.html"]{@code{(std array)}} and @link["modules/std/simd.html"]{@code{(std simd)}}. +@subsection{std/parallel and std/combine} + +Data-parallel work across CPU cores on dedicated compute threads. @code{(std parallel)} splits any @code{Chunkable} collection (@code{Range}, @code{Vec}, @code{Array}, @code{IArray}, @code{Map}, @code{HashSet}, @code{OrderedMap} and @code{OrderedSet}) into parts without copying elements. @code{in-parts} runs a function on each part and returns a vector of the part results; @code{in-parallel} joins the part results in order using @code{Combinable} from @code{(std combine)}; and @code{par-map}, @code{par-filter} and @code{par-for-each} operate over a whole collection. For an array, @code{part->span} gives each worker a writable @code{(Span %a)} over its own non-overlapping slice so an array can be updated in place across cores without locks. @code{(std combine)} provides the associative @code{Combinable} trait (@code{combine} and @code{combine-all}) for types with one natural ordered combination: @code{Vec}, @code{List}, @code{IArray}, @code{string}, sets and tuples. +For more details, see @link["modules/std/parallel.html"]{@code{(std parallel)}} and @link["modules/std/combine.html"]{@code{(std combine)}}. + @subsection{std/fmt} Formatting in two layers: interpolation and layout. @code{println*} prints strings, ints, doubles, chars and bools written in one vec. The layout layer is made of blocks: a @code{Block} is one piece of output, and every combinator takes its content as a block and gives back a block. A string, an int and a character are blocks, and so is a vec of blocks, such as @code{["x = " 7 nl]}. With them you pad, cut and align text, wrap and justify paragraphs, format numbers, and lay out tables with borders, a header, and columns that fit the width. The section on formatting text in "Writing programs" is a guide, with what goes in the brackets and what to write when it does not compile. @@ -117,6 +127,11 @@ For more details, see @link["modules/std/mutable/vec.html"]{@code{(std mutable v An inbox is how .NET code gives work to Bjolang. A thread that is not a fiber, such as a timer, a @code{FileSystemWatcher} or a Kestrel thread, posts an item and returns. A fiber takes the item with its own handlers, parameters and cancellation scope. An inbox always has a capacity, and a @code{WhenFull} policy says what a post does when the inbox is full. @code{inbox-call} is a request and a reply: the .NET caller gets a @code{Task}, and a fiber answers it. For more details, see @link["modules/std/inbox.html"]{@code{(std inbox)}}. +@subsection{std/interop} + +Bridges for .NET APIs that want something a Bjolang function or value is not. A Bjolang function is a @code{Func} whose @code{void} return is the unit value; @code{as-delegate} and @code{as-delegate/type} wrap a function in another .NET delegate type such as @code{Action}, @code{EventHandler} or a library's custom delegate. @code{add-event-handler!} and @code{remove-event-handler!} subscribe a Bjolang function to a .NET object's event by name. @code{clr-default} produces .NET's @code{default(T)} (such as @code{null} for a reference type or a @code{Nullable}) when a .NET call requires one. +For more details, see @link["modules/std/interop.html"]{@code{(std interop)}}. + @subsection{std/ports and std/net} @code{(std ports)} reads and writes bytes on the prelude's ports. The byte reads @code{read-some-evt}, @code{read-bytes-evt} and @code{eof-evt} are CML events: a read that loses its @code{choose} takes nothing, so a read can race a timeout. @code{read-line} and @code{read-bytes} can be used on the same port. The module also drains ports into lists and vecs, streams them as a @code{Seq} or a channel, and has @code{byte-pipe}, a pair of ports with no file and no network, for tests. @code{(std net)} gives a TCP connection as a pair of these ports, so a protocol written for ports also runs over TCP. @code{accept-evt} is an event too: a branch that loses accepts nothing. @@ -158,6 +173,11 @@ For more details, see @link["modules/text/bjodat.html"]{@code{(text bjodat)}} an @code{(text json)} reads JSON, exactly as RFC 8259 has it, into a @code{Json}: a union with a case for each kind of value. @code{parse-json} reads a string and @code{read-json} a port. A document that is not JSON is an @code{Err} that says the line and column where it went wrong. A number without a fraction or an exponent is a @code{JsonInt}, and an object is a @code{(Map string Json)}, so the order of its keys is not kept. @code{(text json-codec)} reads a record straight from the tree and back: @code{def/json-type} declares a record and the instances of @code{->json} and @code{json->} for it, with defaults, optional fields and other names for the keys. An @code{Option} is @code{null} when it is @code{None}. There is no writer yet, so a @code{Json} cannot be written out as text. For more details, see @link["modules/text/json.html"]{@code{(text json)}} and @link["modules/text/json-codec.html"]{@code{(text json-codec)}}. +@subsection{text/csv and text/csv-codec} + +@code{(text csv)} reads and writes CSV (RFC 4180), with configurable @code{#:separator}, @code{#:quote-char} and @code{#:strict?}. It reads in three layers: @code{csv-next!} pulls one row at a time, @code{csv-fold} and @code{csv-for-each} stream rows without keeping the document in memory, and @code{parse-csv} reads the whole input. Each row is an @code{(IArray string)}, and malformed input returns a @code{CsvError} with its line, row and field. @code{write-csv} and @code{csv->string} quote fields only when needed. @code{(text csv-codec)} maps rows to records and back: @code{def/csv-type} declares a record and its @code{CsvRecord} instance, matching columns by header name (or @code{#:csv} alias) or by position when @code{#:header? #f}, with @code{parse-csv-records}, @code{csv-for-each-record} and @code{write-csv-records}. +For more details, see @link["modules/text/csv.html"]{@code{(text csv)}} and @link["modules/text/csv-codec.html"]{@code{(text csv-codec)}}. + @subsection{text/xml} XML, in three modules over @code{System.Xml}. @code{(text xml)} is the tree: names, elements, texts, comments and processing instructions, built by @code{xml-element} and the others, and taken apart by views. @code{(text xml read)} reads XML in four layers: a stream of signals, a fold after SSAX, a tree of your own type, and @code{parse-xml}, which gives a @code{Document}. @code{xml-for-each} reads a large document one element at a time. The defaults are the safe ones: a document with a DOCTYPE is refused unless asked for, and nothing external is ever resolved. @code{(text xml write)} writes a node or a document to a port or a string. It chooses the namespace prefixes, and content that XML cannot hold is an @code{XmlWriteError} rather than an exception. diff --git a/index.sz b/index.sz index f758658..47cb33f 100644 --- a/index.sz +++ b/index.sz @@ -11,7 +11,6 @@ types, traits, hygienic macros and bjoroutines, that compiles to C#. library module, made from the documentation the module was compiled with. @itemlist{ @item{@link["modules/std/"]{@code{std}}: the standard library.} - @item{@link["modules/text/"]{@code{text}}: data formats: bjodat, JSON - and XML.} + @item{@link["modules/text/"]{@code{text}}: data formats and encodings: bjodat, CSV, JSON and XML.} }} } From 93025b7b35ed641604693150b24b655a64d5b042140687a06813be84c707e3f1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Mon, 9 Nov 2026 16:32:49 +0100 Subject: [PATCH 31/38] Add module pages for combine, interop, parallel, csv and csv-codec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The standard library chapter links to these pages; add the generate-docs stubs so the links resolve. Ignore .netcoredbg_hist. ๐Ÿค– Generated with [ECA](https://eca.dev) Co-Authored-By: eca-agent --- .gitignore | 1 + modules/std/combine.sz | 3 +++ modules/std/interop.sz | 3 +++ modules/std/parallel.sz | 3 +++ modules/text/csv-codec.sz | 3 +++ modules/text/csv.sz | 3 +++ 6 files changed, 16 insertions(+) create mode 100644 modules/std/combine.sz create mode 100644 modules/std/interop.sz create mode 100644 modules/std/parallel.sz create mode 100644 modules/text/csv-codec.sz create mode 100644 modules/text/csv.sz diff --git a/.gitignore b/.gitignore index d0db8d0..76bdd28 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ _cache/ +.netcoredbg_hist diff --git a/modules/std/combine.sz b/modules/std/combine.sz new file mode 100644 index 0000000..39e95a8 --- /dev/null +++ b/modules/std/combine.sz @@ -0,0 +1,3 @@ +(meta (title "(std combine)")) + +@generate-docs[#:module "std combine"] diff --git a/modules/std/interop.sz b/modules/std/interop.sz new file mode 100644 index 0000000..82135c3 --- /dev/null +++ b/modules/std/interop.sz @@ -0,0 +1,3 @@ +(meta (title "(std interop)")) + +@generate-docs[#:module "std interop"] diff --git a/modules/std/parallel.sz b/modules/std/parallel.sz new file mode 100644 index 0000000..ffce03e --- /dev/null +++ b/modules/std/parallel.sz @@ -0,0 +1,3 @@ +(meta (title "(std parallel)")) + +@generate-docs[#:module "std parallel"] diff --git a/modules/text/csv-codec.sz b/modules/text/csv-codec.sz new file mode 100644 index 0000000..c14a329 --- /dev/null +++ b/modules/text/csv-codec.sz @@ -0,0 +1,3 @@ +(meta (title "(text csv-codec)")) + +@generate-docs[#:module "text csv-codec"] diff --git a/modules/text/csv.sz b/modules/text/csv.sz new file mode 100644 index 0000000..a66d4ea --- /dev/null +++ b/modules/text/csv.sz @@ -0,0 +1,3 @@ +(meta (title "(text csv)")) + +@generate-docs[#:module "text csv"] From 1a2a43bf5a2691c8a6d90a7e1919e82f0f9fda41b5afd6aa22ca7809d542e003 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 07:50:42 +0100 Subject: [PATCH 32/38] Document (std text) and (std unicode grapheme) Module pages for both, entries in the standard library chapter, and sections on grapheme clusters and on text for editing in the language basics. Co-Authored-By: Claude Opus 5.5 --- _01.Language-basics.sz | 67 +++++++++++++++++++++++++++++++++ _03.Standard-library.sz | 10 +++++ modules/std/text.sz | 3 ++ modules/std/unicode/grapheme.sz | 3 ++ modules/std/unicode/index.sz | 2 + 5 files changed, 85 insertions(+) create mode 100644 modules/std/text.sz create mode 100644 modules/std/unicode/grapheme.sz create mode 100644 modules/std/unicode/index.sz diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 67bdb3d..8bbf250 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1905,6 +1905,8 @@ A @code{string} is stored as UTF-8, and is always valid UTF-8. A @code{char} is (string-count s) ;; 10: characters, O(n). This is "how long is the text". }| +There is a third count too, of the characters a reader sees. An "รฉ" can be one code point, or an e followed by a combining accent, and a flag is two code points. @code{string-grapheme-count} counts those; see the section on grapheme clusters below. + For the same reason there is no @code{string-ref} that takes an index. Finding the n:th character of a UTF-8 string means walking it, so walking is what you do: with a @code{loop}, a fold, or a cursor. @codeblock[#:lang "bjolang"]|{ @@ -1984,6 +1986,31 @@ The folds walk a string for you, and are usually simpler than a cursor: (count-vowels "Hello World") ;; 3 }| +@subsubsection{Grapheme clusters} + +What a reader takes for one character can be several @code{char}s: a letter and its accents, a flag, an emoji with a skin tone or a family of emoji joined by zero-width joiners, a Hangul syllable written in jamo, and CR LF. Unicode calls these grapheme clusters, and an editor's arrow keys and backspace move by them. @code{(std unicode grapheme)} finds them by Unicode's rules (UAX #29): + +@codeblock[#:lang "bjolang"]|{ +(import (std unicode grapheme)) + +(def s "eฬ๐Ÿ‡ธ๐Ÿ‡ช\r\n") +(string-count s) ;; 6 +(string-grapheme-count s) ;; 3 +(string->graphemes s) ;; ["รฉ" "๐Ÿ‡ธ๐Ÿ‡ช" "\r\n"] + +;; Backspace: the string without its last cluster. +(: drop-last-grapheme (-> string string)) +(defun (drop-last-grapheme s) + (if (string-empty? s) + s + (substring s (string-cursor-start s) + (string-cursor-prev-grapheme s (string-cursor-end s))))) + +(drop-last-grapheme "ok ๐Ÿ‘๐Ÿฝ") ;; "ok ": the skin tone goes with the thumb +}| + +@code{string-cursor-next-grapheme} and @code{string-cursor-prev-grapheme} take a cursor that is on a cluster boundary: the start, the end, or a cursor that a grapheme function returned. @code{string-cursor-grapheme-boundary?} tells whether a cursor that came from elsewhere is on one. A cluster has no limit on its length, since a letter can carry any number of accents, but a loop over a string by clusters takes O(n) time in either direction, so a text written to be slow cannot make it quadratic. @code{(std text)} has the same functions for texts. + @subsubsection{Slices} @code{substring}, @code{string-trim} and @code{string-split} copy the text they answer. A slice is the same text without the copy: a string that shares the bytes of the string it was taken from. Only the forms ending in @code{/slice} make one: @@ -2149,6 +2176,46 @@ The @code{Rx} and @code{RxIn} patterns use a regex inside a @code{match}. @code{ Define patterns at the top level, as above, rather than writing @code{#rx(...)} inside a clause. Otherwise the regex is recompiled at every call, which will tank performance. +@subsubsection{Text for editing with (std text)} + +A string is immutable, and every edit copies all of it. For a large text that is edited in many places, such as an editor's buffer, @code{(std text)} has @code{Text}: a balanced tree of UTF-8 chunks, where inserting, deleting, splitting, appending and taking a substring take O(log n) time. An edit shares every chunk it does not change, so the old version stays valid and costs only what differs: undo is keeping the old texts. + +The API is a string's. The functions a string has are there with @code{text-} for @code{string-}, positions are cursors, and the cursor functions take the text and the cursor. @code{text-length} is in bytes, as @code{string-length} is, and @code{text-count} counts characters, in O(1) time here. The tree also counts newlines, so lines are found in O(log n) time. + +@codeblock[#:lang "bjolang"]|{ +(import (std text)) + +(def r (string->text "hello world\nsecond line")) +(def c (text-cursor-forward r (text-cursor-start r) 5)) ;; O(log n), however far +(def r2 (text-insert r c ",")) +(text->string r2) ;; "hello, world\nsecond line" +(text->string (text-line r2 1)) ;; "second line" +(text-cursor-line r2 (option-ref (text-search r2 "second"))) ;; 1 +(text->string r) ;; "hello world\nsecond line": unchanged +}| + +A cursor belongs to one text, and the text an edit returns needs cursors of its own. To keep a position across an edit, such as an editor's caret, keep its index: @code{text-cursor-index} gives it, and @code{text-cursor-forward} from the start of the new text finds it again. + +@codeblock[#:lang "bjolang"]|{ +(def caret (option-ref (text-search r2 "world"))) +(def i (text-cursor-index r2 caret)) ;; 7 +(def r3 (text-insert r2 (text-cursor-start r2) "> ")) +(def caret3 (text-cursor-forward r3 (text-cursor-start r3) (+ i 2))) ;; still on "world" +}| + +A run of edits close together, such as typing, is faster through a @code{TransientText}, which edits in place, takes character indices, and gives a text back with @code{transienttext->text}. A @code{TextBuilder}, or the @code{texting} collector of a loop, builds a text in one pass. And @code{text-cursor-next-grapheme} and @code{text-cursor-prev-grapheme} step by grapheme clusters, as the arrow keys do: + +@codeblock[#:lang "bjolang"]|{ +(def t (text->transienttext r)) +(transienttext-insert! t 5 ",") +(transienttext-insert! t 6 " my") +(text->string (transienttext->text t)) ;; "hello, my world\nsecond line" + +;; Backspace at the end of a text. +(def e (string->text "ok ๐Ÿ‘๐Ÿฝ")) +(text->string (text-delete e (text-cursor-prev-grapheme e (text-cursor-end e)) (text-cursor-end e))) ;; "ok " +}| + @subsection{Traits} A trait is a set of functions that a type can implement. It is how bjolang does what other languages do with interfaces, type classes or overloading: @code{=} is a trait method, so is @code{displayed}, and so are @code{length}, @code{fold} and @code{compare}. diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 9f721ab..3c37432 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -47,6 +47,16 @@ For more details, see @link["modules/std/rx.html"]{@code{(std rx)}}. Reading a line by a pattern, after OCaml's @code{Scanf}. The pattern is a quoted list of what the line holds, in order: @code{(scan "3-17" '(long "-" long))} is 3 and 17, and @code{'(int "x" int ": " int ... #:sep " ")} reads @code{"12x5: 1 0 3"} as two ints and a vec of them. There are items for numbers, a character, a word, the rest of the line, a run of given characters and the text up to one, with a width, a skip, a repetition, an option and a choice. The type of the result comes from the pattern, so @code{'(int "," int)} gives a @code{(Tuple int int)}. The macro compiles the pattern into straight code over the line's bytes, and a pattern of numbers and strings allocates nothing. The whole line must match. @code{try-scan} gives an @code{Err} that says at which column it failed, instead of raising. A scan does not search and does not backtrack: use @code{(std rx)} to find text in a line. For more details, see @link["modules/std/scan.html"]{@code{(std scan)}}. +@subsection{std/text} + +Text that is cheap to edit anywhere, for an editor's buffer or any large text that changes in many places. A @code{Text} is a balanced tree of UTF-8 chunks: inserting, deleting, splitting, appending, taking a substring and finding a line take O(log n) time, and an edit shares everything it does not change, so old versions stay valid for undo. The API is a string's, with @code{text-} for @code{string-}: positions are cursors, the cursor functions take the text and the cursor, and no function takes a byte offset. @code{text-count} is O(1), @code{text-cursor-forward} moves any number of characters in O(log n), and @code{text-cursor-index} gives a cursor's index to carry across an edit. A @code{TransientText} edits in place for a run of edits such as typing, and @code{TextBuilder} and the @code{texting} collector build a text in one pass. @code{text-cursor-next-grapheme} and the others step by grapheme clusters. +For more details, see @link["modules/std/text.html"]{@code{(std text)}}. + +@subsection{std/unicode/grapheme} + +Grapheme clusters, the characters a reader sees, by Unicode's rules (UAX #29): a letter and its accents, a flag, an emoji sequence, CR LF. @code{string-cursor-next-grapheme} and @code{string-cursor-prev-grapheme} move a cursor by one, as an editor's arrow keys and backspace do, and @code{string-grapheme-count} and @code{string->graphemes} count and split. A cluster has no limit on its length, but a loop by clusters takes O(n) time in either direction, so long runs of accents or flags cannot make it quadratic. The rules are two state machines over characters, which other containers can drive with their own cursors: @code{(std text)} does, for texts. +For more details, see @link["modules/std/unicode/grapheme.html"]{@code{(std unicode grapheme)}}. + @subsection{std/http} The @code{std/http} module allows you to build and send HTTP requests. Sending a request three ways โ€” blocking, suspending, or as an event โ€” is the only thing that differs between the entry points. There is no C# in this module: @code{System.Net.Http} is reached with @code{import/extern}, and the response body is an ordinary @code{TextInputPort}. diff --git a/modules/std/text.sz b/modules/std/text.sz new file mode 100644 index 0000000..4e95825 --- /dev/null +++ b/modules/std/text.sz @@ -0,0 +1,3 @@ +(meta (title "(std text)")) + +@generate-docs[#:module "std text"] diff --git a/modules/std/unicode/grapheme.sz b/modules/std/unicode/grapheme.sz new file mode 100644 index 0000000..8f421bd --- /dev/null +++ b/modules/std/unicode/grapheme.sz @@ -0,0 +1,3 @@ +(meta (title "(std unicode grapheme)")) + +@generate-docs[#:module "std unicode grapheme"] diff --git a/modules/std/unicode/index.sz b/modules/std/unicode/index.sz new file mode 100644 index 0000000..59caf18 --- /dev/null +++ b/modules/std/unicode/index.sz @@ -0,0 +1,2 @@ +(meta (title "(std unicode)") + (sort title)) From f07cd79e3e0aa27f0a3da37e00b9acabac9e89734227aa4842b87958adb33ccd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 07:50:51 +0100 Subject: [PATCH 33/38] Document (std os), numeric/si and Ctrl-C and SIGTERM A module page and a standard library entry for (std os), numeric/si and a section on signals in writing programs, and the span conversions in the language basics. --- _01.Language-basics.sz | 2 +- _02_.Writing-programs.sz | 87 ++++++++++++++++++++++++++++++++++++++-- _03.Standard-library.sz | 9 +++++ modules/std/os.sz | 3 ++ 4 files changed, 96 insertions(+), 5 deletions(-) create mode 100644 modules/std/os.sz diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 8bbf250..b01aaa6 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1342,7 +1342,7 @@ To work on part of an array without copying it, make a span: @code{(Span %a)} is (span->array s) ;; #[99 3], a fresh copy }| -A span is valid only while the call that made it runs, so it can be a local variable, a function argument or a return value, and the compiler refuses it anywhere it could outlive the call (such as inside another collection or across a yield point). The section on .NET interop in Part 2 covers the full rules and how spans work with .NET methods. +A span is valid only while the call that made it runs, so it can be a local variable, a function argument or a return value, and the compiler refuses it anywhere it could outlive the call (such as inside another collection or across a yield point). The section on .NET interop in Part 2 covers the full rules and how spans work with .NET methods. It also covers @code{span->} and @code{ro-span->}, which show a span of numbers as numbers of another type (longs as bytes, for example) without copying. @subsubsection{Maps} diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 0f16140..4a2a057 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -671,7 +671,7 @@ Anything else is converted first: |-----------------------------------+---------------------------------------------------------------| | A double @code{x} | @code{(numeric x)}, or @code{(numeric x #:precision 2)} | | A bool @code{b} | @code{(if b "yes" "no")} | -| A long @code{n} | @code{(long->string n)}, or @code{(bytes n)} for a size | +| A long @code{n} | @code{(numeric n)}, or @code{(bytes n)} for a size | | A @code{(Vec string)} @code{xs} | @code{(vec-map BStr xs)} | | A @code{(List %a)} @code{xs} | @code{(joined f xs ", ")}, where @code{f} makes the block of one element | } @@ -784,7 +784,7 @@ A list is written with @code{joined}, which takes a function that makes each ele @subsubsection{Numbers} -An int is a block. A double is not, as there is no one way to write one: @code{numeric} says how, with a precision, digits in groups, and the separators. @code{bytes} and @code{percent} are blocks for two common kinds of number. +An int is a block. A double is not, as there is no one way to write one: @code{numeric} says how, with a precision, digits in groups, and the separators. It takes a number of any type, and writes a long, a @code{decimal} or a @code{bigint} with all its digits. @code{bytes} and @code{percent} are blocks for two common kinds of number. @codeblock[#:lang "bjolang"]|{ (show [(numeric 1234567.891 #:precision 2 #:comma 3) " " @@ -797,9 +797,18 @@ An int is a block. A double is not, as there is no one way to write one: @code{n The digits are always the invariant culture's, so the machine's locale never shows. The separators are settings, described below. +@code{numeric/si} writes a number with an SI prefix before its unit, as SRFI 166 does. With @code{#:binary #t} the prefixes are powers of 1024, and @code{#:sep} is what goes between the number and the prefix. + +@codeblock[#:lang "bjolang"]|{ +(show [(numeric/si 0.00000123 "m" #:sep " ") " " + (numeric/si 2400000000.0 "Hz" #:sep " ") " " + (numeric/si 3986.0 "B" #:binary #t) nl]) +;; 1.2 ยตm 2.4 GHz 3.9KiB +}| + @subsubsection{Paragraphs} -@code{wrapped} fills content to @code{current-width}, and @code{justified} makes it flush against both margins. Every run of whitespace in the content is a break between words. +@code{wrapped} fills content to @code{current-width}, and @code{justified} makes it flush against both margins. Every run of whitespace in the content is a break between words, but a no-break space is not, and a word wider than the line gets a line of its own. @code{current-word-separator?} says which characters separate words. @code{wrapped/char}, as in SRFI 166, breaks at characters instead: also inside a word, and the content's own newlines are kept. It is for text with no spaces to break at, such as a hash or a URL. @codeblock[#:lang "bjolang"]|{ (show [(with-fmt current-width 20 @@ -935,7 +944,7 @@ For part of an output, such as one paragraph, one cell or the numbers of one col ;; 1 234 567,5 }| -The settings are @code{current-pad}, @code{current-width}, @code{current-ellipsis}, @code{current-string-width}, @code{current-precision}, @code{current-comma}, @code{current-comma-sep}, @code{current-decimal-sep} and @code{current-signed}. For text with wide characters or colours in it, which a terminal shows in more or fewer columns than it has characters, set @code{current-string-width} to @code{terminal-width}. +The settings are @code{current-pad}, @code{current-width}, @code{current-ellipsis}, @code{current-string-width}, @code{current-precision}, @code{current-comma}, @code{current-comma-sep}, @code{current-decimal-sep}, @code{current-signed} and @code{current-word-separator?}. For text with wide characters or colours in it, which a terminal shows in more or fewer columns than it has characters, set @code{current-string-width} to @code{terminal-width}. @subsubsection{When it does not compile} @@ -1199,6 +1208,50 @@ A scope also owns resources. @code{(own! thunk)} registers a release on the inne Do not park a bjoroutine's thread by accident. A scope in an ordinary @code{(fun ...)} that a bjoroutine calls parks the thread of that bjoroutine, and the compiler warns about it. Write the scope in the bjoroutine itself. Or, if the function the lambda is given to takes a @code{-?->} parameter, write the lambda as a @code{(bjoroutine ...)}. +@subsubsection{Ctrl-C and SIGTERM} + +By default, Ctrl-C and SIGTERM end a program at once. No @code{finally} runs, and no scope releases what it owns. @code{with-signal-cancel} from @code{(std os)} is a scope that the first of its signals cancels. The fibers stop at their next wait, the scope releases what it owns, and the form returns @code{(Err s)}, with the signal. When the body returns @code{v}, the form returns @code{(Ok v)}: + +@codeblock[#:lang "bjolang"]|{ +(import (std os)) + +(: tick (-bjo-> void)) +(defbjo (tick) + (let go () + (sync (timeout (seconds 1))) + (println "tick") + (go))) + +(defbjo (main) + (match (with-signal-cancel [Interrupt Terminate] + (ignore (own! (fun () (println "released")))) + (spawn (tick)) + (sync (timeout (minutes 10)))) + ((Ok _) 0) + ((Err s) (println #"stopped by ${(signal-name s)}") 0))) +}| + +@codeblock[#:lang "text"]|{ +tick +tick +^C +released +stopped by SIGINT +}| + +At the first signal, the form gives the signals back to the system. So when the cleanup hangs, a second Ctrl-C ends the program at once. + +To do something else on a signal and go on, give @code{on-signal} a function. It calls the function for each signal that comes, in a fiber of its own, until the scope ends; in @code{main}, that is the whole program. The function can be an ordinary one or a bjoroutine: + +@codeblock[#:lang "bjolang"]|{ +(defun (main) + (on-signal [Interrupt] (fun (s) (println #"got ${(signal-name s)}, carrying on"))) + (do-the-work) + 0) +}| + +The function runs at the same time as the rest of the program, so share state with it through @code{(std atomic)} or a channel. To receive signals as values in a @code{choose}, use @code{signal-subscribe} and the event @code{signal-recv}. + @subsubsection{When fibers hang} A scope does not return until its fibers have ended, so a fiber that waits forever makes its scope wait forever too. Two things show you where the fibers wait. @@ -1723,12 +1776,38 @@ A span that a .NET member @em{returns} is not an array. To copy a returned span | @code{(span-slice s #:from #:to)} | @code{(ro-span-slice s #:from #:to)} | a sub-view of @code{s}; nothing is copied | | @code{(span->array s)} | @code{(ro-span->array s)} | a fresh array copy of the span's elements | | @code{(span->ro-span s)} | | the same view, read-only | +| @code{(span-> s)} | @code{(ro-span-> s)} | the same bytes as numbers of another type | } @code{#:from} defaults to @code{0} and @code{#:to} defaults to the length. Neither span type converts to the other implicitly; use @code{span->ro-span} when a @code{RoSpan} is needed. In @code{(std mutable vec)}, @code{(mutablevec-span v start n)} returns a @code{(Span %a)} over @code{n} elements of a @code{MutableVec}'s backing storage (valid until the vector's length changes), which pairs with @code{(mutablevec-grow! v n)} to fill a vector a block at a time. You can also call a span's .NET members directly: @code{(.-Length s)}, @code{(.Slice s 1)}, @code{(.Fill s x)}, @code{(.get_Item s i)} for @code{s[i]}, and @code{(.set_Item s i x)} for @code{s[i] = x} (an indexer returning @code{ref T} reads a copy of the element, and @code{.set_Item} writes through the reference). +@strong{Numbers as other numbers.} @code{span->} and @code{ro-span->} show the memory of a span as numbers of another type, with no copy. This is what you need when a .NET API wants bytes and you have longs, or the other way around: an Arrow buffer, a file format, a hash. The type of the result comes from where you use it, so give it with a signature or @code{(: ...)}: + +@codeblock[#:lang "bjolang"]|{ +(def bytes (: (make-array 16) (Array byte))) +(: (Span long)) +(def longs (span-> (array->span bytes))) +(span-set! longs 1 -1L) +(array-ref bytes 8) ;; 255uy: the longs are the bytes + +(: (RoSpan byte)) +(def raw (ro-span-> (array->ro-span #[1.5 2.5]))) +(ro-span-length raw) ;; 16 +(ro-span->array raw) ;; the doubles' bytes, copied once +}| + +The new length is the length in bytes divided by the size of the new element. Writing through one span writes through the other, and the bytes are in the machine's order, which is little-endian on x64 and ARM64. Three rules keep this safe: + +@read-list{ +- @strong{Only plain numbers.} Both types must be integers, @code{float} or @code{double}. They contain no references, and every bit pattern is a valid value, so nothing you write as one type can break the other. @code{bool} and @code{char} are not numbers, so a span of them is a type error. A span of @code{decimal} or @code{bigint} type-checks, because they are numbers, but they are not plain, so it panics. +- @strong{Whole elements.} If the length in bytes is not a whole number of the new elements (7 bytes as longs, for example), the call panics. It does not silently cut off the end. Slice the span first. +- @strong{Read-only stays read-only.} @code{ro-span->} gives a @code{RoSpan} and there is no way to make it writable. A string's bytes are only available as a @code{(RoSpan byte)}, so nothing can change them. For the same reason, a program cannot name @code{System.Runtime.InteropServices.MemoryMarshal}, @code{SequenceMarshal} or @code{System.Runtime.CompilerServices.Unsafe}. +} + +@strong{What a span is made from stays alive.} A span over memory that a .NET object owns (an Arrow buffer, a @code{Memory}) holds no reference to that object. If nothing else used the object, the garbage collector could free the memory while you were still reading the span. The compiler keeps the object alive for you: a span in a local variable keeps it alive until the end of the variable's scope, even on a return, a loop jump or an exception. A span passed to a call keeps it alive until the call returns. An array needs none of this, because a span over an array keeps the array. + @strong{Where a span may not go.} A spanโ€”or any other .NET @code{ref struct}โ€”is a view of memory that is valid only while the call that made it runs. It can be a local variable, a parameter, or a return value of a function called by name, and the compiler refuses it everywhere it could outlive the call: @read-list{ diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 3c37432..542785b 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -142,6 +142,15 @@ For more details, see @link["modules/std/inbox.html"]{@code{(std inbox)}}. Bridges for .NET APIs that want something a Bjolang function or value is not. A Bjolang function is a @code{Func} whose @code{void} return is the unit value; @code{as-delegate} and @code{as-delegate/type} wrap a function in another .NET delegate type such as @code{Action}, @code{EventHandler} or a library's custom delegate. @code{add-event-handler!} and @code{remove-event-handler!} subscribe a Bjolang function to a .NET object's event by name. @code{clr-default} produces .NET's @code{default(T)} (such as @code{null} for a reference type or a @code{Nullable}) when a .NET call requires one. For more details, see @link["modules/std/interop.html"]{@code{(std interop)}}. +@subsection{std/os} + +The operating system the program runs on: @code{windows?}, @code{linux?}, @code{macos?}, @code{freebsd?} and the others, @code{os-platform?} by name, and @code{windows-version-at-least?} and the other version checks, with the minor version, the build and the revision as keywords. Each is .NET's @code{OperatingSystem} method, called where it is written. .NET's core library is built for one system, so the method returns a constant there, and the JIT removes the branch that cannot run. A program is built once and chooses when it runs, and code for one system can be in a module of its own: a module that nothing calls does nothing. + +It also has the facts about the machine and the process: @code{os-description}, @code{os-version}, @code{os-architecture}, @code{processor-count}, @code{available-memory}, @code{machine-name}, @code{user-name}, @code{process-id} and others. And the user's folders: @code{home-directory}, @code{config-directory}, @code{data-directory}, @code{cache-directory}, @code{documents-directory}, @code{desktop-directory} and @code{temp-directory}, which follow the XDG variables on Linux and each system's own place elsewhere, and @code{make-temp-directory} for a new directory that only its owner can enter. + +And the signals, as union cases such as @code{Interrupt}, @code{Terminate} and @code{Hangup}. @code{with-signal-cancel} is a scope that the first of its signals cancels, so that a program cleans up on Ctrl-C or SIGTERM; a second signal ends the program at once. @code{on-signal} calls a function for each signal that comes, until the scope ends. @code{signal-subscribe} gives signals as values from the event @code{signal-recv}, and signals of one kind that wait are merged into one. +For more details, see @link["modules/std/os.html"]{@code{(std os)}}. + @subsection{std/ports and std/net} @code{(std ports)} reads and writes bytes on the prelude's ports. The byte reads @code{read-some-evt}, @code{read-bytes-evt} and @code{eof-evt} are CML events: a read that loses its @code{choose} takes nothing, so a read can race a timeout. @code{read-line} and @code{read-bytes} can be used on the same port. The module also drains ports into lists and vecs, streams them as a @code{Seq} or a channel, and has @code{byte-pipe}, a pair of ports with no file and no network, for tests. @code{(std net)} gives a TCP connection as a pair of these ports, so a protocol written for ports also runs over TCP. @code{accept-evt} is an event too: a branch that loses accepts nothing. diff --git a/modules/std/os.sz b/modules/std/os.sz new file mode 100644 index 0000000..b904a08 --- /dev/null +++ b/modules/std/os.sz @@ -0,0 +1,3 @@ +(meta (title "(std os)")) + +@generate-docs[#:module "std os"] From 902ff84662cf39d044ada83c8b7d8ccf3b82bfbfc9ee19dfb3089f9e471ca894 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 11:26:01 +0100 Subject: [PATCH 34/38] Document string escapes, including \u{...} Co-Authored-By: Claude Opus 5.5 --- _01.Language-basics.sz | 2 ++ 1 file changed, 2 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index b01aaa6..e3f3e96 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1917,6 +1917,8 @@ For the same reason there is no @code{string-ref} that takes an index. Finding t Characters are written @code{#\a}, @code{#\space}, @code{#\newline}. They can be compared with @code{<} and friends or @code{char=?}, converted with @code{char->int} and @code{int->char}, and classified with @code{char-alphabetic?}, @code{char-numeric?}, @code{char-whitespace?}, @code{char-upper-case?} and @code{char-lower-case?}. +In a string, a backslash begins an escape: @code{\n}, @code{\t}, @code{\r}, @code{\"} and @code{\\}, and @code{\u{...}}, a code point in 1 to 6 hex digits, as in @code{"caf\u{E9}"} or @code{"\u{1F600}"}. An interpolated string also has @code{\$}, a dollar sign that opens no hole. Any other escape is an error, so a backslash is written @code{\\}, or the text is written as a raw string, @code{"""..."""}, which has no escapes. A backslash at the end of a line continues the string on the next. A character can be written by its code point as well, as @code{#\xE9}. + @subsubsection{Common operations} @codeblock[#:lang "bjolang"]|{ From dc629fbd3e7279fa3249db5603297f405280ea49907ed43ce27241963f1baf3c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 12:08:33 +0100 Subject: [PATCH 35/38] finding: a collector that ends the loop Co-Authored-By: Claude Opus 5.5 --- _01.Language-basics.sz | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index e3f3e96..987da21 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1587,10 +1587,20 @@ An accumulator names a collector and what to feed it: | @code{(summing x)} | the sum of @code{x} | | @code{(counting x)} | how many rounds reached it | | @code{(folding init expr)} | starts at @code{init}; each round, @code{expr} is the new value | +| @code{(finding x)} | @code{(Some x)} for the first round that reaches it, which ends the loop, or @code{None} | } @code{folding} is the general one: inside @code{expr} the accumulator's own name is the value so far. +@code{finding} stops the loop: the round that finds is the last, and the clauses after it in that round do not run. With @code{#:when}, or a loose @code{:when} in a comprehension, it finds the first value the condition holds for. + +@codeblock[#:lang "bjolang"]|{ +{finding x (:for x [3 5 8 9]) (:when (even? x))} ;; (Some 8) +{finding x :when (> x 100) (:for x [3 5 8 9])} ;; None +}| + +It is a collector macro, written with @code{def/acc-clause}: a macro at the head of an accumulator's collector, which returns loop clauses that go where the @code{(:acc ...)} was. @code{finding} returns a @code{folding} and a @code{(:finish ...)}. + @codeblock[#:lang "bjolang"]|{ (def xs [3 9 2 7]) From 7d1cc99efe8288de8570d15f72619d54ffd98b64253e1cf9b4573997d4e20fbe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 12:46:25 +0100 Subject: [PATCH 36/38] The new collectors in the accumulator table Co-Authored-By: Claude Opus 5.5 --- _01.Language-basics.sz | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 987da21..67d96bf 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1588,15 +1588,25 @@ An accumulator names a collector and what to feed it: | @code{(counting x)} | how many rounds reached it | | @code{(folding init expr)} | starts at @code{init}; each round, @code{expr} is the new value | | @code{(finding x)} | @code{(Some x)} for the first round that reaches it, which ends the loop, or @code{None} | +| @code{(anding test)} | whether @code{test} holds in every round; the first that fails ends the loop | +| @code{(oring test)} | whether @code{test} holds in some round; the first that holds ends the loop | +| @code{(taking n x)} | a @code{List} of the first @code{n}; the @code{n}th ends the loop | +| @code{(maximizing x)} | @code{(Some x)} of the greatest, by @code{Ord}, or @code{None} (and @code{minimizing}) | +| @code{(maximizing-by k x)} | @code{(Some x)} of the greatest key @code{k}, or @code{None} (and @code{minimizing-by}) | +| @code{(tallying x)} | a @code{Map} from each value to how many times it came | +| @code{(grouping k x)} | a @code{Map} from each key @code{k} to the @code{List} of its values | } @code{folding} is the general one: inside @code{expr} the accumulator's own name is the value so far. -@code{finding} stops the loop: the round that finds is the last, and the clauses after it in that round do not run. With @code{#:when}, or a loose @code{:when} in a comprehension, it finds the first value the condition holds for. +@code{finding}, @code{anding}, @code{oring} and @code{taking} stop the loop: the round that decides is the last, and the clauses after it in that round do not run. With @code{#:when}, or a loose @code{:when} in a comprehension, it finds the first value the condition holds for. @codeblock[#:lang "bjolang"]|{ {finding x (:for x [3 5 8 9]) (:when (even? x))} ;; (Some 8) {finding x :when (> x 100) (:for x [3 5 8 9])} ;; None +{anding (odd? x) (:for x [3 5 8 9])} ;; #f, after the round of 8 +{maximizing-by (string-length w) w (:for w (list "ab" "cde" "f"))} ;; (Some "cde") +{grouping (% x 2) x (:for x [3 5 8 9])} ;; a map from 0 to (8) and 1 to (3 5 9) }| It is a collector macro, written with @code{def/acc-clause}: a macro at the head of an accumulator's collector, which returns loop clauses that go where the @code{(:acc ...)} was. @code{finding} returns a @code{folding} and a @code{(:finish ...)}. From 6634469ae805063cad62fef8ef0ab79fd8e10069132750ecffd715964e423ba8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 13:23:06 +0100 Subject: [PATCH 37/38] someing in the accumulator table Co-Authored-By: Claude Opus 5.5 --- _01.Language-basics.sz | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 67d96bf..603f162 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1588,6 +1588,7 @@ An accumulator names a collector and what to feed it: | @code{(counting x)} | how many rounds reached it | | @code{(folding init expr)} | starts at @code{init}; each round, @code{expr} is the new value | | @code{(finding x)} | @code{(Some x)} for the first round that reaches it, which ends the loop, or @code{None} | +| @code{(someing e)} | the first @code{(Some v)} the @code{Option} @code{e} gives, which ends the loop, or @code{None} | | @code{(anding test)} | whether @code{test} holds in every round; the first that fails ends the loop | | @code{(oring test)} | whether @code{test} holds in some round; the first that holds ends the loop | | @code{(taking n x)} | a @code{List} of the first @code{n}; the @code{n}th ends the loop | @@ -1599,17 +1600,18 @@ An accumulator names a collector and what to feed it: @code{folding} is the general one: inside @code{expr} the accumulator's own name is the value so far. -@code{finding}, @code{anding}, @code{oring} and @code{taking} stop the loop: the round that decides is the last, and the clauses after it in that round do not run. With @code{#:when}, or a loose @code{:when} in a comprehension, it finds the first value the condition holds for. +@code{finding}, @code{someing}, @code{anding}, @code{oring} and @code{taking} stop the loop: the round that decides is the last, and the clauses after it in that round do not run. With @code{#:when}, or a loose @code{:when} in a comprehension, it finds the first value the condition holds for. @codeblock[#:lang "bjolang"]|{ {finding x (:for x [3 5 8 9]) (:when (even? x))} ;; (Some 8) {finding x :when (> x 100) (:for x [3 5 8 9])} ;; None +{someing (map-try-ref ages n) (:for n names)} ;; the age of the first name in ages, or None {anding (odd? x) (:for x [3 5 8 9])} ;; #f, after the round of 8 {maximizing-by (string-length w) w (:for w (list "ab" "cde" "f"))} ;; (Some "cde") {grouping (% x 2) x (:for x [3 5 8 9])} ;; a map from 0 to (8) and 1 to (3 5 9) }| -It is a collector macro, written with @code{def/acc-clause}: a macro at the head of an accumulator's collector, which returns loop clauses that go where the @code{(:acc ...)} was. @code{finding} returns a @code{folding} and a @code{(:finish ...)}. +It is a collector macro, written with @code{def/acc-clause}: a macro at the head of an accumulator's collector, which returns loop clauses that go where the @code{(:acc ...)} was. @code{(finding x)} returns @code{(someing (Some x))}, and @code{someing} returns a @code{folding} and a @code{(:finish ...)}. @codeblock[#:lang "bjolang"]|{ (def xs [3 9 2 7]) From 5e072834047ad1e189829ce65531d92a213655ad86599f29fcf4d674ca2e1881 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Sun, 15 Nov 2026 21:59:22 +0100 Subject: [PATCH 38/38] Document (std stream), port->stream and run/stream Co-Authored-By: Claude Opus 5.5 --- _01.Language-basics.sz | 2 +- _02_.Writing-programs.sz | 62 ++++++++++++++++++++++++++++++++++++++-- _03.Standard-library.sz | 9 ++++-- modules/std/stream.sz | 3 ++ 4 files changed, 71 insertions(+), 5 deletions(-) create mode 100644 modules/std/stream.sz diff --git a/_01.Language-basics.sz b/_01.Language-basics.sz index 603f162..8697086 100644 --- a/_01.Language-basics.sz +++ b/_01.Language-basics.sz @@ -1530,7 +1530,7 @@ There are three ways to repeat something in bjolang. A named @code{let} (or any @subsubsection{loop} -A @code{loop} is a list of clauses that run in order, once per element. @code{(:for x coll)} walks anything that can be walked: lists, vecs, arrays, maps, strings, ranges and sequences. An accumulator collects a result, and @code{=> expr} says what the loop returns: +A @code{loop} is a list of clauses that run in order, once per element. @code{(:for x coll)} walks anything that can be walked: lists, vecs, arrays, maps, strings, ranges, sequences, and streams, whose values come from work that suspends (Part 2, Concurrency). An accumulator collects a result, and @code{=> expr} says what the loop returns: @codeblock[#:lang "bjolang"]|{ (loop (:for x (list 1 2 3 4 5 6)) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index 4a2a057..033cecf 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -580,7 +580,7 @@ To avoid leaking resources, you will usually want to use @code{call-with-input-f The variants @code{read-line} and @code{read-char} (without the @code{/opt} suffix) also exist, but throw an end-of-file exception if called at EOF. -Convenience helpers in @code{(std ports)} can ingest an entire port into collections: @code{port->lines}, @code{port->list}, @code{port->vec}, as well as lazy sequences via @code{port->seq} and @code{file->seq}: +Convenience helpers in @code{(std ports)} can ingest an entire port into collections: @code{port->lines}, @code{port->list}, @code{port->vec}, as well as lazy sequences via @code{port->seq} and @code{file->seq}. In a bjoroutine, @code{port->stream} is the lazy one, a @code{Stream} (see Streams, below): @codeblock[#:lang "bjolang"]|{ (import (std ports)) @@ -618,7 +618,7 @@ Prefixing an identifier with a comma (e.g. @code{,who}) splices the variable's v | @code{(run form)} | A running @code{Proc} handle with streaming I/O ports | } -All of these functions return a @code{Result} and execute synchronously, blocking the calling thread until completion. When running inside a bjoroutine, non-blocking asynchronous alternatives are available; refer to the @code{(std run)} module documentation for details. +All of these functions return a @code{Result} and execute synchronously, blocking the calling thread until completion. When running inside a bjoroutine, non-blocking asynchronous alternatives are available: @code{run/stream} gives the output as a @code{Stream} that a loop reads as it comes (see Streams, below), and the @code{(std run)} module documentation has the rest. @subsection{Formatting text} @@ -1208,6 +1208,64 @@ A scope also owns resources. @code{(own! thunk)} registers a release on the inne Do not park a bjoroutine's thread by accident. A scope in an ordinary @code{(fun ...)} that a bjoroutine calls parks the thread of that bjoroutine, and the compiler warns about it. Write the scope in the bjoroutine itself. Or, if the function the lambda is given to takes a @code{-?->} parameter, write the lambda as a @code{(bjoroutine ...)}. +@subsubsection{Streams: loops over work that suspends} + +A @code{Seq} cannot suspend: it is a .NET iterator, and an iterator cannot also await. A @code{Stream}, from @code{(std stream)}, is the sequence for a bjoroutine. It is a driver, @code{(:for x s)}, in a loop or a comprehension. Its producer is a bjoroutine that hands each value to the loop with @code{emit!}, so the producer can wait for anything: + +@codeblock[#:lang "bjolang"]|{ +(import (std stream)) + +(: ticks (-> int (Stream int))) +(defun (ticks n) + (make-stream (bjoroutine (out) + (loop (:for i (range 0 n)) + (:do (sync (timeout (milliseconds 100))) + (println #"tick ${i}") + (emit! out i)))))) + +(defbjo (main) + (println {listing t (:for t (ticks 3))}) + (println {taking 2 t (:for t (ticks 1000))}) + 0) +}| + +@codeblock[#:lang "text"]|{ +tick 0 +tick 1 +tick 2 +(0 1 2) +tick 0 +tick 1 +(0 1) +}| + +A stream is a recipe: holding one runs nothing. Each loop over it starts the producer in a fiber of its own, and the producer computes a value only when the loop asks for it. That is why the second loop prints two ticks and not three. The loop stops the producer however it ends: when the producer returns, at @code{taking} or @code{finding}, at a @code{:finish}, when a driver beside it runs out, or at a raise or a @code{ret}. The stopped producer raises @code{Cancelled} where it waits, so its own releases run. What the producer raises for its own reasons is raised in the loop. @code{{streaming expr clause ...}} writes a producer as a comprehension, @code{{streaming (fetch url) (:for url urls)}}, the @code{seqing} of work that suspends. + +Ports and programs come as streams too. @code{port->stream} in @code{(std ports)} reads one item at a time, so a loop left early leaves the port just after the last item it took. @code{run/stream} in @code{(std run)} runs a program for each loop and kills it when the loop ends: + +@codeblock[#:lang "bjolang"]|{ +(def p (open-input-string "Host: example.com\nAccept: */*\n\nthe body\n")) +(def headers + (loop (:for line (port->stream read-line p)) + (:until (string-empty? line)) + (:acc hs (listing line)) + => hs)) +(println headers) ;; (Host: example.com Accept: */*) +(println (read-all p)) ;; the body + +{taking 2 line (:for line (run/stream '(seq 1 1000000) read-line))} ;; (1 2), and seq is killed +}| + +A channel that something else fills is read with @code{in-chan}. A channel has no close, so @code{in-chan} also takes the event that ends the reading, such as the producer's @code{promise-join}. A send is a rendezvous, so a producer that has returned has handed over every value it sent. This loop does not own the producer: leaving it early leaves the rest on the channel for another reader. + +@codeblock[#:lang "bjolang"]|{ +(def jobs (make-chan)) +(def feeder (bjo (feed jobs 4))) ;; sends 0 1 4 9 +{listing x (:for x (in-chan jobs (promise-join feeder)))} ;; (0 1 4 9) +}| + +In a bjoroutine each step of these loops suspends. In ordinary code it parks the thread, as @code{sync/blocking} does. + @subsubsection{Ctrl-C and SIGTERM} By default, Ctrl-C and SIGTERM end a program at once. No @code{finally} runs, and no scope releases what it owns. @code{with-signal-cancel} from @code{(std os)} is a scope that the first of its signals cancels. The fibers stop at their next wait, the scope releases what it owns, and the form returns @code{(Err s)}, with the signal. When the body returns @code{v}, the form returns @code{(Ok v)}: diff --git a/_03.Standard-library.sz b/_03.Standard-library.sz index 542785b..aa4891c 100644 --- a/_03.Standard-library.sz +++ b/_03.Standard-library.sz @@ -69,7 +69,7 @@ For more details, see @link["modules/std/url.html"]{@code{(std url)}}. @subsection{std/run} -The @code{std/run} module provides a way to run external programs. A form is one thing to run, written as a quoted literal, like @code{'(cat "hej.txt")}. You can connect programs in a pipe with @code{'(pipe (cat "a") (wc -l))}, and redirect input and output to files. Redirections can nest, and you can even splice computed arguments or write stages in Bjolang. +The @code{std/run} module provides a way to run external programs. A form is one thing to run, written as a quoted literal, like @code{'(cat "hej.txt")}. You can connect programs in a pipe with @code{'(pipe (cat "a") (wc -l))}, and redirect input and output to files. Redirections can nest, and you can even splice computed arguments or write stages in Bjolang. @code{run/stream} gives a program's output as a @code{Stream}: each loop over it runs the program, and the program is killed however the loop ends. For more details, see @link["modules/std/run.html"]{@code{(std run)}}. @subsection{std/random} @@ -151,9 +151,14 @@ It also has the facts about the machine and the process: @code{os-description}, And the signals, as union cases such as @code{Interrupt}, @code{Terminate} and @code{Hangup}. @code{with-signal-cancel} is a scope that the first of its signals cancels, so that a program cleans up on Ctrl-C or SIGTERM; a second signal ends the program at once. @code{on-signal} calls a function for each signal that comes, until the scope ends. @code{signal-subscribe} gives signals as values from the event @code{signal-recv}, and signals of one kind that wait are merged into one. For more details, see @link["modules/std/os.html"]{@code{(std os)}}. +@subsection{std/stream} + +Loops over values that suspending work produces. A @code{Stream} is a driver, @code{(:for x s)}, whose producer is a bjoroutine that hands each value to the loop with @code{emit!}. Each loop over it starts the producer, which computes a value only when the loop asks for it, and stops it however the loop ends. @code{make-stream} makes one from a bjoroutine, and @code{{streaming expr clause ...}} from loop clauses. @code{in-chan} reads a channel that something else fills, until an event such as the producer's @code{promise-join} commits. +For more details, see @link["modules/std/stream.html"]{@code{(std stream)}}. + @subsection{std/ports and std/net} -@code{(std ports)} reads and writes bytes on the prelude's ports. The byte reads @code{read-some-evt}, @code{read-bytes-evt} and @code{eof-evt} are CML events: a read that loses its @code{choose} takes nothing, so a read can race a timeout. @code{read-line} and @code{read-bytes} can be used on the same port. The module also drains ports into lists and vecs, streams them as a @code{Seq} or a channel, and has @code{byte-pipe}, a pair of ports with no file and no network, for tests. @code{(std net)} gives a TCP connection as a pair of these ports, so a protocol written for ports also runs over TCP. @code{accept-evt} is an event too: a branch that loses accepts nothing. +@code{(std ports)} reads and writes bytes on the prelude's ports. The byte reads @code{read-some-evt}, @code{read-bytes-evt} and @code{eof-evt} are CML events: a read that loses its @code{choose} takes nothing, so a read can race a timeout. @code{read-line} and @code{read-bytes} can be used on the same port. The module also drains ports into lists and vecs, streams them as a @code{Seq} or, for a bjoroutine, as a @code{Stream} with @code{port->stream}, and has @code{byte-pipe}, a pair of ports with no file and no network, for tests. @code{(std net)} gives a TCP connection as a pair of these ports, so a protocol written for ports also runs over TCP. @code{accept-evt} is an event too: a branch that loses accepts nothing. For more details, see @link["modules/std/ports.html"]{@code{(std ports)}} and @link["modules/std/net.html"]{@code{(std net)}}. @subsection{std/test and std/simpletest} diff --git a/modules/std/stream.sz b/modules/std/stream.sz new file mode 100644 index 0000000..c8334fb --- /dev/null +++ b/modules/std/stream.sz @@ -0,0 +1,3 @@ +(meta (title "(std stream)")) + +@generate-docs[#:module "std stream"]