From d05f4c91dd3dc16eaa74fe8ba685e40a2da9cc32f695f731fd59380dd16a6874 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?= Date: Mon, 26 Oct 2026 09:51:10 +0100 Subject: [PATCH] cleaned up appendix and 002 --- _02_.Writing-programs.sz | 67 ++++++++++++++++++---------------------- _04.Appendices.sz | 64 ++++++++++++-------------------------- 2 files changed, 49 insertions(+), 82 deletions(-) diff --git a/_02_.Writing-programs.sz b/_02_.Writing-programs.sz index e3ec147..2853270 100644 --- a/_02_.Writing-programs.sz +++ b/_02_.Writing-programs.sz @@ -94,7 +94,7 @@ Export Error: Exported item 'helper' is missing a mandatory type signature at ba @subsubsection{The standard library, and the prelude} -A list-shaped module path such as @code{(std random)} refers to a standard library module. The compiler resolves these relative to the Bjolang installation rather than the current directory, so @code{(std random)} always points to the same library regardless of where you invoke @code{bjo}. A few notable modules include: +A list-shaped module path such as @code{(std random)} refers to a library module. The compiler resolves these relative to the Bjolang installation rather than the current directory, so @code{(std random)} always points to the same library regardless of where you invoke @code{bjo}. A few notable modules include: @read-table{ | Module | What it has | @@ -1477,9 +1477,9 @@ The tables in @code{BjolangRuntime/Cml/design.md} in the main repository have ev @subsection{Effects} -Our deck is shuffled using @code{shuffle-vec}, which is non-deterministic. While randomness is ideal during gameplay, it makes automated tests brittle when verifying deterministic outcomes like trick winners. The traditional object-oriented or functional workaround is dependency injection—passing an explicit shuffle function down through every layer of the call stack. +The deck is shuffled with @code{shuffle-vec}, so a hand dealt in a test is a different hand every run. A test for who wins a trick needs the same deck each time. The usual fix is to pass a shuffle function down through every function that deals, which puts a parameter in a lot of code that has nothing to do with shuffling. -Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. With effects, a function simply declares and performs an abstract operation, allowing an enclosing caller higher up the call stack to dynamically decide how that operation is handled. +Bjolang has *tail-resumptive algebraic effects* instead. A function declares an operation and performs it, and whoever is further up the call stack decides what it does. The handler runs once, and the value it returns is the value the operation returns. @subsubsection{Declaring an effect} @@ -1495,7 +1495,7 @@ Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. (vec-slice (shuffle-deck (full-deck)) 0 n)) }| -@code{shuffle-deck} is exposed as a regular function matching the declared signature. When called without an active handler in the dynamic scope, it falls back to the default implementation—in this case, the real shuffle. In a test, you can override this behavior using @code{with-handler}: +@code{shuffle-deck} becomes an ordinary function with the signature it was declared with. Called with no handler in scope it runs the @code{#:default}, which here is the real shuffle. A test installs a handler with @code{with-handler}: @codeblock[#:lang "bjolang"]|{ (deal-hand 5) ;; five random cards @@ -1503,16 +1503,15 @@ Bjolang provides an alternative mechanism: *tail-resumptive algebraic effects*. (deal-hand 5)) ;; [2♣ 3♣ 4♣ 5♣ 6♣], every time }| -The handler intercepts operations executed anywhere within the dynamic extent of the @code{with-handler} block, including inside deeply nested helper calls and child fibers, without requiring @code{deal-hand} to pass any parameters around. +Every call to @code{shuffle-deck} inside the @code{with-handler} body runs the handler: in @code{deal-hand}, in helpers it calls, and in fibers it spawns. Nothing is passed down as a parameter. -Several key semantics govern effect handlers: +An effect declared without @code{#:default} and performed with no handler in scope raises an exception. Leave the default off for operations that only make sense with a handler installed. -@read-list{ -- Default implementations: If an effect omits @code{#:default} and is performed without an active handler, the runtime raises an exception. Omit defaults for operations that always require explicit environmental context. -- One-shot execution: Handlers in Bjolang behave like standard functions and return a single value. Full multi-shot delimited continuations are not supported: a handler cannot resume its caller multiple times or abort resumption except by throwing an exception. -- Effect coloring: Operations declared with @code{->} can be invoked from any standard function. Operations declared with @code{-bjo->} may only be invoked from within bjoroutines, and their handlers must likewise be bjoroutines. -- Handler composition: Invoking an operation from inside its own handler invokes that handler recursively. To delegate to an enclosing outer handler, retrieve it using @code{(handler-of op)}: -} +A handler is a function and returns one value. It cannot resume its caller twice, and the only ways out of it are to return or to raise. There are no multi-shot continuations. + +An operation declared with @code{->} can be performed from any function. One declared with @code{-bjo->} can only be performed from a bjoroutine, and its handler has to be a bjoroutine too. + +Performing an operation inside its own handler calls that handler again, so it recurses. To pass the work to the handler that was there before yours, ask for it with @code{(handler-of op)}: @codeblock[#:lang "bjolang"]|{ (with-handler ((log (let ((outer (handler-of log))) @@ -1522,7 +1521,7 @@ Several key semantics govern effect handlers: @subsubsection{The prelude's own effects} -Several core runtime facilities in the prelude are implemented as effects out of the box, allowing you to intercept or virtualize them in tests without modifying application code: +Part of the prelude already works this way, so a test can replace it without touching the code under test: @read-table{ | Operation | Default | @@ -1530,13 +1529,13 @@ Several core runtime facilities in the prelude are implemented as effects out of | @code{(log s)} | Writes a line to standard error | | @code{(warn s)} | Writes a warning line to standard error | | @code{(getenv name)} | Reads an environment variable as an @code{Option} | -| @code{(monotonic-ms)} | High-resolution monotonic clock for elapsed timing | -| The @code{FS} operations | Physical filesystem operations | +| @code{(monotonic-ms)} | A monotonic clock, in milliseconds | +| The @code{FS} operations | The real filesystem | } -Because all prelude file I/O routes through the built-in @code{FS} effect operations, tests can replace the entire filesystem with an in-memory mock using @code{with-fake-fs} (covered in the Testing section below). +All file I/O in the prelude goes through the @code{FS} operations, so a test can run against a fake filesystem with @code{with-fake-fs} instead of the disk. The testing section has the details. -Similarly, you can capture diagnostic output in memory instead of dumping it to the terminal by installing a custom handler for @code{log}: +Log output is easy to catch the same way. Install a handler for @code{log} and the lines end up wherever you put them: @codeblock[#:lang "bjolang"]|{ (def lines (make-box (list))) @@ -1546,15 +1545,15 @@ Similarly, you can capture diagnostic output in memory instead of dumping it to (box-ref lines) ;; '("two" "one") }| -Keep in mind that @code{with-handler} only establishes a dynamic binding; unlike a concurrency scope, it does not await background fibers. If code inside the handler spawns asynchronous fibers that emit logs, ensure the concurrency scope enclosing those fibers is placed *inside* the @code{with-handler} block so the handler remains active until all fibers finish writing. +A handler is a dynamic binding, not a scope. @code{with-handler} does not wait for fibers the body spawns. If the body spawns fibers that log, put the scope *inside* the handler, or the handler is gone and they write to stderr after all. @subsection{.NET interop} -Because Bjolang compiles directly to C#, you have full, seamless access to the entire .NET ecosystem. By declaring the CLR classes, methods, and properties you need along with their signatures, the compiler exposes them as ordinary Bjolang functions and types. +Bjolang compiles to C#, so the whole .NET library is within reach. You write the member you want and its signature, and the compiler turns it into an ordinary Bjolang function or type. @subsubsection{Methods: import/extern} -For example, we can render suit symbols in color in the terminal using .NET's @code{System.Console}: +Suit symbols in colour, with @code{System.Console}: @codeblock[#:lang "bjolang"]|{ (import "deck.bjo") @@ -1585,7 +1584,7 @@ For example, we can render suit symbols in color in the terminal using .NET's @c 0) }| -Each entry in @code{import/extern} specifies a local Bjolang identifier, the fully qualified CLR member, and its signature. For overloaded methods (such as @code{Console.Write}), the compiler determines which overload to bind based on the provided signature. Instance methods accept the target object as their first parameter. Properties and fields can be accessed via @code{#:get} and modified with @code{#:set}: +An entry in @code{import/extern} is three things: a local name, the CLR member it stands for, and a signature. Where the CLR has overloads, as @code{Console.Write} does, the signature picks one. An instance method takes its object as the first parameter. A field or property is read with @code{#:get} and, with @code{#:set}, written: @codeblock[#:lang "bjolang"]|{ (import/extern @@ -1598,11 +1597,11 @@ max-int ;; 2147483647 .NET's own string, @code{System.String}, is UTF-16 and a different type from @code{string}. A .NET method that takes or returns one, or an array of them, is converted where it is called, so @code{trim} above takes and gives a Bjolang string; a @code{null} from .NET arrives as @code{""}. A .NET string inside another type, such as a @code{(Seq System.String)}, is not converted: @code{string->clr-string} and @code{clr-string->string} do it by hand. -Avoid binding an external method to the name of an existing prelude function. Locally bound names and prelude symbols take precedence over @code{import/extern} declarations, which means an alias named @code{println} would simply be shadowed and ignored. +Do not give an external method the name of a prelude function. A local name and a prelude symbol both win over an @code{import/extern}, so an alias called @code{println} is ignored. @subsubsection{Classes: import/class} -The @code{import/class} form maps a .NET class to a Bjolang type, exposing its constructor as a function suffixed with a dot (e.g. @code{StringBuilder.}). CLR enums map similarly, with individual members accessed using dot notation like @code{Type.Member} (seen with @code{ConsoleColor.Red} above). +@code{import/class} gives a .NET class a Bjolang type, and its constructor a function with a trailing dot: @code{StringBuilder.} builds one. A CLR enum works the same way, and its members are written @code{Type.Member}, as @code{ConsoleColor.Red} above. @codeblock[#:lang "bjolang"]|{ (import/class @@ -1618,17 +1617,11 @@ The @code{import/class} form maps a .NET class to a Bjolang type, exposing its c A method that @code{string} does not have, like @code{ToUpper}, is called on the string converted to .NET's, so its indices and lengths are .NET's UTF-16 ones and not @code{string-length}'s bytes. -As shown in the example above, you do not need to declare every method up front with @code{import/extern}. Once an object's type is known to the type checker, you can invoke any public method using @code{(.Method obj args ...)} and inspect properties using @code{(.-Property obj)}. Because @code{Append} returns the builder itself and Bjolang enforces explicit handling of return values, discard unused returns intentionally using @code{ignore}. +You do not have to declare every method up front. Once the type checker knows an object's type, any public method is @code{(.Method obj args ...)} and any property is @code{(.-Property obj)}. @code{Append} returns the builder itself, and a return value has to be used, so the unused ones above go through @code{ignore}. @subsubsection{When .NET fails} -Idiomatic .NET code reports errors by throwing exceptions. Bjolang provides three distinct ways to translate these exceptions into safe value types: - -@read-list{ -- Generic @code{try} blocks: Wrap arbitrary code in @code{try}, as introduced in Part 1. -- Exception-mapped constructors: Annotating a constructor with @code{#:exceptions} in @code{import/class} causes the constructor to return a @code{Result} instead of throwing on specified exceptions. -- Output parameter conversion: Common .NET methods adhering to the @code{bool TryParse(..., out T result)} pattern automatically transform into Bjolang functions that return an @code{(Option T)}. -} +.NET reports errors by throwing. There are three ways to get a value instead: @code{try} around the call, as in Part 1; a constructor declared with @code{#:exceptions} in @code{import/class}, which returns a @code{Result} for those exceptions instead of throwing; and a method with the @code{bool TryParse(..., out T result)} shape, which becomes a function returning @code{(Option T)}. @codeblock[#:lang "bjolang"]|{ (import/class @@ -1648,7 +1641,7 @@ Idiomatic .NET code reports errors by throwing exceptions. Bjolang provides thre @subsubsection{Async methods, and blocking calls} -A .NET method returning a @code{Task} or @code{Task} can be imported using the @code{#:async} modifier. Calling an async method from within a bjoroutine suspends the fiber until the task completes without blocking an OS thread, with no explicit @code{await} required: +A method that returns a @code{Task} or @code{Task} is imported with @code{#:async}. Called from a bjoroutine it suspends the fiber until the task is done and blocks no thread. There is no @code{await} to write: @codeblock[#:lang "bjolang"]|{ (import/extern @@ -1659,11 +1652,11 @@ A .NET method returning a @code{Task} or @code{Task} can be imported using th 0) }| -Conversely, CLR methods known to perform synchronous, thread-blocking I/O should be tagged with @code{#:blocking}. Calling a blocking method from within a bjoroutine triggers a compiler warning; you can resolve this warning by offloading the call to a dedicated thread pool thread via @code{(sync (blocking (fun () ...)))}. +A CLR method that blocks its thread is tagged @code{#:blocking}. Calling one from a bjoroutine is a compiler warning, and the fix is to give it a thread of its own: @code{(sync (blocking (fun () ...)))}. @subsubsection{Generics} -Generic .NET types are imported by parameterizing them with type variables, and generic methods specify those type variables in their signature: +A generic .NET type is imported with type variables, and a generic method names them in its signature: @codeblock[#:lang "bjolang"]|{ (import/class @@ -1674,15 +1667,15 @@ Generic .NET types are imported by parameterizing them with type variables, and (-> (Dict %k %v) %k (out %v) (Option %v))))) }| -At the boundary between systems, Bjolang types map cleanly to standard .NET equivalents: a @code{Func} corresponds to @code{(-> %a %b)}, @code{IEnumerable} maps to @code{(Seq %a)}, C# tuples map to @code{Tuple}, and native arrays @code{T[]} correspond to @code{(Array %a)}. Note that constructing generic .NET classes dynamically and pattern matching on .NET objects via constructor patterns are not currently supported. +Bjolang types and .NET types line up: @code{(-> %a %b)} is a @code{Func}, @code{(Seq %a)} an @code{IEnumerable}, @code{(Array %a)} a @code{T[]}, and a C# tuple is a @code{Tuple}. Two things do not work yet: building a generic .NET class from a type only known at runtime, and constructor patterns on .NET objects. @subsubsection{Culture and formatting} -Bjolang programs execute using the invariant culture by default. For example, floating-point numbers format with a decimal point (@code{2.5}), avoiding regional discrepancies on machines where local culture settings would otherwise produce @code{2,5}. If your application requires localized formatting, you can configure @code{CultureInfo.CurrentCulture} explicitly. +A Bjolang program runs in the invariant culture, so a double formats with a decimal point (@code{2.5}) and not as @code{2,5} on a machine where it otherwise would. Set @code{CultureInfo.CurrentCulture} if you want local formatting. @subsubsection{Calling Bjolang from C#} -Every Bjolang module compiles to a public static class named @code{_Module}, exposing top-level functions defined with @code{defun} as static methods and @code{def} values as static fields. Identifiers with characters invalid in C# are automatically mangled (for example, hyphenated names like @code{full-deck} become @code{fullsubdeck}), while collections remain Bjolang's persistent data structures. You can inspect the emitted C# source code directly by passing the debug flag: @code{bjo build -d}, which generates @code{out.cs}. +Every module compiles to a public static class called @code{_Module}: the @code{defun}s in it are static methods and the @code{def}s are static fields. A name with characters C# will not take is mangled, so @code{full-deck} becomes @code{fullsubdeck}. Collections stay Bjolang's persistent ones. To read the C#, build with @code{-d} and open @code{out.cs}. @subsection{Macros} diff --git a/_04.Appendices.sz b/_04.Appendices.sz index 26c6767..0f5bdbf 100644 --- a/_04.Appendices.sz +++ b/_04.Appendices.sz @@ -1,54 +1,14 @@ @section{Appendices} -@subsection{Coming from Scheme or ML} +@subsection{Language inspiration} -Bjolang sits at the crossroads of two major functional programming lineages. From the Scheme and Lisp tradition, it draws its uniform S-expression syntax, prefix notation, code-as-data perspective, and hygienic macro system. From the ML family—specifically OCaml and F#—it inherits static Hindley-Milner type inference, algebraic data types, compile-time exhaustiveness checking, traits, and high-performance immutable data structures. +Bjolang is firmly rooted in two large programming language families, and draws inspiration from both. I spent most of my life writing r6rs scheme. I think it is a beautiful language, and it is still often what I reach for to try out new things. The second is ML, mostly ocaml, with it's strong tradition of always managing to select the right abstractions. If you come from any of these languages you will feel at home in one way or the other. Some things I took from both -- like immutability -- will make adherents of both churches feel right at home and some things will feel weird. -Depending on which background you bring to Bjolang, parts of the language will feel like second nature while other parts will require shifting your mental habits. +When deciding on API for bjolang I mostly asked myself "What would ocaml/scheme do?". With ocaml there is a rich set of libraries solving real world problems, so inspiration was easy to find, and from scheme I mostly had a look at racket (which is kind of r6.1rs). -@subsubsection{Coming from Scheme} +A special nod goes to Alex Shinn of Chibi scheme. Both @code{(std fmt)}, @code{(std rx)} and @code{(std scan)} are heavily influenced by programs of his. In fact, the whole idea of the @code{loop} form comes from an earlier project of mine extending his foof-loop. -If you have written Scheme, Racket, or Common Lisp, the visual rhythm of Bjolang will feel immediately comfortable: - -@read-list{ -- S-expression syntax and prefix notation: Expressions are nested parenthesized lists, and operators are ordinary functions called in prefix position: @code{(+ 1 2 (* 3 4))}. -- Familiar control forms: Core branching and binding forms like @code{let}, @code{let*}, @code{cond}, @code{when}, @code{unless}, and @code{begin} behave as you expect. -- First-class closures: Functions are first-class values created with @code{fun} (or @code{defun} at top level), capturing lexical scope, and passing into or returning from higher-order functions. -- Persistent linked lists: The @code{List} type provides standard singly-linked functional lists with @code{Cons} and @code{Nil}, along with familiar combinators such as @code{list-head}, @code{list-tail}, @code{list-map}, and @code{list-filter}. -- Hygienic macros: Bjolang features a compile-time syntactic macro system based on pattern matching with @code{syntax-match}, syntax quotation with @code{#'(...)}, and explicit identifier injection with @code{inject}. -} - -However, several fundamental differences will quickly become apparent: - -@read-list{ -- Static typing and mandatory top-level signatures: Dynamic typing is absent. While types inside function bodies are inferred automatically using Hindley-Milner type inference, every top-level definition and exported value requires an explicit signature: @code{(: name type)}. Type errors are caught at compile time before any code runs. -- No truthiness tricks or @code{#f}-as-nil: In Scheme, functions like @code{memq} or @code{assoc} return either a value or @code{#f} to signal absence, and @code{#f} is often treated as a general falsey sentinel. In Bjolang, absent values are strictly represented by the @code{(Option %a)} union (@code{Some} or @code{None}), and operations that may fail return @code{(Result %e %a)} (@code{Ok} or @code{Err}). Booleans (@code{#t} and @code{#f}) are strictly booleans; conditionals do not coerce non-boolean values. -- Vectors and maps are persistent by default: In Scheme, vectors are mutable flat arrays. In Bjolang, @code{(Vec %a)} is an immutable Relaxed Radix Balanced (RRB) tree with efficient O(log n) random indexing, updates, and concatenation. Similarly, @code{(Map %k %v)} is an immutable Compressed Hash-Array Mapped Prefix-tree (CHAMP). In-place mutation is never the default: mutable state requires explicit declaration with @code{def/mutable} and modification with @code{set!}. -- Structural pattern matching over @code{car}/@code{cdr}: Destructuring lists, records, and unions is done using the @code{match} construct rather than long chains of @code{car}, @code{cdr}, or @code{cadr}. The compiler checks pattern matches for exhaustiveness and warns or errors if any potential value is unaccounted for. -- Concurrency without @code{call/cc}: Scheme systems often implement coroutines or thread libraries using first-class continuations (@code{call/cc}). Bjolang does not provide @code{call/cc}. Instead, it offers direct-style cooperative fibers (bjoroutines), typed channels, and Concurrent ML (CML) event combinators running on a dedicated fiber scheduler. -} - -@subsubsection{Coming from F# or OCaml} - -If you come from F#, OCaml, or Haskell, the conceptual model of Bjolang's type system and runtime behavior will feel very natural: - -@read-list{ -- Hindley-Milner type inference: Types within local expressions and function bodies are completely inferred. Polymorphic functions declare type variables with a leading percent sign, such as @code{%a} or @code{%elem}. -- Algebraic data types: Data structures are modeled as sum types (@code{Union}) and product types (@code{Record} on the heap, or value-type @code{Struct} on the stack). -- Exhaustive pattern matching: The compiler tracks which variants of a union or struct combination have been handled in a @code{match} or @code{def} binding, rejecting patterns that fail to cover every possible case. -- Immutability by default: Data structures, record fields, and bindings are immutable unless explicitly declared mutable. -- Traits and ad-hoc polymorphism: Bjolang's @code{def/trait} and @code{impl} system works much like Haskell typeclasses or Rust traits. Traits can define default method implementations, require super-traits, and specify associated types. -- Direct-style concurrency: Concurrency is written in direct style without callback pyramids, closely resembling F#'s async computations or OCaml 5 effect handlers. -} - -The main shifts you will encounter coming from ML are: - -@read-list{ -- Syntax and parentheses: There is no indentation-based syntax (like F#) or @code{let ... in} keyword framing (like OCaml). Everything is an S-expression with prefix notation. While this takes an afternoon to get used to, it makes code structure uniform and simplifies automated tooling. -- First-class compile-time macros: Rather than relying on external preprocessors like OCaml's PPX or compiler plugins, Bjolang allows writing hygienic macros in the language itself using @code{def/macro}. These transformers run inside the compiler process during the expansion phase. -- Concurrent ML events instead of raw tasks: Rather than manipulating raw asynchronous tasks or promises directly, Bjolang builds upon Concurrent ML. The primitive unit of asynchronous composition is the @code{(Event %a)}. Events can be combined with @code{choose}, wrapped with transformations via @code{wrap}, and selectively synchronized using @code{sync}. -- Automatic colour twins: In many typed asynchronous languages, asynchronous functions have distinct types (@code{Task<'T>} or @code{Async<'T>}) that cannot be passed to standard higher-order functions like @code{List.map} without creating asynchronous duplicates (@code{List.mapAsync}). Bjolang solves this "function colouring problem" at the compiler level: higher-order functions declaring @code{-?->} parameters automatically generate both synchronous and suspending twins, allowing the exact same @code{list-map} to be used whether your callback performs I/O or pure computation. -} +The concurrency story is that of ConcurrentML. A language almost as old as me, that has lived on in various smaller projects. It came to my attention through Andy Wingo, who in his turn got it from the sages of Racket. There are various resources online about it, but I would really recommend watching Andy's speeches about it. There is also a talk circulating online from Mike Sperber talking about it, which can probably be found on the large streaming site. @subsection{Reading compiler errors} @@ -414,3 +374,17 @@ Key editing features and keybindings include: - @code{C-c C-o}: Follow the reference at point, opening the target URL or jumping directly to an included file. - Document navigation: Seamless integration with @code{outline-minor-mode} and @code{imenu} allows folding sections and navigating complex documents through hierarchical headings. } + +@subsection{Why it is a sad world on dotnet} + +The reason bjolang runs on dotnet is mostly because of inertia. I had a lot of code already written for c#, and reimplementing RRB trees or CHAMPs in ocaml is nothing I felt like doing. Running on dotnet comes with tradeoffs, most notably function colouring. Had I gone with ocaml5, I would have gotten effect handlers, which means delimited control, which means everything can potentially be a bjoroutine. Now I have async, which means colouring and compiling functions in two ways and sad faces all the way down. + +The longer I have come with bjolang, the more I have started looking sideways at ocaml. When it comes to selecting abstractions, ocaml and racket are somehow almost always spot on, which means my main design guideline is "when in doubt, look at ocaml/racket/(guile)". A lot of the things that were not implemented in bjolang are things that simply were too expensive on dotnet, but more or less free on ocaml. + +Most of it is delimited control, but having the world's best STM and full effect handlers would make my life a breeze. + +@subsubsection{So... when will you switch?} + +Probably never. Interop is too nice to have. Just being able to Nuget a nice markdown parser and write some glue is just too damn nice. I did some playing with ocaml, and there are some things I really do not like. Minor gc collections stop all domains. Boxed generics. + +But the effect handlers, man...