bjoli/BjoManual
SHA256
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Benchmarks: Go and Hopac in one comparison, and the service run again

Browse source 
The primitive benchmarks are one table with all six columns, Bjolang's
two, Go's two and Hopac's two, with the introduction once and what each
comparison says after it. What each operation allocates is one paragraph
for all three.

The service section has a new run on the current runtime. Go with as
much memory as .NET has the shortest tail at 250k/s on four cores;
Bjolang has it at every higher load. Over three runs the row at 500k/s
on all cores moved too much to rank, and the text says so.

🤖 Generated with [ECA](https://eca.dev)

Co-Authored-By: eca-agent <git@eca.dev>
This commit is contained in:
Linus Björnstam 2026-10-14 11:45:01 +02:00
parent 970754b3be
commit ba5c7afc9d
1 changed files with 23 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

55
_02_.Writing-programs.sz
View file

@ -921,38 +921,29 @@ Two dealers can run at the same time. Neither card can end up in two hands, or b
@subsubsection{Compared with Go and Hopac}
Go is the obvious point of comparison: goroutines and unbuffered channels are the same model of fibers meeting at a rendezvous. The main repository has the same benchmarks written in both languages (@code{bench/bjolang/cmlbench.bjo} and @code{go run . -suite cmlbench} in @code{bench/go}). Every Bjolang channel operation races the cancellation token of its scope, so Go is measured twice: as plain channel operations, and with each one selecting against @code{ctx.Done()}, which is what a go program has to stop execution. Bjolang is measured twice as well: with events and @code{sync}, and with @code{chan-put} and @code{chan-get}, so the difference between the two bjolang columns is the cost of what building an event and syncing it costs. The best of five runs, in nanoseconds per operation, on a Ryzen 9 5900X under Linux with .NET 10 and Go 1.25:
Go and Hopac are the two points of comparison. Go is the obvious one: goroutines and unbuffered channels are the same model of fibers meeting at a rendezvous. Hopac is the tougher one. It is a Concurrent ML library for F#, running on the same .NET runtime with the same JIT and garbage collector, so the differences between it and Bjolang are in the design rather than in the platform. The main repository has the same benchmarks in all three (@code{bench/bjolang/cmlbench.bjo}, @code{go run . -suite cmlbench} in @code{bench/go}, and @code{dotnet run -c Release -- cmlbench} in @code{bench/hopac}).
Every Bjolang channel operation races the cancellation token of its scope. Neither Go nor Hopac has that built in, so each is measured twice: plain, and with every operation raced against what a program in it has to use to stop its work. In Go that is a @code{select} against @code{ctx.Done()}; in Hopac it is an @code{IVar} (Hopac's write-once variable) that is filled to stop the jobs. Bjolang is measured twice as well: with events and @code{sync}, and with @code{chan-put} and @code{chan-get}, so the difference between the two Bjolang columns is what building an event and syncing it costs. The best of five runs, in nanoseconds per operation, on a Ryzen 9 5900X under Linux with .NET 10 and Go 1.25:
@read-table{
| Benchmark | Bjolang, @code{sync} | Bjolang, @code{chan-put} | Go | Go, with @code{ctx.Done()} |
|----------------------------------------+----------------------+--------------------------+-----+----------------------------|
| Ring: 1000 fibers, a million messages | 70 | 73 | 117 | 241 |
| The same ring inside a nested scope | 63 | — | — | 242 |
| Spawning a million fibers and waiting | 210 | — | 190 | — |
| An 8-way choose where one branch fires | 181 | 205 | 403 | 451 |
| Benchmark | Bjolang, @code{sync} | Bjolang, @code{chan-put} | Go | Go, @code{ctx.Done()} | Hopac | Hopac, cancel @code{IVar} |
|----------------------------------------+----------------------+--------------------------+-----+-----------------------+-------+---------------------------|
| Ring: 1000 fibers, a million messages | 70 | 73 | 117 | 241 | 51 | 3487 |
| The same ring, one cancel IVar per job | — | — | — | — | — | 195 |
| The same ring inside a nested scope | 63 | — | — | 242 | — | 3427 |
| Spawning a million fibers and waiting | 210 | — | 190 | — | 80 | — |
| An 8-way choose where one branch fires | 181 | 205 | 403 | 451 | 127 | 136 |
}
In the choose row only the sender can use @code{chan-put}, since the receiving side has to use channel-receive (and thus allocates). On the ring, syncing an event costs nothing measurable any more: a send or a receive synced alone does not build an event. For some reason, this ring inside a nested scope is faster in bjolang, and I haven't figured out why yet.
Against Go, Bjolang is ahead on the ring and the choose, and further ahead once Go has to watch @code{ctx.Done()}, which doubles its ring. Go spawns a little faster. Against Hopac it is the other way round without cancellation: Hopac's ring and choose take about 30% less time than Bjolang's, and its spawns about a third. Bjolang's numbers include watching the scope's token at every operation, which Hopac's plain rows do not do. Once cancellation is added, the ring turns. A thousand jobs racing one shared @code{IVar} all take that IVar's lock at every operation, and the ring slows to three and a half microseconds a message. Giving each job an IVar of its own brings it back to 195 ns, at the price of keeping track of a thousand IVars and filling all of them to stop the ring. The choose does not turn: with a cancel IVar Hopac's takes 136 ns, against Bjolang's 181.
The spawn row is about twice what it was with the scheduler Bjolang used before, which handed every fiber to the .NET thread pool, and the choose row about one and a half times. With it (@code{BJO_SCHEDULER=pool} in the environment, kept as a way back) a spawn costs 105 ns and the choose 113. The spawn row is a single fiber spawning a million fibers that do nothing, and it is slow only on many cores: on two or four cores the current scheduler spawns faster than the pool, 61 and 72 ns against 87 and 112. The scheduler that replaced it runs fibers from run queues of its own, in the shape of Go's, because the pool let work that had waited longest wait longer still under load. That is worth more in a loaded service, as the service benchmark below shows, than these two rows on an idle machine.
These are small benchmarks on one machine, and they only measure the cost of the concurrency primitives, not of the work between rendezvous. Go is ahead in other ways: a Go channel operation never allocates, where a Bjolang @code{choose} allocates 56 bytes and a spawn 136 (a send or a receive synced alone allocates nothing). Any Go function can block without being written as a bjoroutine and the Go scheduler preempts a goroutine that computes for too long, where a fiber has to be moved off the pool with @code{spawn/thread}.
These are small benchmarks on one machine, and they only measure the cost of the concurrency primitives, not of the work between rendezvous. They do not show what each operation allocates either. A Go channel operation never allocates and a goroutine costs 16 bytes. A Bjolang send or receive synced alone allocates nothing, a @code{choose} 56 bytes and a spawn 136. Hopac allocates 208 bytes a message on the ring, 424 a choose and 184 a spawn. Go is ahead in other ways too: any Go function can block without being written as a bjoroutine and the Go scheduler preempts a goroutine that computes for too long, where a fiber has to be moved off the pool with @code{spawn/thread}.
What Bjolang offers that Go does not is in the CML vocabulary rather than in the numbers. A Go @code{select} is a statement, its cases are fixed in the source, and a channel operation cannot be passed to a function, wrapped, or combined with others. In Bjolang an event is a value, which is what made @code{within} above a two-line function that works on any event, and @code{choose}, @code{wrap}, @code{guard} and @code{with-nack} build new events out of old ones. Go has no structured concurrency either. Waiting for goroutines, stopping them, and reporting their failures are a @code{sync.WaitGroup}, a @code{context.Context} passed to every function and checked at every channel operation, and an @code{errgroup}, written by hand each time. A goroutine that nothing waits for leaks without a sound. In Bjolang a scope does all of that by construction: it waits for its fibers, raises their failures when it ends, releases the files they opened, and cancels them through a token that every @code{sync} watches without being asked.
Hopac is the tougher comparison. It is a Concurrent ML library for F#, running on the same .NET runtime with the same JIT and garbage collector, so the differences between it and Bjolang are in the design rather than in the platform. The same benchmarks are in @code{bench/hopac} (@code{dotnet run -c Release -- cmlbench}). Hopac has no cancellation built in, so like Go it is measured twice: plain, and with every operation raced against an @code{IVar} (Hopac's write-once variable) that is filled to stop the jobs. Same machine, best of five, nanoseconds per operation:
@read-table{
| Benchmark | Bjolang, @code{sync} | Hopac | Hopac, racing a cancel @code{IVar} |
|-----------------------------------------+----------------------+-------+------------------------------------|
| Ring: 1000 fibers, a million messages | 70 | 51 | 3487 |
| The same ring, one cancel IVar per job | — | — | 195 |
| The same ring inside a nested scope | 63 | — | 3427 |
| Spawning a million fibers and waiting | 210 | 80 | — |
| An 8-way choose where one branch fires | 181 | 127 | 136 |
}
Without cancellation Hopac is ahead: its ring and its choose take about 30% less time than Bjolang's, and its spawns about a third. Bjolang's numbers include watching the scope's token at every operation, which Hopac's plain rows do not do. Once cancellation is added, the ring turns. A thousand jobs racing one shared @code{IVar} all take that IVar's lock at every operation, and the ring slows to three and a half microseconds a message. Giving each job an IVar of its own brings it back to 195 ns, at the price of keeping track of a thousand IVars and filling all of them to stop the ring. The choose does not turn: with a cancel IVar Hopac's takes 136 ns, against Bjolang's 181. Hopac allocates more: 208 bytes a message on the ring and 424 a choose, against Bjolang's 0 and 56. Hopac can run jobs in parallel and wait for all of them, but a job cannot be stopped from the outside, so stopping jobs when one of them fails, or when their owner is done with them, is the same hand-written race against an IVar as in the table.
What Bjolang offers that Go does not is in the CML vocabulary rather than in the numbers. A Go @code{select} is a statement, its cases are fixed in the source, and a channel operation cannot be passed to a function, wrapped, or combined with others. In Bjolang an event is a value, which is what made @code{within} above a two-line function that works on any event, and @code{choose}, @code{wrap}, @code{guard} and @code{with-nack} build new events out of old ones. Go has no structured concurrency either. Waiting for goroutines, stopping them, and reporting their failures are a @code{sync.WaitGroup}, a @code{context.Context} passed to every function and checked at every channel operation, and an @code{errgroup}, written by hand each time. A goroutine that nothing waits for leaks without a sound. In Bjolang a scope does all of that by construction: it waits for its fibers, raises their failures when it ends, releases the files they opened, and cancels them through a token that every @code{sync} watches without being asked. Hopac can run jobs in parallel and wait for all of them, but a job cannot be stopped from the outside, so stopping jobs when one of them fails, or when their owner is done with them, is the same hand-written race against an IVar as in the table.
@subsubsection{A service under load}
@ -965,10 +956,10 @@ In a closed loop, 1000 clients send 1000 requests each, each one as soon as the
@read-table{
| Program | all cores: wall | CPU | p99.9 | memory | 4 cores: wall | CPU | p99.9 |
|------------------+-----------------+---------+--------+--------+---------------+---------+--------|
| Go | 400 ms | 5542 ms | 5.8 ms | 27 MB | 479 ms | 1847 ms | 3.4 ms |
| Go, same memory | 259 | 4057 | 3.2 | 402 | 425 | 1680 | 3.1 |
| Hopac | 436 | 8854 | 12.7 | 387 | 294 | 1146 | 3.8 |
| Bjolang | 256 | 5194 | 5.6 | 427 | 411 | 1587 | 4.7 |
| Go | 397 ms | 5552 ms | 5.7 ms | 27 MB | 475 ms | 1832 ms | 3.4 ms |
| Go, same memory | 258 | 4076 | 3.2 | 399 | 414 | 1641 | 3.5 |
| Hopac | 411 | 8903 | 9.8 | 387 | 296 | 1161 | 4.2 |
| Bjolang | 254 | 5211 | 5.5 | 423 | 407 | 1571 | 4.7 |
}
On all cores Bjolang and Go with the same memory finish together, and Go uses a fifth less CPU and has the shorter tail. On four cores Hopac is clearly the fastest, and Bjolang and Go are close.
@ -978,13 +969,13 @@ In an open loop, 10000 clients send on a fixed schedule at a given rate, so the
@read-table{
| Load | Go | Go, same memory | Hopac | Bjolang |
|----------------------+-------------+-----------------+-------------+-------------|
| 4 cores, 250k/s | 5.0 / 8.1 | 2.1 / 4.9 | 2.7 / 4.7 | 2.5 / 3.0 |
| 4 cores, 500k/s | 33.1 / 38.0 | 25.0 / 31.1 | 7.3 / 8.7 | 3.0 / 4.2 |
| all cores, 500k/s | 9.2 / 12.3 | 6.6 / 11.0 | 23.5 / 30.5 | 5.0 / 6.7 |
| all cores, 1M/s | 31.1 / 34.8 | 32.2 / 40.0 | 27.1 / 34.5 | 7.4 / 11.7 |
| 4 cores, 250k/s | 5.6 / 9.0 | 2.0 / 2.2 | 2.8 / 4.3 | 2.2 / 3.1 |
| 4 cores, 500k/s | 28.0 / 32.6 | 41.9 / 48.6 | 6.9 / 8.3 | 3.1 / 3.7 |
| all cores, 500k/s | 7.0 / 9.1 | 8.0 / 12.7 | 24.5 / 31.1 | 3.6 / 6.3 |
| all cores, 1M/s | 33.9 / 37.1 | 69.4 / 73.5 | 27.5 / 32.3 | 5.9 / 8.6 |
}
A client that is early sleeps until its request is due, in whole milliseconds rounded up, the same in all three programs, so every latency here has up to a millisecond of that in it, and below about 2 ms the numbers say little. Bjolang's p99.9 is the shortest at every load, and at a million requests a second its tail is a quarter to a third of the others'. At 250k/s on four cores the three are close, and Go with the same memory has the shortest p99 there. At 500k/s on all cores Go with the same memory comes near: another run of the same gave Bjolang 3.6 ms and Go 2.1 at p99, so that row is not settled. Hopac keeps all 24 cores busy on all cores at either rate, and why has not been looked into.
A client that is early sleeps until its request is due, in whole milliseconds rounded up, the same in all three programs, so every latency here has up to a millisecond of that in it, and below about 2 ms the numbers say little. At 250k/s on four cores the three are close, and Go with the same memory has the shortest tail. At every higher load Bjolang has the shortest, and at a million requests a second its tail is a quarter of Hopac's or less, and a fifth of Go's or less. Some rows move from run to run: over three runs at 500k/s on all cores, Go with the same memory had a p99 of 2.1 to 8.0 ms and Bjolang of 3.6 to 5.0, so that row is not settled. Go's tail at 500k/s on four cores is long with either memory, and its time once a request has started is as long, so the delay is in running the requests, not before them. Hopac keeps all 24 cores busy on all cores at either rate, and why has not been looked into.
The tables in @code{BjolangRuntime/Cml/design.md} in the main repository have every column, including the time a request took once its client had started it.