Document spans, interop, csv, parallel, and combine
- Language basics: add Span/RoSpan section with construction, operations and lifetime rules - Writing programs: add full Spans in .NET interop subsubsection (arrays-as-spans, #:copy for span returns, span values and their restrictions); update performance notes to remove "without spans" - Standard library: split std/units+rational into two subsections with fuller descriptions; add std/parallel+combine, std/interop, and text/csv+csv-codec subsections - index: mention CSV in the text module description 🤖 Generated with [ECA](https://eca.dev) (anthropic/claude-sonnet-4-6) Co-Authored-By: eca-agent <git@eca.dev>
This commit is contained in:
parent
d16bf477a1
commit
6ee8ca62b9
4 changed files with 109 additions and 7 deletions
|
|
@ -1332,6 +1332,18 @@ An @code{Array} is a plain .NET array: mutable, fixed in length, and exactly wha
|
|||
|
||||
Every evaluation of an array literal makes a fresh array, so a function that returns @code{#[0 0]} returns a new one each time.
|
||||
|
||||
To work on part of an array without copying it, make a span: @code{(Span %a)} is a writable view (.NET's @code{Span<T>}) and @code{(RoSpan %a)} is a read-only view (@code{ReadOnlySpan<T>}).
|
||||
|
||||
@codeblock[#:lang "bjolang"]|{
|
||||
(def s (array->span a #:from 1 #:to 3))
|
||||
(span-length s) ;; 2
|
||||
(span-ref s 0) ;; 2
|
||||
(span-set! s 0 99) ;; writes into a: a is now #[42 99 3]
|
||||
(span->array s) ;; #[99 3], a fresh copy
|
||||
}|
|
||||
|
||||
A span is valid only while the call that made it runs, so it can be a local variable, a function argument or a return value, and the compiler refuses it anywhere it could outlive the call (such as inside another collection or across a yield point). The section on .NET interop in Part 2 covers the full rules and how spans work with .NET methods.
|
||||
|
||||
@subsubsection{Maps}
|
||||
|
||||
@code{Map} is an immutable hash map. The @code{#map(...)} literal takes entries written as @code{(key value)}, @code{[key value]} or @code{(key . value)}.
|
||||
|
|
|
|||
|
|
@ -1669,6 +1669,77 @@ A generic .NET type is imported with type variables, and a generic method names
|
|||
|
||||
Bjolang types and .NET types line up: @code{(-> %a %b)} is a @code{Func<A, B>}, @code{(Seq %a)} an @code{IEnumerable<T>}, @code{(Array %a)} a @code{T[]}, and a C# tuple is a @code{Tuple}. Two things do not work yet: building a generic .NET class from a type only known at runtime, and constructor patterns on .NET objects.
|
||||
|
||||
@subsubsection{Spans}
|
||||
|
||||
Many .NET methods take a @code{Span<T>} or a @code{ReadOnlySpan<T>} instead of an array. You can work with them in two ways: pass an @code{(Array %a)} directly, or use the prelude's @code{(Span %a)} and @code{(RoSpan %a)} types.
|
||||
|
||||
@strong{Arrays where .NET takes a span.} A @code{ReadOnlySpan<T>} or @code{Span<T>} parameter accepts an @code{(Array %a)}, both in an @code{import/extern} signature and in a @code{(.Method obj ...)} call. The span views the array in place, so a method that writes into a @code{Span<T>} mutates the array. When a method is overloaded, an overload taking @code{T[]} wins over a span overload, and @code{ReadOnlySpan<T>} wins over @code{Span<T>}:
|
||||
|
||||
@codeblock[#:lang "bjolang"]|{
|
||||
(import/extern
|
||||
(ascii? (: System.Text.Ascii.IsValid (-> (Array byte) bool))) ;; ReadOnlySpan<byte>
|
||||
(write-be! (: System.Buffers.Binary.BinaryPrimitives.WriteInt32BigEndian (-> (Array byte) int void))) ;; Span<byte>
|
||||
(reverse! (: System.MemoryExtensions.Reverse (-> (Array %a) void)))) ;; Span<T>
|
||||
}|
|
||||
|
||||
A span that a .NET member @em{returns} is not an array. To copy a returned span into a fresh @code{(Array %a)}, mark the @code{import/extern} clause with @code{#:copy}:
|
||||
|
||||
@codeblock[#:lang "bjolang"]|{
|
||||
(import/class
|
||||
(Encoding (: System.Text.Encoding)))
|
||||
|
||||
(import/extern
|
||||
(utf8 (: System.Text.Encoding.UTF8 #:get))
|
||||
(preamble (: System.Text.Encoding.Preamble (-> Encoding (Array byte)) #:get #:copy))
|
||||
(drop-span (: System.MemoryExtensions.AsSpan (-> (Array %a) int (Array %a)) #:copy)))
|
||||
|
||||
(preamble utf8) ;; utf8.Preamble.ToArray()
|
||||
}|
|
||||
|
||||
@code{#:copy} emits @code{.ToArray()} on the call itself, so no span is kept in a local variable and the imported binding can even be passed as a first-class function value. Without @code{#:copy}, declaring a span return as an array is a compile error. @code{#:copy} cannot be combined with @code{#:async}, @code{#:set}, or @code{(out T)} parameters.
|
||||
|
||||
@strong{Spans as values.} To slice an array or pass views around without allocating, use @code{(Span %a)} (.NET's @code{Span<T>}) and @code{(RoSpan %a)} (@code{ReadOnlySpan<T>}), which are in the prelude:
|
||||
|
||||
@codeblock[#:lang "bjolang"]|{
|
||||
(: sum (-> (Span long) long))
|
||||
(defun (sum s)
|
||||
(let loop ((i 0) (acc 0L))
|
||||
(if (< i (span-length s))
|
||||
(loop (+ i 1) (+ acc (span-ref s i)))
|
||||
acc)))
|
||||
|
||||
(def xs #[1L 2L 3L 4L])
|
||||
(sum (array->span xs #:from 1 #:to 3)) ;; 5, and nothing is copied
|
||||
(span-set! (array->span xs) 0 10L) ;; xs[0] = 10
|
||||
}|
|
||||
|
||||
@read-table{
|
||||
| @code{Span} | @code{RoSpan} | Does |
|
||||
|--------------------------------------+-----------------------------------------+-----------------------------------------------------|
|
||||
| @code{(array->span a #:from #:to)} | @code{(array->ro-span a #:from #:to)} | a view of @code{a} from @code{#:from} to @code{#:to}|
|
||||
| @code{(span-length s)} | @code{(ro-span-length s)} | number of elements |
|
||||
| @code{(span-ref s i)} | @code{(ro-span-ref s i)} | element at @code{i} |
|
||||
| @code{(span-set! s i x)} | | writes @code{x} into the underlying storage |
|
||||
| @code{(span-slice s #:from #:to)} | @code{(ro-span-slice s #:from #:to)} | a sub-view of @code{s}; nothing is copied |
|
||||
| @code{(span->array s)} | @code{(ro-span->array s)} | a fresh array copy of the span's elements |
|
||||
| @code{(span->ro-span s)} | | the same view, read-only |
|
||||
}
|
||||
|
||||
@code{#:from} defaults to @code{0} and @code{#:to} defaults to the length. Neither span type converts to the other implicitly; use @code{span->ro-span} when a @code{RoSpan} is needed. In @code{(std mutable vec)}, @code{(mutablevec-span v start n)} returns a @code{(Span %a)} over @code{n} elements of a @code{MutableVec}'s backing storage (valid until the vector's length changes), which pairs with @code{(mutablevec-grow! v n)} to fill a vector a block at a time.
|
||||
|
||||
You can also call a span's .NET members directly: @code{(.-Length s)}, @code{(.Slice s 1)}, @code{(.Fill s x)}, @code{(.get_Item s i)} for @code{s[i]}, and @code{(.set_Item s i x)} for @code{s[i] = x} (an indexer returning @code{ref T} reads a copy of the element, and @code{.set_Item} writes through the reference).
|
||||
|
||||
@strong{Where a span may not go.} A span—or any other .NET @code{ref struct}—is a view of memory that is valid only while the call that made it runs. It can be a local variable, a parameter, or a return value of a function called by name, and the compiler refuses it everywhere it could outlive the call:
|
||||
|
||||
@read-list{
|
||||
- Inside another value: a vec, list, tuple, record, union, @code{Option}, or @code{Result}.
|
||||
- As a type argument of a generic function or trait (such as @code{(println s)} or @code{loop}'s @code{:for} over a collection); walk a span by index instead.
|
||||
- As a parameter or result of a function @em{value} (a @code{(fun ...)} or a function passed as a value is a .NET delegate, which cannot take or return a span). Write a local @code{(defun ...)} and call it by name.
|
||||
- Captured from an outer scope by a @code{(fun ...)}, a local @code{(defun ...)}, a non-tail-recursive @code{let} loop, a @code{(seq ...)}, or a hoisted form. Pass the span to the local function as an argument instead.
|
||||
- In scope across a suspension point in a bjoroutine, or across a @code{(yield ...)} in a @code{(seq ...)}. Do the span work in a synchronous @code{(defun ...)} helper so the span ends before the fiber suspends or yields.
|
||||
- As a parameter or return value of a bjoroutine, an argument to @code{(bjo ...)}, or a module-level @code{def}.
|
||||
}
|
||||
|
||||
@subsubsection{Culture and formatting}
|
||||
|
||||
A Bjolang program runs in the invariant culture, so a double formats with a decimal point (@code{2.5}) and not as @code{2,5} on a machine where it otherwise would. Set @code{CultureInfo.CurrentCulture} if you want local formatting.
|
||||
|
|
@ -2528,7 +2599,7 @@ A percentage can be written @code{2} or @code{2%}. A benchmark file is an ordina
|
|||
@code{prepare} runs a benchmark's setup and warms it up and calibrates it, as a run would. @code{sample} takes one sample of @code{k} calls, and @code{:ns} is the whole sample's time. Anything wrong is answered with @code{(error "why")}, and the session goes on. A served program prints nothing else, so its output is the protocol and nothing more. A script that compares two builds made by two different compilers, which is not something @code{bjo bench} does by itself, can drive both of them this way, or use @code{--against PATH --against-build "CMD"}.
|
||||
|
||||
@subsection{Performance}
|
||||
Bjolang is compiled into pretty straight-forward c#, but without spans or ref structs. It performs pretty similar to c#, except for some small corners of the language that are different.
|
||||
Bjolang is compiled into pretty straight-forward c#. It performs pretty similar to c#, except for some small corners of the language that are different.
|
||||
|
||||
@subsubsection{Strings}
|
||||
Strings in bjolang are utf8, as such they do @em{not} support random access. This means we navigate them differently, using string-cursors, which is explained in the part about strings. Navigating a string using cursors is slower than blasting through integer indices, and if your code needs to be really fast, there are some things you can think about:
|
||||
|
|
@ -2545,7 +2616,7 @@ When parsing strings with a known format, (std scan) will be a lot faster than (
|
|||
@subsubsection{loops}
|
||||
Most of the time you should use the @code{loop} form. It will be the fastest, and is usually flexible enough to do what you need. The only case is for hot loops (ones that are run often) that are performance sensitive. For those, using the various @code{-fold}, @code{-for-each}, and @code{-iter} functions will be faster for the collections that are tree-based. Those collections are: @code{Vec}, @code{Map}, @code{HashSet}, @code{OrderedMap}, and @code{OrderedMap}. The overhead of loop vs. fold is often something like 3x, with fold being as fast or slightly faster thana @code{foreach} on a C# @code{List<T>}. It does not, however, beat using indexed iteration over the @code{List<T>} due to the @code{foreach} doing some safety checks to detect concurrent mutation.
|
||||
|
||||
The absolutely fastest collection to loop over is, unsurprisingly, the Array. In bjolang it should be just as fast as in c# (again, without spans).
|
||||
The absolutely fastest collection to loop over is, unsurprisingly, the Array (or a @code{Span} over part of one). In bjolang it should be just as fast as in c#.
|
||||
|
||||
The loop form gets turned into a kind of goto soup. For the single level case, it is just as efficient as a c# @code{while} loop, but for really complex cases with many levels and subloops with clauses the JIT in dotnet may struggle. In that case you either live with the clarity and simplicity it gives, or you reach for the glorified gotos of names lets. This is probably not going to be a problem though. I tried to rewrite the worst loop form in bjolang (a 16 line monstrosity with one large multi-level claused subloop and a regular subloop) and there was no difference.
|
||||
|
||||
|
|
|
|||
|
|
@ -12,16 +12,26 @@ For more details, see @link["modules/std/prelude.html"]{@code{(std prelude)}}.
|
|||
Numbers beyond the operators. Integer division has Racket's names: @code{quotient} and @code{remainder} are what @code{/} and @code{%} do, and @code{modulo} takes the sign of the divisor, so @code{(modulo -1 10)} is 9. There are @code{gcd}, @code{lcm}, an integer @code{expt} and an exact @code{integer-sqrt} at every integer type, the @code{exact-} roundings from a double to an int, and the rest of @code{System.Math} over doubles. The prelude has @code{abs}, @code{min}, @code{max} and @code{clamp} from here without an import.
|
||||
For more details, see @link["modules/std/maths.html"]{@code{(std maths)}}.
|
||||
|
||||
@subsection{std/units and std/rational}
|
||||
@subsection{std/rational}
|
||||
|
||||
Units of measure, and exact fractions. @code{(std units)} declares SI for the units in types and quantity literals that "Language basics" describes: the eight base dimensions, the SI and binary prefixes, and about 270 units from @code{in} and @code{mi} to @code{eV}, @code{Sv} and @code{rpm}. The constants @code{c}, @code{gn} and @code{planck} are units too, so @code{(number-in v c)} is a speed as a fraction of light's. It is also a library for units at run time, when a unit comes as text, as in a calculator: @code{(convert-query "6000 m + 50 ft in mi")} gives @code{(Ok "3.737697 mi")}, and a @code{Measurement} is a number with a dimension that is checked when the program runs. Absolute temperatures are the types @code{Celsius} and @code{Fahrenheit} rather than units, and the decibels are functions. @code{(std rational)} has the operations of the built-in @code{rational} by name, the roundings to a @code{bigint}, and conversions to and from text, rounded once to a @code{double} or a @code{decimal}.
|
||||
For more details, see @link["modules/std/units.html"]{@code{(std units)}} and @link["modules/std/rational.html"]{@code{(std rational)}}.
|
||||
Exact fractions over @code{bigint}. @code{Rational} is the built-in @code{rational} type, kept in lowest terms with a positive denominator so equal fractions are equal values. @code{(rational n d)} constructs a fraction, @code{rational-num} and @code{rational-den} take it apart, and @code{rational+}, @code{rational*}, @code{rational-recip} and @code{rational-expt} work alongside the standard arithmetic operators. @code{string->rational} reads both @code{"1|60"} and decimal text such as @code{"0.3048"}, and @code{rational->double} and @code{rational->decimal} convert back.
|
||||
For more details, see @link["modules/std/rational.html"]{@code{(std rational)}}.
|
||||
|
||||
@subsection{std/units}
|
||||
|
||||
Units of measure, both at compile time and at run time. Importing the module declares the SI base dimensions and @code{information} (@code{b}), the SI and binary prefixes, and about 270 units with @code{def/dimension}, @code{def/prefix} and @code{def/unit}—the definitions that quantity literals like @code{<120 km/h>}, unit-annotated types like @code{(double m/s)}, and @code{number-in} use at compile time and erase at run time. For units that arrive as text at run time, @code{string->measurement}, @code{measurement-in} and @code{convert-query} (e.g. @code{(convert-query "6000 m + 50 ft in mi")}) parse, check dimensions and convert against a runtime unit table.
|
||||
For more details, see @link["modules/std/units.html"]{@code{(std units)}}.
|
||||
|
||||
@subsection{std/array and std/simd}
|
||||
|
||||
Two modules for numbers in bulk, with the CPU's vector instructions. @code{(std array)} has whole-array operations: @code{array-sum}, @code{array-dot}, @code{array-min}, @code{array-max}, @code{array-add!}, @code{array-scale!}, @code{array-count}, @code{array-index-of} and @code{array-fill!}. Each is one loop in the runtime, which works on as many elements per instruction as a vector register holds when the element is a primitive number, and one at a time for a type such as @code{bigint}. At @code{double}, a sum adds lane by lane, so its last bits can differ from a left fold. @code{(std simd)} has the vectors themselves, for the loops that the kernels do not cover. @code{(Simd128 %a)} and @code{(Simd256 %a)} are .NET's @code{Vector128<T>} and @code{Vector256<T>}, with a name per element type: @code{Float64x4} is four doubles, @code{UInt8x32} is 32 bytes. The operations are the methods of one trait, @code{Simd}, at both widths: @code{simd-add}, @code{simd-mul}, the comparisons @code{simd-lt} and the others, which give masks, @code{simd-select}, @code{simd-count-true}, @code{simd-sum} and more. A vector is made with @code{simd256-load}, @code{simd128-splat} or @code{simd256-zero}, and @code{simd256-whole} says where the elements after the last whole vector start. A 128-bit vector is fast on every x86-64 and ARM64 CPU. A 256-bit vector needs AVX2, and @code{simd256-accelerated?} says whether the CPU has it. The section on performance in "Writing programs" says when vectors pay.
|
||||
For more details, see @link["modules/std/array.html"]{@code{(std array)}} and @link["modules/std/simd.html"]{@code{(std simd)}}.
|
||||
|
||||
@subsection{std/parallel and std/combine}
|
||||
|
||||
Data-parallel work across CPU cores on dedicated compute threads. @code{(std parallel)} splits any @code{Chunkable} collection (@code{Range}, @code{Vec}, @code{Array}, @code{IArray}, @code{Map}, @code{HashSet}, @code{OrderedMap} and @code{OrderedSet}) into parts without copying elements. @code{in-parts} runs a function on each part and returns a vector of the part results; @code{in-parallel} joins the part results in order using @code{Combinable} from @code{(std combine)}; and @code{par-map}, @code{par-filter} and @code{par-for-each} operate over a whole collection. For an array, @code{part->span} gives each worker a writable @code{(Span %a)} over its own non-overlapping slice so an array can be updated in place across cores without locks. @code{(std combine)} provides the associative @code{Combinable} trait (@code{combine} and @code{combine-all}) for types with one natural ordered combination: @code{Vec}, @code{List}, @code{IArray}, @code{string}, sets and tuples.
|
||||
For more details, see @link["modules/std/parallel.html"]{@code{(std parallel)}} and @link["modules/std/combine.html"]{@code{(std combine)}}.
|
||||
|
||||
@subsection{std/fmt}
|
||||
|
||||
Formatting in two layers: interpolation and layout. @code{println*} prints strings, ints, doubles, chars and bools written in one vec. The layout layer is made of blocks: a @code{Block} is one piece of output, and every combinator takes its content as a block and gives back a block. A string, an int and a character are blocks, and so is a vec of blocks, such as @code{["x = " 7 nl]}. With them you pad, cut and align text, wrap and justify paragraphs, format numbers, and lay out tables with borders, a header, and columns that fit the width. The section on formatting text in "Writing programs" is a guide, with what goes in the brackets and what to write when it does not compile.
|
||||
|
|
@ -117,6 +127,11 @@ For more details, see @link["modules/std/mutable/vec.html"]{@code{(std mutable v
|
|||
An inbox is how .NET code gives work to Bjolang. A thread that is not a fiber, such as a timer, a @code{FileSystemWatcher} or a Kestrel thread, posts an item and returns. A fiber takes the item with its own handlers, parameters and cancellation scope. An inbox always has a capacity, and a @code{WhenFull} policy says what a post does when the inbox is full. @code{inbox-call} is a request and a reply: the .NET caller gets a @code{Task}, and a fiber answers it.
|
||||
For more details, see @link["modules/std/inbox.html"]{@code{(std inbox)}}.
|
||||
|
||||
@subsection{std/interop}
|
||||
|
||||
Bridges for .NET APIs that want something a Bjolang function or value is not. A Bjolang function is a @code{Func} whose @code{void} return is the unit value; @code{as-delegate} and @code{as-delegate/type} wrap a function in another .NET delegate type such as @code{Action}, @code{EventHandler} or a library's custom delegate. @code{add-event-handler!} and @code{remove-event-handler!} subscribe a Bjolang function to a .NET object's event by name. @code{clr-default} produces .NET's @code{default(T)} (such as @code{null} for a reference type or a @code{Nullable<T>}) when a .NET call requires one.
|
||||
For more details, see @link["modules/std/interop.html"]{@code{(std interop)}}.
|
||||
|
||||
@subsection{std/ports and std/net}
|
||||
|
||||
@code{(std ports)} reads and writes bytes on the prelude's ports. The byte reads @code{read-some-evt}, @code{read-bytes-evt} and @code{eof-evt} are CML events: a read that loses its @code{choose} takes nothing, so a read can race a timeout. @code{read-line} and @code{read-bytes} can be used on the same port. The module also drains ports into lists and vecs, streams them as a @code{Seq} or a channel, and has @code{byte-pipe}, a pair of ports with no file and no network, for tests. @code{(std net)} gives a TCP connection as a pair of these ports, so a protocol written for ports also runs over TCP. @code{accept-evt} is an event too: a branch that loses accepts nothing.
|
||||
|
|
@ -158,6 +173,11 @@ For more details, see @link["modules/text/bjodat.html"]{@code{(text bjodat)}} an
|
|||
@code{(text json)} reads JSON, exactly as RFC 8259 has it, into a @code{Json}: a union with a case for each kind of value. @code{parse-json} reads a string and @code{read-json} a port. A document that is not JSON is an @code{Err} that says the line and column where it went wrong. A number without a fraction or an exponent is a @code{JsonInt}, and an object is a @code{(Map string Json)}, so the order of its keys is not kept. @code{(text json-codec)} reads a record straight from the tree and back: @code{def/json-type} declares a record and the instances of @code{->json} and @code{json->} for it, with defaults, optional fields and other names for the keys. An @code{Option} is @code{null} when it is @code{None}. There is no writer yet, so a @code{Json} cannot be written out as text.
|
||||
For more details, see @link["modules/text/json.html"]{@code{(text json)}} and @link["modules/text/json-codec.html"]{@code{(text json-codec)}}.
|
||||
|
||||
@subsection{text/csv and text/csv-codec}
|
||||
|
||||
@code{(text csv)} reads and writes CSV (RFC 4180), with configurable @code{#:separator}, @code{#:quote-char} and @code{#:strict?}. It reads in three layers: @code{csv-next!} pulls one row at a time, @code{csv-fold} and @code{csv-for-each} stream rows without keeping the document in memory, and @code{parse-csv} reads the whole input. Each row is an @code{(IArray string)}, and malformed input returns a @code{CsvError} with its line, row and field. @code{write-csv} and @code{csv->string} quote fields only when needed. @code{(text csv-codec)} maps rows to records and back: @code{def/csv-type} declares a record and its @code{CsvRecord} instance, matching columns by header name (or @code{#:csv} alias) or by position when @code{#:header? #f}, with @code{parse-csv-records}, @code{csv-for-each-record} and @code{write-csv-records}.
|
||||
For more details, see @link["modules/text/csv.html"]{@code{(text csv)}} and @link["modules/text/csv-codec.html"]{@code{(text csv-codec)}}.
|
||||
|
||||
@subsection{text/xml}
|
||||
|
||||
XML, in three modules over @code{System.Xml}. @code{(text xml)} is the tree: names, elements, texts, comments and processing instructions, built by @code{xml-element} and the others, and taken apart by views. @code{(text xml read)} reads XML in four layers: a stream of signals, a fold after SSAX, a tree of your own type, and @code{parse-xml}, which gives a @code{Document}. @code{xml-for-each} reads a large document one element at a time. The defaults are the safe ones: a document with a DOCTYPE is refused unless asked for, and nothing external is ever resolved. @code{(text xml write)} writes a node or a document to a port or a string. It chooses the namespace prefixes, and content that XML cannot hold is an @code{XmlWriteError} rather than an exception.
|
||||
|
|
|
|||
3
index.sz
3
index.sz
|
|
@ -11,7 +11,6 @@ types, traits, hygienic macros and bjoroutines, that compiles to C#.
|
|||
library module, made from the documentation the module was compiled with.
|
||||
@itemlist{
|
||||
@item{@link["modules/std/"]{@code{std}}: the standard library.}
|
||||
@item{@link["modules/text/"]{@code{text}}: data formats: bjodat, JSON
|
||||
and XML.}
|
||||
@item{@link["modules/text/"]{@code{text}}: data formats and encodings: bjodat, CSV, JSON and XML.}
|
||||
}}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue