A language server for Bjolang

Runs the compiler in-process against the Bjolang checkout beside it, and
answers diagnostics, an outline, go to definition, hover and completion,
plus a request of its own for macro expansion.

Diagnostics come from the compiler's own check of the editor's text,
saved or not: on open, on save, and 400 ms after typing pauses. The
outline, definitions, hovers and completions read only as far as the
reader, with no type checking, so they answer on every request: the
top-level names of the file, its imports and the prelude, local names
from the binding forms the parser knows, the declarations and docs as
written, and the builtins with their types. emacs/bjolang-lsp.el shows a
macro call's expansion, one step or all of it, in a buffer.

test/smoke.py drives the server as an editor would.

🤖 Generated with [ECA](https://eca.dev)

Co-Authored-By: eca-agent <git@eca.dev>

README.md

@@ -0,0 +1,128 @@
+# BjolangLsp
+
+A language server for Bjolang. It does five things:
+
+- Diagnostics: it checks a file with the compiler's frontend, as
+  `bjolang --check` does, and sends the errors and warnings to the editor.
+  It checks the editor's text, saved or not: when a file is opened or
+  saved, and 400 ms after typing pauses. Every open file is read from its
+  buffer, so an unsaved include or `.bjodoc` counts too.
+- An outline (`textDocument/documentSymbol`): the file's top-level
+  definitions, with a type's constructors or fields, a trait's methods and an
+  `impl`'s functions under them.
+- Go to definition (`textDocument/definition`): a local name goes to where
+  it is bound, the innermost binding of it; a top-level name to the file's
+  own definition, else one in a module it imports or includes, searched
+  nearest first, and then the prelude.
+- Hover (`textDocument/hover`) for the same names. A top-level name shows
+  the `(: name Type)` declaration and the definition's head as written, then
+  its doc from the module's `(:doc ...)` forms and `.bjodoc` files, with the
+  doc markup (`@code`, `@strong`, `@em`, `@itemlist`) as Markdown. A
+  constructor or a field shows its type's doc, with its own `case` or
+  `field` line. A builtin, which the compiler defines rather than a module,
+  shows its type and its doc from the prelude's docs. A local name shows
+  its binding as written, a typed parameter with its type, and what binds
+  it.
+- Completion (`textDocument/completion`): the local names in scope, then
+  the top-level names of the file and of every module it sees, the
+  builtins, and the special forms, each name once and the nearest first. An
+  item replaces the whole word before the cursor, as the lexer reads a word,
+  so `string->` is one. Its doc is the hover's, sent along for a local and
+  when the client asks for anything else (`completionItem/resolve`).
+  Nothing is offered in a comment or a string.
+
+And one request of its own, `bjolang/expandMacro`: the expansion of the
+macro call at a position, one step or all the way down, from the last check
+of the file, which runs first if a change is waiting for it. The compiler
+reports each expansion it makes through `Macro.observe` while it checks, so
+an expansion costs nothing beyond the check. Names hygiene renamed are shown
+as the template wrote them, unless `hygienic` asks for the renamed ones.
+`loop`, `seql`, `->`, `defun` and the parser's other forms are not macros,
+and have no expansion.
+
+The outline, definitions, hovers and completions read only as far as the
+reader: no macro is expanded and nothing is type checked, so they answer on
+every request. A form being typed is read as if it were closed; text that
+does not read even so leaves the last outline in place.
+
+Local names are the ones the parser's own forms bind: parameters of a
+`defun`, `fun` and the other `def…` heads, `let`, `let*`, `letrec`,
+`let/mono`, named `let` and `with-open` bindings, `with-return`, `match`
+patterns, the clauses of `loop`, `seql` and a comprehension, and `def` or
+`defun` inside a body, for the forms after it.
+
+## Building
+
+It expects the Bjolang checkout beside this one, at `../Bjolang`, and runs
+that compiler in-process through a project reference.
+
+```
+(cd ../Bjolang && ./build_compiler.sh && ./build_std.sh)
+dotnet build -c Release
+test/smoke.py
+```
+
+The server is `bin/Release/net10.0/BjolangLsp.dll`, started as
+`dotnet bin/Release/net10.0/BjolangLsp.dll`. It talks LSP on stdin and
+stdout, and logs to stderr.
+
+## How it finds things
+
+- The standard library and the runtime: through the checkout path baked in
+  at build time. `BJOLANG_HOME` overrides it.
+- Staleness: an imported module is judged against the installed compiler,
+  `../Bjolang/bin/compiler/Bjolang.dll`, not against the server's own copy of
+  the compiler, and a stale import is built by running that compiler. Built
+  by the server, its build record would name the server, and the command line
+  would build it again.
+- Unsaved text: the compiler reads a source file through
+  `Paths.readSource`, and the server hands it the open buffers in
+  `Paths.sourceOverrides` for the length of a check.
+
+## Emacs (Doom, lsp-mode)
+
+```elisp
+(after! lsp-mode
+  (add-to-list 'lsp-language-id-configuration '(bjo-mode . "bjolang"))
+  (lsp-register-client
+   (make-lsp-client
+    :new-connection (lsp-stdio-connection
+                     '("dotnet" "/home/linis/Programmering/BjolangLsp/bin/Release/net10.0/BjolangLsp.dll"))
+    :activation-fn (lsp-activate-on "bjolang")
+    :server-id 'bjolang-lsp)))
+
+(add-hook 'bjo-mode-hook #'lsp)
+```
+
+With eglot instead:
+
+```elisp
+(add-to-list 'eglot-server-programs
+             '(bjo-mode . ("dotnet" "/home/linis/Programmering/BjolangLsp/bin/Release/net10.0/BjolangLsp.dll")))
+```
+
+## What it does not do yet
+
+- Columns. Most diagnostics carry only a line, and the whole line is marked.
+- A check of a large file waits for the requests before it, and they for it:
+  everything that touches the compiler runs on one thread, because the
+  compiler's state is global.
+- Names a macro binds, at top level or locally. A loop variable is taken to
+  be in scope across its whole loop, which is wider than the loop allows.
+- Which definition. A name defined in two imported modules goes to the
+  nearer one, which is not always the one the compiler would pick, and a
+  module the file does not export from is searched like one it does.
+- Errors in another file. A diagnostic located in an imported or included
+  file is shown on the first line of the file being checked, with its
+  location in the message.
+- Hover for a special form (`record-ref`, `loop`, `match`), which has no
+  definition to find and no doc.
+- Types that are inferred rather than written. A hover shows the
+  declaration if there is one, and nothing about a name without it.
+- Completion of record fields after `record-ref`, of module paths in an
+  `import`, or of keyword arguments. A string that began on an earlier line
+  is not seen as one, so a multi-line string gets completions.
+- Find references, rename.
+- Packages: no `--roots` is passed, so only the standard library resolves.
+- Stale imports are built by the installed compiler, which was tested only
+  for the standard library being current, not for a stale import.