lsp-bjolang/README.md

BjolangLsp

A language server for Bjolang. It does five things:

• Diagnostics: it checks a file with the compiler's frontend, as bjolang --check does, and sends the errors and warnings to the editor. It checks the editor's text, saved or not: when a file is opened or saved, and 400 ms after typing pauses. Every open file is read from its buffer, so an unsaved include or .bjodoc counts too.
• An outline (textDocument/documentSymbol): the file's top-level definitions, with a type's constructors or fields, a trait's methods and an impl's functions under them.
• Go to definition (textDocument/definition): a local name goes to where it is bound, the innermost binding of it; a top-level name to the file's own definition, else one in a module it imports or includes, searched nearest first, and then the prelude.
• Hover (textDocument/hover) for the same names. A top-level name shows the (: name Type) declaration and the definition's head as written, then its doc from the module's (:doc ...) forms and .bjodoc files, with the doc markup (@code, @strong, @em, @itemlist) as Markdown. A constructor or a field shows its type's doc, with its own case or field line. A builtin, which the compiler defines rather than a module, shows its type and its doc from the prelude's docs. A local name shows its binding as written, a typed parameter with its type, and what binds it.
• Completion (textDocument/completion): the local names in scope, then the top-level names of the file and of every module it sees, the builtins, and the special forms, each name once and the nearest first. An item replaces the whole word before the cursor, as the lexer reads a word, so string-> is one. Its doc is the hover's, sent along for a local and when the client asks for anything else (completionItem/resolve). Nothing is offered in a comment or a string.

And one request of its own, bjolang/expandMacro: the expansion of the macro call at a position, one step or all the way down, from the last check of the file, which runs first if a change is waiting for it. The compiler reports each expansion it makes through Macro.observe while it checks, so an expansion costs nothing beyond the check. Names hygiene renamed are shown as the template wrote them, unless hygienic asks for the renamed ones. loop, seql, ->, defun and the parser's other forms are not macros, and have no expansion.

The outline, definitions, hovers and completions read only as far as the reader: no macro is expanded and nothing is type checked, so they answer on every request. A form being typed is read as if it were closed; text that does not read even so leaves the last outline in place.

Local names are the ones the parser's own forms bind: parameters of a defun, fun and the other def… heads, let, let*, letrec, let/mono, named let and with-open bindings, with-return, match patterns, the clauses of loop, seql and a comprehension, and def or defun inside a body, for the forms after it.

Building

It expects the Bjolang checkout beside this one, at ../Bjolang, and runs that compiler in-process through a project reference.

(cd ../Bjolang && ./build_compiler.sh && ./build_std.sh)
dotnet build -c Release
test/smoke.py

The server is bin/Release/net10.0/BjolangLsp.dll, started as dotnet bin/Release/net10.0/BjolangLsp.dll. It talks LSP on stdin and stdout, and logs to stderr.

How it finds things

• The standard library and the runtime: through the checkout path baked in at build time. BJOLANG_HOME overrides it.
• Staleness: an imported module is judged against the installed compiler, ../Bjolang/bin/compiler/Bjolang.dll, not against the server's own copy of the compiler, and a stale import is built by running that compiler. Built by the server, its build record would name the server, and the command line would build it again.
• Unsaved text: the compiler reads a source file through Paths.readSource, and the server hands it the open buffers in Paths.sourceOverrides for the length of a check.

Emacs (Doom, lsp-mode)

(after! lsp-mode
  (add-to-list 'lsp-language-id-configuration '(bjo-mode . "bjolang"))
  (lsp-register-client
   (make-lsp-client
    :new-connection (lsp-stdio-connection
                     '("dotnet" "/home/linis/Programmering/BjolangLsp/bin/Release/net10.0/BjolangLsp.dll"))
    :activation-fn (lsp-activate-on "bjolang")
    :server-id 'bjolang-lsp)))

(add-hook 'bjo-mode-hook #'lsp)

emacs/bjolang-dape.el debugs a program with dape and netcoredbg:

(load "/home/linis/Programmering/BjolangLsp/emacs/bjolang-dape.el")

M-x dape RET bjolang RET in a file with a main builds it with -d and runs it under the debugger. Breakpoints in the .bjo file bind, stepping goes by its lines, and the call stack names them. The locals are the generated C#'s, compiler temporaries included, and values show as their C# types, which expand into their fields.

With eglot instead:

(add-to-list 'eglot-server-programs
             '(bjo-mode . ("dotnet" "/home/linis/Programmering/BjolangLsp/bin/Release/net10.0/BjolangLsp.dll")))

What it does not do yet

• Columns. Most diagnostics carry only a line, and the whole line is marked.
• A check of a large file waits for the requests before it, and they for it: everything that touches the compiler runs on one thread, because the compiler's state is global.
• Names a macro binds, at top level or locally. A loop variable is taken to be in scope across its whole loop, which is wider than the loop allows.
• Which definition. A name defined in two imported modules goes to the nearer one, which is not always the one the compiler would pick, and a module the file does not export from is searched like one it does.
• Errors in another file. A diagnostic located in an imported or included file is shown on the first line of the file being checked, with its location in the message.
• Hover for a special form (record-ref, loop, match), which has no definition to find and no doc.
• Types that are inferred rather than written. A hover shows the declaration if there is one, and nothing about a name without it.
• Completion of record fields after record-ref, of module paths in an import, or of keyword arguments. A string that began on an earlier line is not seen as one, so a multi-line string gets completions.
• Find references, rename.
• Packages: no --roots is passed, so only the standard library resolves.
• Stale imports are built by the installed compiler, which was tested only for the standard library being current, not for a stale import.