lsp-bjolang/README.md

# BjolangLsp

A language server for Bjolang. It checks a file against its project's
packages (see "Projects"), and it does five things:

- Diagnostics: it checks a file with the compiler's frontend, as
  `bjolang --check` does, and sends the errors and warnings to the editor.
  It checks the editor's text, saved or not: when a file is opened or
  saved, and 400 ms after typing pauses. Every open file is read from its
  buffer, so an unsaved include or `.bjodoc` counts too. A diagnostic marks
  the form at the line and column the compiler names: an atom or a string
  whole, a bracketed form to its closing bracket, and one over several lines
  as far as its head, `(defun` or `(match`.
- An outline (`textDocument/documentSymbol`): the file's top-level
  definitions, with a type's constructors or fields, a trait's methods and an
  `impl`'s functions under them.
- Go to definition (`textDocument/definition`): a local name goes to where
  it is bound, the innermost binding of it; a top-level name to the file's
  own definition, else one in a module it imports or includes, searched
  nearest first, and then the prelude.
- Hover (`textDocument/hover`) for the same names. A top-level name shows
  the `(: name Type)` declaration and the definition's head as written, then
  its doc from the module's `(:doc ...)` forms and `.bjodoc` files, with the
  doc markup (`@code`, `@strong`, `@em`, `@itemlist`) as Markdown. A
  constructor or a field shows its type's doc, with its own `case` or
  `field` line. A builtin, which the compiler defines rather than a module,
  shows its type and its doc from the prelude's docs. A local name shows
  its binding as written, a typed parameter with its type, and what binds
  it.
- Completion (`textDocument/completion`): the local names in scope, then
  the top-level names of the file and of every module it sees, the
  builtins, and the special forms, each name once and the nearest first. An
  item replaces the whole word before the cursor, as the lexer reads a word,
  so `string->` is one. Its doc is the hover's, sent along for a local and
  when the client asks for anything else (`completionItem/resolve`).
  Nothing is offered in a comment or a string.

And one request of its own, `bjolang/expandMacro`: the expansion of the
macro call at a position, one step or all the way down, from the last check
of the file, which runs first if a change is waiting for it. The compiler
reports each expansion it makes through `Macro.observe` while it checks, so
an expansion costs nothing beyond the check. Names hygiene renamed are shown
as the template wrote them, unless `hygienic` asks for the renamed ones.
`loop`, `seql`, `->`, `defun` and the parser's other forms are not macros,
and have no expansion.

The outline, definitions, hovers and completions read only as far as the
reader: no macro is expanded and nothing is type checked, so they answer on
every request. A form being typed is read as if it were closed; text that
does not read even so leaves the last outline in place.

Local names are the ones the parser's own forms bind: parameters of a
`defun`, `fun` and the other `def…` heads, `let`, `let*`, `letrec`,
`let/mono`, named `let` and `with-open` bindings, `with-return`, `match`
patterns, the clauses of `loop`, `seql` and a comprehension, and `def` or
`defun` inside a body, for the forms after it.

## Building

It expects the Bjolang checkout beside this one, at `../Bjolang`, and runs
that compiler in-process through a project reference.

```
(cd ../Bjolang && ./build_compiler.sh && ./build_std.sh)
dotnet build -c Release
test/smoke.py
```

The server is `bin/Release/net10.0/BjolangLsp.dll`, started as
`dotnet bin/Release/net10.0/BjolangLsp.dll`. It talks LSP on stdin and
stdout, and logs to stderr.

## Projects

A file in a project, a directory with a `manifest.bjodat` at or above it, is
checked against the project's packages, as `bjo` compiles it: the roots file
`.bjo/roots` that says where each package is, `.bjo/frameworks` for the .NET
shared frameworks the packages declared, and `.bjo/nuget/root` for restored
NuGet packages. `bjo` writes them; `bjo fetch` is enough. Imports from the
packages then check, and go to definition, hover and completion reach them.

A compiler process serves one roots file, so a server serves the project of
the first file it is asked about, from start to exit. A warning on the first
line of a file says when it cannot serve that file as the command line
would:

- the project's packages are not fetched yet: run `bjo fetch`, then restart
  the server;
- the roots file changed after the server started, as `bjo fetch` after a
  change to the manifest does: restart the server;
- the file is in another project than the one the server serves: open it in
  a server of its own, which lsp-mode and eglot do per project root.

## How it finds things

- The standard library and the runtime: through the checkout path baked in
  at build time. `BJOLANG_HOME` overrides it.
- Staleness: an imported module is judged against the installed compiler,
  `../Bjolang/bin/compiler/Bjolang.dll`, not against the server's own copy of
  the compiler, and a stale import is built by running that compiler. Built
  by the server, its build record would name the server, and the command line
  would build it again.
- Unsaved text: the compiler reads a source file through
  `Paths.readSource`, and the server hands it the open buffers in
  `Paths.sourceOverrides` for the length of a check.

## Emacs (Doom, lsp-mode)

```elisp
(after! lsp-mode
  (add-to-list 'lsp-language-id-configuration '(bjo-mode . "bjolang"))
  (lsp-register-client
   (make-lsp-client
    :new-connection (lsp-stdio-connection
                     '("dotnet" "/home/linis/Programmering/BjolangLsp/bin/Release/net10.0/BjolangLsp.dll"))
    :activation-fn (lsp-activate-on "bjolang")
    :server-id 'bjolang-lsp)))

(add-hook 'bjo-mode-hook #'lsp)
```

`emacs/bjolang-dape.el` debugs a program with `dape` and
[netcoredbg](https://github.com/Samsung/netcoredbg/releases):

```elisp
(load "/home/linis/Programmering/BjolangLsp/emacs/bjolang-dape.el")
```

`M-x dape RET bjolang RET` in a file with a `main` builds it with `-d` and
runs it under the debugger. Breakpoints in the `.bjo` file bind, stepping
goes by its lines, and the call stack names them. The locals are the
generated C#'s, compiler temporaries included, and values show as their C#
types, which expand into their fields.

With eglot instead:

```elisp
(add-to-list 'eglot-server-programs
             '(bjo-mode . ("dotnet" "/home/linis/Programmering/BjolangLsp/bin/Release/net10.0/BjolangLsp.dll")))
```

## What it does not do yet

- Exact ends. The compiler says where a diagnostic starts, and the form that
  starts there is marked, as far as its head when it spans several lines.
  That is the form the message is about in most cases, not in all.
- A check of a large file waits for the requests before it, and they for it:
  everything that touches the compiler runs on one thread, because the
  compiler's state is global.
- Names a macro binds, at top level or locally. A loop variable is taken to
  be in scope across its whole loop, which is wider than the loop allows.
- Which definition. A name defined in two imported modules goes to the
  nearer one, which is not always the one the compiler would pick, and a
  module the file does not export from is searched like one it does.
- Errors in another file. A diagnostic located in an imported or included
  file is shown on the first line of the file being checked, with its
  location in the message.
- Hover for a special form (`record-ref`, `loop`, `match`), which has no
  definition to find and no doc.
- Types that are inferred rather than written. A hover shows the
  declaration if there is one, and nothing about a name without it.
- Completion of record fields after `record-ref`, of module paths in an
  `import`, or of keyword arguments. A string that began on an earlier line
  is not seen as one, so a multi-line string gets completions.
- Find references, rename.
- More than one project. A server serves the project of the first file it is
  asked about; see "Projects" above.
- Stale imports are built by the installed compiler, which was tested only
  for the standard library being current, not for a stale import.