goof-loop/README.md

# goof-loop - a scheme looping facility

goof-loops aims to be an amalgamation of the racket for loops and Alex Shinn's (chibi-loop). We are many that found racket's for loops a breeze of fresh air, but in the end their most general forms (for/fold and for/foldr) are kinda odd to work with. If you choose not to use those general for loops, you cannot express arbitrary transformations, like say a fibonacci sequence, since for clauses cannot reference eachother. goof-loop tries to fix this. 

Compared to foof-loop, some things are added. Apart from minor syntactic changes, subloops are supported. The best way is to show:

``` scheme
(define lst '((1 2) dud (3 4) (5 6)))
(loop ((:for a (in-list lst))
       (:when (pair? a))
       (:for b (in-list a))
       (:acc acc (summing b))))
;; => 21
```

 Any :when, :unless, :break, :final, :bind, :do or :subloop clause will break out a subloop if any subsequent for clauses are found.

## Beta warning

This is beta quality software, and some minor details are likely to change. I have gotten most kinks worked out though, but if I ever figure out how to include branching, the body-part of the macro will become a :body clause.

## Documentation

The current WIP documentation can be found here: https://bjoli.srht.site/doc.html

It is written in a weird markdown/xml chimaera. You can find it in documentation doc.xml (for the weird format) and documentation/doc.html for the slightly more accessible HTML format.


## Features

### Lexical order of clauses

``` scheme
(loop ((:for a (in-list 1 2 3)
       (:bind (b (expensive-operation1 a)))
       (:when (test? b))
       (:bind (c (expensive-operation2 b)))
       (:when test2? c)
       (:acc acc (listing c)))))
```

There is one caveat to this: some accumulating clauses (currently only vectoring with :length specified) have an implicit :break clause. This is tested AFTER the accumulation takes place. So: if the last position of the vector is set, the loop halts.

This of course also means accumulation is done before the body is executed, which is a bit counterintuitive.

``` scheme
(loop ((:for a (in-list '(1 2 3)))
       (:acc vec (vectoring a (:length 2)))
       ;; implicit :break (= vec-index 2)
       (:acc sum (summing a))))
;; => #(1 2) 1
```

### Ability to be explicit about returned values
If you have no "final expression", denoted by => expr, one is added that returns all final values of the :acc clauses in the loop. If none is present, the return value is unspecified.

``` scheme
(loop ((:for a (up-from 1 5))
       (:acc sum (summing a))))
;; => 10

(loop ((:for a (up-from 1 5))
       (:acc sum (summing a)))
  => (- sum))
;; => -10
```

### Loop naming to make it "fold right"

You can of course still have a larger control of when to loop by naming your loop:

``` scheme
(loop loopy-loop ((:for a (up-from 1 (to 11))))
  => '()
  (if (odd? a)
      (cons (* a (- a)) (loopy-loop))
      (cons (* a a) (loopy-loop))))
;; => (-1 4 -9 16 -25 36 -49 64 -81 100)
```

Replace that cons with stream-cons and you have a lazy construct.

### Named updates

``` scheme
;; Shamelessly stolen from Taylor Campbell's foof-loop documentation
(define (partition list predicate)
  (loop continue ((:for element (in-list list))
                  (:acc satisfied (folding '()))
                  (:acc unsatisfied (folding '())))
     => (values (reverse satisfied)
                (reverse unsatisfied))
     (if (predicate element)
         (continue (=> satisfied (cons element satisfied)))
         (continue (=> unsatisfied (cons element unsatisfied))))))

(partition '(1 2 3 4 5) odd?)
;; => (1 3 5) (2 4)
```

### Exposing loop variables

The iterator protocol allows exposing the loop variables

``` scheme
(loop name ((:for elt pair (in-list '(1 2 3))))
  => '()
  (if (null? (cdr pair))
      (list elt)
      (cons* elt ': (name))))
      
;; => (1 : 2 : 3)
```
In the above example, pair is bound to the pair where elt is the car.

### Higher order loop protocol

goof supports a higher order looping protocol, based on srfi-158 generators:

``` scheme

(loop ((:for food (in-list '(banana cake grape cake bean cake)))
       (:for true? (in-cycle (in-list '(#t #f)))))
  (display "The ") 
  (display food)
  (display " is a ")
  (if true?
     (display food)
     (display "LIE!"))
  (newline))
```

In the above example true? never ends, but restarts every time the list is exhausted. 

### :final is context sensitive (compared to Racket's #:final)

``` scheme

(loop ((:for elt (in-list '( 1 2 3)))
       (:final (= elt 2))
       (:for ab (in-list '(a b)))
       (:acc acc (listing (cons elt ab))))
  
;; => ((1 . a) (1 . b) (2 . a) (2 . b))
```

The racket counterpart would result in ((1 . a) (1 . b) (2 . a)). This comes at :final clauses being marginally less efficient than racket's #:final.

### :for-clauses can refer to eachother

The iterative fibonacci loop is weird to write using for/fold. goof fixes this:
``` scheme
(loop ((:for a (in 0 b))
       (:for b (in 1 (+ a b)))
       (:for count (up-from 0 (:to 100)))
       (:acc acc (listing b)))
  (display b) (newline))
```
### Accumulators and arbitrary code can be placed in subloops

``` scheme
(loop ((:for a (in-list '(1 2 3)))
       (:acc aa (summing a))
       (:do (display "Entering subloop!") (newline))
       :subloop
       (:for b (up-from a (:to (+ a 2))))
       (:acc ab (listing b))))
;; |> Entering subloop! 
;; |> Entering subloop! 
;; |> Entering subloop! 
;; => 6 (1 2 2 3 3 4)
```

### Pattern matching

For clauses which bind "body bindings" (every one except (in ...), in-port, in-generator and in-file) can use pattern matching based on Alex Shinn's excellent match.scm.

``` scheme
(loop ((:for (key . val) (in-list '((a . 1) (b . 2) c . 3)))
       (:acc sum (summing val))))
;; => 6

This also works with :bind clauses.
```

### Simple forms
I also provide simplified forms for many common operations. Omitting :for is allowed, and :acc clauses are not allowed.

``` scheme
(loop/list ((a (up-from 0 3)))
  a)
;; => (0 1 2)

(loop/sum ((:for a (up-from 1 4))) a)
;; => 6

(loop/product ((a (in-list '(2 3 4))))
  a)
;; => 24

(loop/first ((a (in-list '(a b c 3 4 d))) (:when (integer? a)))
  (display a)
  a)
;; => displays 3 and returns 3.

(loop/last ((a (in-list '(a b c d e f))) (:break (eq? a 'e)))
  a)
;; => 'd

(loop/and ((a (in-list '(1 2 3 'error))))
  (< a 3))
;; => #f

(loop/or ((a (in-list '(1 2 3 4))))
  (symbol? a))
;; => #f

(loop/list/parallel ((a (in-list '(42 41 43))))
  (expensive-function a))
;; => same result as loop/list, but faster if the problem parallelizes well  

```

## Speed

Speed is good. Despite the rather involved expansion you can see in the documentation, due to inlining and dead-code elimination, the actual expansion shows some good code:

``` scheme
 ,expand (loop ((:for a (in-list '(1 2 3 4)))
              (:when (even? a)) 
              (:acc acc (listing a))))
$0 = (let* ((final-fun
         (lambda (acc) ((@@ (goof) values) acc)))
       (tmp-kons (@@ (goof) cons)))
  (let loop ((cursor-1 '()) (cursor '(1 2 3 4)))
    (if ((@@ (goof) not) ((@@ (goof) pair?) cursor))
      (final-fun ((@@ (goof) reverse) cursor-1))
      (let ((a ((@@ (goof) car) cursor))
            (succ ((@@ (goof) cdr) cursor)))
        (if (even? a)
          (let ((cursor (tmp-kons a cursor-1)))
            (if #f #f)
            (loop cursor succ))
          (loop cursor-1 succ))))))

;; This is mostly fluff that is removed using DCE, unrolling and inlining:
> ,opt (loop ((:for a (in-list '(1 2 3 4)))
              (:when (even? a))
              (:acc acc (listing a))))

$1 = (let* ((cursor (list 2)) (cursor (cons 4 cursor)))
  ((@@ (goof) reverse) cursor))
  
;; well dang, the loop was optimized away almost completely...
    
;; loop/list, being less general, produces faster code that can be more easily optimized
> ,opt (loop/list ((a (in-list '(1 2 3 4)))
                   (:when (even? a)))
         a)
$2 = (list 2 4)

;; Removing the opportunity to completely optimize the loop away
> ,opt (loop/list ((a (in-list (read)))
                   (:when (even? a)))
         a)

;; This is actually the preferred way to do it in guile. Guile re-sizes the stack, so no stack overflows
$3 = (let loopy-loop ((cursor (read)))
  (if (pair? cursor)
    (let ((a (car cursor)) (succ (cdr cursor)))
      (if (even? a)
        (cons a (loopy-loop succ))
        (loopy-loop)))))


;; The code expansion of the partition procedure above produces
(define (partition list predicate)
  (let loopy-loop ((satisfied '()) (unsatisfied '()) (cursor list))
    (if (pair? cursor)
      (let ((element (car cursor)) (succ (cdr cursor)))
        (if (predicate element)
          (loopy-loop (cons element satisfied)
                      unsatisfied
                      succ)
          (loopy-loop satisfied
                      (cons element unsatisfied)
                      succ)))
      (values (reverse satisfied) (reverse unsatisfied)))))


```


## Differences from foof-loop

This used to be a pretty vast collection of examples. goof-loof is now different enough from foof loop that you can't expect to carry your foof-loop skills over to goof-loop. There are however two notable regressions.

### Regressions compared to foof-loop
only accumulating clauses are visible in the final-expression. This is due to sequence clauses not being promoted through to outer loops (since they should not keep their state if an inner loop is exited).

Due to clause reordering, positional updates are not supported. If you want to update your loop vars, do so using named update (see below). 

## Todo
Tests!

Finish documentation.

Figure out if I can do anything about branching. I would love to remove the body and just have loop clauses. I don't think I can do that without some serious voodoo if I want to keep the current syntax. One idea would be to define all accumulators in the start of the loop, and then bind identifiers using local macros:

``` scheme
(loop (accumulators ...)
      (clauses ...)
      => final-expr)

(loop ((vectoring a)
       (listing b))
       (:for i (up-from 1 11))
       (:save b (* i i))
       (:if (odd? i)
            (:subloop (:for ab (in-list '(a b)))
                      (:save a (cons i ab)))
            (:subloop (:for cd (in-list '(c d)))
                      (:save a (cons i cd))))
       (:save a 'next))
(1 4 9 16 ...)
#((1 . a) (1 . b) next (2 . c) (2 . d) next ...)
```

But this solution isn't great. The current situation isn't either, though. The body is executed AFTER all other clauses and is only really useful for things like branching, which would be much nicer to have in the clauses. The few times one wants a right fold, a simple :body clause will do the trick. 

## foof, what a guy
I have previously expressed some admiration for Alex Shinn and I will do it again. The source of chibi loop is extremely elegant, and all but the hairiest part is written in syntax-rules. Not only has he written my two favourite SRFIs, his input in all the other discussions I have seen is always on-point, pragmatic and generally fantastic. He neither knows of this project, nor embraces it in any way. Y'all should go look at the source of (chibi loop) though.

## Licence
goof started off by copying the iterator protocol of (chibi loop), and things sort of went downhill from there. Despite this, there is still quite a lot of code (especially in iterators.scm) that I didn't write myself. I only made it ugly. Thus goof is licensed under the same BSD-styled license Alex uses for chibi-loop.