Big change: lexical scoping

This introduces lexical scoping of for clauses. See README.md

README.md

@@ -1,17 +1,6 @@
 # goof-loop - a scheme looping facility
 
-goof-loops aims to be an amalgamation of the racket for loops and Alex Shinn's (chibi-loop). We are many that found racket's for loops a breeze of fresh air, but in the end their most general forms (for/fold and for/foldr) are kinda odd to work with. If you choose not to use those general for loops, you cannot express arbitrary transformations, like say a fibonacci sequence, since for clauses cannot reference eachother. goof-loop tries to fix this:
+goof-loops aims to be an amalgamation of the racket for loops and Alex Shinn's (chibi-loop). We are many that found racket's for loops a breeze of fresh air, but in the end their most general forms (for/fold and for/foldr) are kinda odd to work with. If you choose not to use those general for loops, you cannot express arbitrary transformations, like say a fibonacci sequence, since for clauses cannot reference eachother. goof-loop tries to fix this.
-
-```
-(loop ((:for a (in 0 b))
-       (:for b (in 1 (+ a b)))
-       (:for count (up-from 0 (to 1000)))
-       (:acc acc (listing b)))
-  => acc
-  (display b) (newline))
-```
-
-The above example will display and accumulate the 1000 first fibonacci numbers. Doing the same thing in racket requires you to manually handle all the state in fold-variables using for/fold. It is a simple example, but proves the usefulness of goof-loop.
 
 Compared to foof-loop, some things are added. Apart from minor syntactic changes, subloops are supported. The best way is to show:
 
@@ -26,18 +15,6 @@ Compared to foof-loop, some things are added. Apart from minor syntactic changes
 
 This will sum all the sublists of lst and produce the result 21. Any :when, :unless, :break, :final, or :subloop clause will break out a subloop if any subsequent for clauses are found.
 
-Accumulators can be in any of the loop's stages:
-
-```
-(loop ((:for a (in-list '(1 2 3)))
-       (:acc aa (summing a))
-       :subloop
-       (:for b (up-from a (to (+ a 2))))
-       (:acc ab (listing b)))
-  => (values aa ab))
-;; => (values 6  (1 2 2 3 3 4))
-```
-
 ## Beta warning
 
 This is beta quality software, and some minor details are likely to change. I have gotten most kinks worked out though.
@@ -51,6 +28,11 @@ It is written in a weird markdown/xml chimaera. You can find it in documentation
 
 ## Differences from foof-loop
 
+### lexical
+
+foof-loop has a lot of code movement going on, and it can be hard to understand exactly where things end up. goof employs a more strict lexical hierarchy. The following is not possible in (chibi loop):
+d into only after the above clauses have been evaluated.
+
 ### syntactical
 
 for-clauses are split into :for and :acc clauses. This is because the addition of subloops means we have to treat accumulators differently.
@@ -90,8 +72,20 @@ Due to clause reordering, positional updates are not supported. If you want to u
  guard was a procedure, but now it is an expression.
 
     (with var 10 (- var 1) negative?) => (:for var (in 10 (- var 10) (negative? var)))
+## Features
 
-### similarities
+### Lexical order of clauses
+
+    (loop ((:for a (in-list 1 2 3)
+           (:bind b (expensive-operation1 a))
+           (:when (test? b))
+           (:bind c (expensive-operation2 b))
+           (:when test2? c)
+           (:acc acc (listing c))))
+      => acc)
+      
+
+### Loop naming to make it "fold right"
 
 You can of course still have a larger control of when to loop by naming your loop:
 
@@ -105,7 +99,7 @@ You can of course still have a larger control of when to loop by naming your loo
 ;; => (-1 4 -9 16 -25 36 -49 64 -81 100)
 ```
 
-Named updates also work.
+### Named updates
 
 ```
 ;; Shamelessly stolen from Taylor Campbell's foof-loop documentation
@@ -123,6 +117,61 @@ Named updates also work.
 ;; => (values (1 3 5) (2 4))
 ```
 
+### Exposing loop variables
+
+The iterator protocol allows exposing the loop variables
+
+```
+(loop name ((:for elt pair (in-list '(1 2 3))))
+  => '()
+  (if (null? (cdr pair))
+      (list elt)
+      (cons* elt ': (name))))
+      
+;; => (1 : 2 : 3)
+```
+
+### :final is context sensitive (compared to Racket's #:final)
+
+``` scheme
+
+(loop ((:for elt (in-list '( 1 2 3)))
+       :final (= elt 2)
+       (:for ab (in-list '(a b)))
+       (:acc acc (listing (cons elt ab)))
+  => acc))
+  
+;; => ((1 . a) (1 . b) (2 . a) (2 . b))
+```
+
+The racket counterpart would result in ((1 . a) (1 . b) (2 . a))
+
+### for-clauses can refer to eachother
+
+The iterative fibonacci loop is weird to write using for/fold. goof fixes this:
+``` scheme
+(loop ((:for a (in 0 b))
+       (:for b (in 1 (+ a b)))
+       (:for count (up-from 0 (to 100)))
+       (:acc acc (listing b)))
+  => acc
+  (display b) (newline))
+```
+### Accumulators and arbitrary code can be placed in subloops
+
+``` scheme
+(loop ((:for a (in-list '(1 2 3)))
+       (:acc aa (summing a))
+       (:do (display "Entering subloop!") (newline))
+       :subloop
+       (:for b (up-from a (:to (+ a 2))))
+       (:acc ab (listing b)))
+  => (values aa ab))
+;; => 6 (1 2 2 3 3 4)
+```
+
+
+
 ### Simple forms
 I also provide simplified forms for many common operations. Omitting :for is allowed, and :acc clauses are not allowed.
 
@@ -161,9 +210,9 @@ I also provide simplified forms for many common operations. Omitting :for is all
 
 ```
 
-### Speed
+## Speed
 
-Speed is good. Despite the rather involved expansion you can see in the documentation, due to dead-code elimination, the actual expansion shows some good code:
+Speed is good. Despite the rather involved expansion you can see in the documentation, due to inlining and dead-code elimination, the actual expansion shows some good code:
 
 ```
 > ,opt (loop ((:for a (in-list '(1 2 3 4)))
@@ -220,8 +269,6 @@ Tests!
 
 Finish documentation.
 
-add generator support for all provided iterators
-
 ## foof, what a guy
 
 I have previously expressed some admiration for Alex and I will do it again. The source of chibi loop is extremely elegant, and all but the hairiest part is written in syntax-rules. Not only has he written my two favourite SRFIs, his input in all the other discussions I have seen is always on-point, pragmatic and generally fantastic. He neither knows of this project, nor embraces it in any way. Y'all should go look at the source of (chibi loop) though.

goof-impl.scm

@@ -29,7 +29,7 @@
 
 (define-aux-syntaxes
   ;; Auxiliary syntax for the loop clauses
-  :when :unless :break :final :bind :subloop :for :acc
+  :when :unless :break :final :bind :do :subloop :for :acc
   ;; Auxiliary syntax for the iterators.
   :gen
   ;; auxiliary auxiliary syntax
@@ -121,7 +121,7 @@
 
 ;; cl sorts all the clauses into subloops and positions everything where it should be.
 (define-syntax cl
-  (syntax-rules (=> :for :acc :when :unless :break :final :bind :subloop)
+  (syntax-rules (=> :for :acc :when :unless :break :final :do :bind :subloop)
     ((_ orig name l a v c r f ff user () => expr . body)
      (emit orig name l a v c r f ff user expr . body))
     ((_ orig name l a v c r f ff user () . body)
@@ -143,8 +143,12 @@
      (cl orig name l a v c r f ff ((cur-ub ... (:break expr)) . ub-rest) (clauses ...) . body))
     ;; user final
     ;; This pushes a #t to the user when expression, thus forcing a subloop if a for-clause is found afterwards.
-    ((_ orig name l a v c r f ff ((cur-uw ...) . uw-rest)((:final expr) clauses ...) . body)
+    ((_ orig name l a v c r f ff user ((:final expr) clauses ...) . body)
-     (cl orig name l a v c r f ff ((cur-uw ... (:final expr)) . uw-rest) (clauses ...) . body))
+     (final :acc ((_) (expr)) cl-next/acc orig name l a v c r f ff user (clauses ...) . body))
+
+    ;; User do - sideffecting stuff.
+    ((_ orig name l a v c r f ff ((cur-uw ...) . uw-rest) ((:do expr ...) clauses ...) . body)
+     (cl orig name l a v c r f ff ((cur-uw ... (:do expr ...)) . uw-rest) (clauses ...) . body))
     
     ;; Explicit subloop. Shorthand for (:when #t)
     ((_ orig name l a v c r f ff ((cur-uw ...) . uw-rest) (:subloop clauses ...) . body)
@@ -164,9 +168,9 @@
 
     ;; ERROR HANDLING?
     ((_ orig name l a v c r f ff user (clause . rest) . body)
-     (syntax-error "Invalid clause in loop" clause orig))
+     (syntax-error "Invalid clause in loop" clause orig))))
+
 
-    ))
 
 
 ;; HOLY CODE-DUPLICATION-BATMAN!
@@ -184,15 +188,15 @@
         checks
         ((refs ...))
         (finals ...)
-        ff ul uw ((cur-ub ...) . ub-rest) uf clauses . body)
+        ff ((cur-ub ...) . ub-rest) clauses . body)
      (cl orig name
          ((lets ... new-lets ...))
-         ((accs ... (accvar accinit accupdate) ...))
+         ((accs ... (accvar accinit accvar) ...))
          vars
          checks
          ((refs ... new-refs ...))
          (finals ... new-finals ...)
-         ff ul uw ((cur-ub ... new-checks ...) . ub-rest) uf clauses . body))
+         ff ((cur-ub ... (:break  new-checks) ... (:bind (accvar accupdate) ...)) . ub-rest) clauses . body))
     ;; We have ONE subloop!
     ((_ (new-lets ...) ((accvar accinit accupdate) ...) (new-checks ...) (new-refs ...) (new-finals ...)
         orig name
@@ -202,15 +206,15 @@
         checks
         ((refs ...) . refs-rest)
         (finals ...)
-        ff ul uw ((cur-ub ...) . ub-rest) uf clauses . body)
+        ff ((cur-ub ...) . ub-rest) clauses . body)
      (cl orig name
          (lets ... (outermost-lets ... new-lets ...))
-         ((accs ... (accvar accvar accupdate) ...) ((oldacc oldinit oldupdate) ... (accvar accinit accvar) ...))
+         ((accs ... (accvar accvar accvar) ...) ((oldacc oldinit oldupdate) ... (accvar accinit accvar) ...))
          vars
          checks
          ((refs ... new-refs ...) . refs-rest)
          (finals ... new-finals ...)
-         ff ul uw ((cur-ub ... new-checks ...) . ub-rest) uf clauses . body))
+         ff ((cur-ub ... (:break  new-checks) ... (:bind (accvar accupdate) ...)) . ub-rest) clauses . body))
     ;; We have several subloops!
     ((_ (new-lets ...) ((accvar accinit accupdate) ...) (new-checks ...) (new-refs ...) (new-finals ...)
         orig name
@@ -220,16 +224,16 @@
         checks
         ((refs ...) . refs-rest)
         (finals ...)
-        ff ul uw ((cur-ub ...) . ub-rest) uf clauses . body)
+        ff ((cur-ub ...) . ub-rest) clauses . body)
      (cl orig name
          (lets ... (outermost-lets ... new-lets ...))
-         ((accs ... (accvar accvar accupdate) ...) ((oldacc oldinit oldupdate) ... (accvar accvar accvar) ...) ...
+         ((accs ... (accvar accvar accvar) ...) ((oldacc oldinit oldupdate) ... (accvar accvar accvar) ...) ...
           ((oldestacc oldestinit oldestupdate) ... (accvar accinit accvar) ...))
          vars
          checks
          ((refs ... new-refs ...) . refs-rest)
          (finals ... new-finals ...)
-         ff ul uw ((cur-ub ... new-checks ...) . ub-rest) uf clauses . body))))
+         ff ((cur-ub ... (:break  new-checks) ... (:bind (accvar accupdate) ...)) . ub-rest) clauses . body))))
 
 ;; Integrating for clauses is not as involved, since they only want to be introduced into the current
 ;; loop. Any propagation of for finalizers (ff) is done by push-new-subloop
@@ -259,15 +263,13 @@
 
 
 (define-syntax user
-  (syntax-rules (:when :bind :break :final :nop)
+  (syntax-rules (:when :bind :break :do :nop)
     ((_ final-expr next outer () body ...)
      (begin body ...))
-
     ((_ f n o (:nop . rest) . body)
      (user f n o rest . body))
-
     ((_ f n o ((:bind pairs ...) . rest) . body)
-     (let (pairs ...)
+     (ref-let (pairs ...)
        (user f n o rest . body)))
     ((_ f n o ((:when test) . rest) . body)
      (cond
@@ -277,8 +279,12 @@
      (cond
       (expr final-expr ...)
       (else (user (final-expr ...) n o rest . body))))
+    ((_ f n o ((:do expr ...) . rest) . body)
+     (begin
+       expr ...
+       (user f n o rest . body)))))
+
 
-    ))
 
 ;; If there are no subloops, we emit to the simple case
 (define-syntax emit
@@ -413,6 +419,7 @@
                       (user (ff-cur ... ff-above ... final)
                             (intermediate-loop accstep ... step ...)
                             #f
+                            (us ...)
                             (emit-many/rest orig
                                             name
                                             (intermediate-loop accstep ... step ...)
@@ -427,14 +434,14 @@
                                             . body)))))))))
 
 (define-syntax forify
-(syntax-rules (%acc)
+  (syntax-rules (%acc)
     ((_ orig name () ((%acc . acc-rest) . argsrest) . body)
      (forify* orig name () ((:for ensure-once (up-from 0 1)) (%acc . acc-rest) . argsrest) . body))
     ((_ . rest)
      (forify* . rest))))
 
 (define-syntax forify*
-  (syntax-rules (:for :acc :when :unless :break :final :subloop :let :let* %acc)
+  (syntax-rules (:for :acc :when :unless :break :final :subloop :bind :do %acc)
     ((_ o n done-clauses () . body)
      (%loop o n done-clauses . body))
     ((_      o n (s ...) ((:for c-rest ...) clauses ...) . body)
@@ -447,12 +454,12 @@
      (forify* o n (s ... (:break expr)) (clauses ...) . body))
     ((_      o n (s ...) ((:final expr) clauses ...) . body)
      (forify* o n (s ... (:final expr)) (clauses ...) . body))
+    ((_      o n (s ...) ((:do expr ...) clauses ...) . body)
+     (forify* o n (s ... (:do expr ...)) (clauses ...) . body))
     ((_      o n (s ...) (:subloop clauses ...) . body)
      (forify* o n (s ... :subloop) (clauses ...) . body))
-    ((_      o n (s ...) ((:let id id* ... expr) clauses ...) . body)
+    ((_      o n (s ...) ((:bind pairs ...) clauses ...) . body)
-     (forify* o n (s ... (:let id id* ... expr)) (clauses ...) . body))
+     (forify* o n (s ... (:bind pairs ...)) (clauses ...) . body))
-    ((_      o n (s ...) ((:let* id id* ... expr) clauses ...) . body)
-     (forify* o n (s ... (:let* id id* ... expr)) (clauses ...) . body))
     ((_      o n (s ...) ((%acc c-rest ...) clauses ...) . body)
      (forify* o n (s ... (:acc c-rest ...)) (clauses ...) . body))
     ((_      o n (s ...) ((:acc c-rest ...) clauses ...) . body)

goof.scm

@@ -47,7 +47,7 @@
              loop/or
              loop/list/parallel
 
-             :when :unless :break :final :bind :subloop :for :acc
+             :when :unless :break :final :bind :subloop :do :for :acc
              :length :fill
              :to :by
 

goof/iterators.scm

@@ -508,6 +508,21 @@
            ((var var))
            . rest))))
 
+
+;; this is an internal "accumulator". It is used for final tests
+;; :final in goof differs from in racket. It is lexical, meaning it
+;; is tested where it is placed in the clauses, and any subloop is
+;; executed completely.
+(define-syntax final
+  (syntax-rules (:acc)
+    ((_ :acc ((var) (test)) n . rest)
+     (n ()
+        ((final #f test))
+        (final)
+        ()
+        ()
+        . rest))))
+
 ;;; Here starts generator clauses.
 
 (define (generator->list gen)

tests.scm

@@ -37,8 +37,8 @@
     a)
   '(0 1 2 3))
 
-(test-equal ":let and :let*"
+(test-equal ":bind"
-  (loop/list ((a (up-from 0 5)) (:let b (+ a 1)) (:let* c (+ b 1)))
+  (loop/list ((a (up-from 0 5)) (:bind (b (+ a 1))) (:bind (c (+ b 1))))
     c)
   '(2 3 4 5 6))