iterate

Can't iterate over nested Hashtables

Found by Bob Hutchison. It's related to 2.1, "Macrolets in the iterate form":

 (iter (for (k1 v1) in-hashtable ht1)
       (iter (for (k2 v2) in-hashtable ht2)
             (format t "~A ~A~%" k1 k2)))

This fails with the typical macrolet error. Why? Hah.

It's the fault of the new IN-HASHTABLE code that makes use of WITH-HASHTABLE-ITERATOR. The iterator is specified to be defined by a symbol-macrolet. I think we'll have to make a rudimentary macrolet walker. Argh. Hmpf.

This affects nested package walking, as well. A solution to one will probably be a solution to both.

PREVIOUS and INITIALLY

 (iter (for el in '(1 2 3 4))
       (for p-el previous el)
       (for pp-el previous p-el initially 0)
       (collect pp-el))

gives (0 NIL 1 2); should be (0 0 1 2) according to the manual. I suspect it's a doc bug. Found by Carlos Ungil.

(iter (for el in '(1 2 3 4))
      (for pp-el previous el back 2 initially 0)
      (collect pp-el))

gives the right result.

So does:

(iter (for el in '(1 2 3 4))
      (for p-el previous el initially 0)
      (for pp-el previous p-el initially 0)
      (collect pp-el))

Without an INITIALLY 0, p-el gets set to NIL the first time through the loop. So, pp-el's value is NIL the second time through the loop. The for...previous documentation talks about the "first" and "second" value. But it doesn't detail if the initial value counts as a "first" value.

Wishlist

Macrolets in the iterate form

(iterate (for i from 0 to 20)
         (macrolet ((collect-if-divisible (variable divisor result-var)
		      `(when (zerop (mod ,variable ,divisor))
			 (collect ,variable into ,result-var))))
           (collect-if-divisible i 3 result-3)
	   (collect-if-divisible i 5 result-5))
    	 (finally (return (values result-3 result-5))))

would be nice to have. Currently has to be rewritten:

(macrolet ((collect-if-divisible (variable divisor result-var)
		   `(when (zerop (mod ,variable ,divisor))
			 (collect ,variable into ,result-var))))
  (iterate (for i from 0 to 20)
           (collect-if-divisible i 3 result-3)
	   (collect-if-divisible i 5 result-5)
	   (finally (return (values result-3 result-5)))))

of course, this does not work in all cases.

Already done (DONE)

synonyms

Another bug found by Carlos Ungil. I have now imported most examples from the manual into the test suite (-:

(iter (generating (key . item) in '((a . 1) (b . 2) (c .3)))
      (collect (next key))
      (collect (next item)))
TYPE ERROR : The value ITEM is not of type LIST.
in the manual: (a 2 c)

iter:symbol-synonym was broken.

spurious warnings when using QUOTE, GO or the FOR...NEXT clause

(let ((y '(a b c)))
  (iter (for x next (if y (pop y) (terminate)))
        (collect x)))

gave:

; Warning: Unknown special form GO. The lisp environment claims
; that GO is a special operator, but ITERATE doesn't know how to
; handle it.
; While executing: ITERATE::WALK-SPECIAL-FORM
(A B C)

that is, the right result plus one or more warnings about unknown special operators. This was a bad check that I had introduced to catch other bugs in implementations that kept some macros as special operators. The check is fixed in versions of iterate from 1.0.5 upwards.

Didn't build on Corman Lisp