For practical purposes it would be wise to use existing libraries, but since the question is about how to implemented lazy lists, we will do it from scratch.
Closures
Lazy iteration is a matter of producing an object that can generate the new value of a lazy sequence each time it is asked to do so.
A simple approach for this is to return a closure, i.e. a function that closes over variables, which produces values while updating its state by side-effect.
If you evaluate:
(let ((a 0))
(lambda () (incf a)))
You obtain a function object that has a local state, namely here the variable named a.
This is a lexical binding to a location that is exclusive to this function, if you evaluate a second time the same expression, you'll obtain a different anonymous function that has its own local state.
When you call the closure, the value stored in a in incremented and its value is returned.
Let's bind this closure to a variable named counter, call it multiple times and store the successive results in a list:
(let ((counter (let ((a 0))
(lambda () (incf a)))))
(list (funcall counter)
(funcall counter)
(funcall counter)
(funcall counter)))
The resulting list is:
(1 2 3 4)
Simple iterator
In your case, you want to have an iterator that starts counting from 5 when writing:
(inf 5)
This can implemented as follows:
(defun inf (n)
(lambda ()
(shiftf n (1+ n))))
Here is there is no need to add a let, the lexical binding of an argument to n is done when calling the function.
We assign n to a different value within the body over time.
More precisely, SHIFTF assigns n to (1+ n), but returns the previous value of n.
For example:
(let ((it (inf 5)))
(list (funcall it)
(funcall it)
(funcall it)
(funcall it)))
Which gives:
(5 6 7 8)
Generic iterator
The standard dolist expects a proper list as an input, there is no way you can put another kind of data and expect it to work (or maybe in an implementation-specific way).
We need a similar macro to iterate over all the values in an arbitrary iterator.
We also need to specify when iteration stops.
There are multiple possibilities here, let's define a basic iteration protocol as follows:
- we can call
make-iterator on any object, along with arbitrary arguments, to obtain an iterator
- we can call
next on an iterator to obtain the next value.
- More precisely, if there is a value,
next returns the value and T as a secondary value; otherwise, next returns NIL.
Let's define two generic functions:
(defgeneric make-iterator (object &key)
(:documentation "create an iterator for OBJECT and arguments ARGS"))
(defgeneric next (iterator)
(:documentation "returns the next value and T as a secondary value, or NIL"))
Using generic functions allows the user to define custom iterators, as long as they respect the specified behaviour above.
Instead of using dolist, which only works with eager sequences, we define our own macro: for.
It hides calls to make-iterator and next from the user.
In other words, for takes an object and iterates over it.
We can skip iteration with (return v) since for is implemented with loop.
(defmacro for ((value object &rest args) &body body)
(let ((it (gensym)) (exists (gensym)))
`(let ((,it (make-iterator ,object ,@args)))
(loop
(multiple-value-bind (,value ,exists) (next ,it)
(unless ,exists
(return))
,@body)))))
We assume any function object can act as an iterator, so we specialize next for values f of class function, so that the function f gets called:
(defmethod next ((f function))
"A closure is an interator"
(funcall f))
Also, we can also specialize make-iterator to make closures their own iterators (I see no other good default behaviour to provide for closures):
(defmethod make-iterator ((function function) &key)
function)
Vector iterator
For example, we can built an iterator for vectors as follows. We specialize make-iterator for values (here named vec) of class vector.
The returned iterator is a closure, so we will be able to call next on it.
The method accepts a :start argument defaulting to zero:
(defmethod make-iterator ((vec vector) &key (start 0))
"Vector iterator"
(let ((index start))
(lambda ()
(when (array-in-bounds-p vec index)
(values (aref vec (shiftf index (1+ index))) t)))))
You can now write:
(for (v "abcdefg" :start 2)
(print v))
And this prints the following characters:
#c
#d
#e
#f
#g
List iterator
Likewise, we can build a list iterator.
Here to demonstrate other kind of iterators, let's have a custom cursor type.
(defstruct list-cursor head)
The cursor is an object which keeps a reference to the current cons-cell in the list being visited, or NIL.
(defmethod make-iterator ((list list) &key)
"List iterator"
(make-list-cursor :head list))
And we define next as follows, specializeing on list-cursor:
(defmethod next ((cursor list-cursor))
(when (list-cursor-head cursor)
(values (pop (list-cursor-head cursor)) t)))
Ranges
Common Lisp also allows methods to be specialized with EQL specializers, which means the object we give to for might be a specific keyword, for example :range.
(defmethod make-iterator ((_ (eql :range)) &key (from 0) (to :infinity) (by 1))
(check-type from number)
(check-type to (or number (eql :infinity)))
(check-type by number)
(let ((counter from))
(case to
(:infinity
(lambda () (values (incf counter by) t)))
(t
(lambda ()
(when (< counter to)
(values (incf counter by) T)))))))
A possible call for make-iterator would be:
(make-iterator :range :from 0 :to 10 :by 2)
This also returns a closure.
Here, for example, you would iterate over a range as follows:
(for (v :range :from 0 :to 10 :by 2)
(print v))
The above expands as:
(let ((#:g1463 (make-iterator :range :from 0 :to 10 :by 2)))
(loop
(multiple-value-bind (v #:g1464)
(next #:g1463)
(unless #:g1464 (return))
(print v))))
Finally, if we add small modification to inf (adding secondary value):
(defun inf (n)
(lambda ()
(values (shiftf n (1+ n)) T)))
We can write:
(for (v (inf 5))
(print v)
(when (= v 7)
(return)))
Which prints:
5
6
7
