Previously we were getting a pointer into the value stack for the parts
list. However, that pointer would not necessarily be a valid pointer later
any more, as we're also manipulating the value stack.
This introduces the higher level API apfl_set_list_member_by_index to
modify a list and also adds an option for our test suite to run the tests
with valgrind --tool=memcheck. This should catch such errors in the future.
The iterative runner used functions that could throw errors on OOM before
protecting itself using apfl_call_protected. An error at that point would
have resulted in calling the panic handler as a last resort.
This now introduces the apfl_do_protected function which allows running C
code protected without needing to wrap it in a cfunc value, thus removing
the need for the functions that could throw in the iterative runner.
It's now possible to assign to a key of a dictionary and even to a nested
key path.
This patch changes the way matchers work a bit:
First, a function call stack frame now has a stack of matchers that are
manipulateable instead of a single matcher.
Second, the matcher is now in charge of setting the matched values to the
variables (previously the caller of the matcher needed to extract the
matched values and assign them itself). This change simplifies code
generation, especially for chained assignments and dictionary key paths.
This removes the last usage of APFL_ERR_NOT_IMPLEMENTED :)
A function can now have multiple subfunctions with their own parameter
list. These parameters are now no longer constrained to variables and
blanks only, but can also be consts and list destructurings (predicates
are also already compiled but don't get evaluated yet). The first
subfunction that matches the argument list gets evaluated.
This allows the destructuring of lists into individual values.
We can have arbitrarily nested lists, can check for constant values and can
have up to one '~'-prefixed variable per list, that will capture the
remaining elements of the list.
It is implemented as a second set of bytecode instructions, which define a
matcher. These matchers should also enable us to implement the same pattern
matching capabiities for function parameters.
Not all matching features are implemented yet, predicate matching and
matching into a dictionary key is not implemented yet.
We can now define and call functions. Lexical closure scopes are also
working :).
It's limited to simple functions or complex functions with a single
argument list of only variable names for now.
The new roots callback mechanism makes it possible to traverse the gc
objects on the stack without needing the stack to be an gc object.
This also uncovered a nasty bug in apfl_stack_pop where we passed in a
wrong mem pointer into apfl_resizable_splice. We should probably find a way
to make the apfl_resizable_* functions a bit safer, the need for casting
stuff to void** easily hides errors.
This let's us get rid of that awkward hashmap in the GC that was used as a
set, makes determining the roots more flexible and now gc_init can't even
fail any more, as there are no allocations happening here any more :)
This way we can see the parse errors again in evaluation mode
Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the
top of the stack (the formatted error) to stderr but don't nicely handle
if there is nothing on the stack (apfl_debug_print_val will print a rather
cryptic "stack index invalid" error). Also the whole dance we need to do to
put the formatted error onto the stack feels rather awkward (there should
probably a function for this) and we also should probably try to push an
error description on the stack in case this moving-string-to-stack business
fails.
Now "only" all other errors need to be put on the stack as a string :)
- peek functions will now return pointers directly
- cursor access methods can now return an error, if the cursor is already
at the end
Also rewrote some cursor loops to use the HASHMAP_EACH macro.
Instead of the previous refcount base garbage collection, we're now using
a basic tri-color mark&sweep collector. This is done to support cyclical
value relationships in the future (functions can form cycles, all values
implemented up to this point can not).
The collector maintains a set of roots and a set of objects (grouped into
blocks). The GC enabled objects are no longer allocated manually, but will
be allocated by the GC. The GC also wraps an allocator, this way the GC
knows, if we ran out of memory and will try to get out of this situation by
performing a full collection cycle.
The tri-color abstraction was chosen for two reasons:
- We don't have to maintain a list of objects that need to be marked, we
can simply grab the next grey one.
- It should allow us to later implement incremental collection (right now
we only do a stop-the-world collection).
This also switches to a bytecode based evaluation of the code: We no longer
directly evaluate the AST, but first compile it into a series of
instructions, that are evaluated in a separate step. This was done in
preparation for inplementing functions: We only need to turn a function
body into instructions instead of evaluating the node again with each call
of the function. Also, since an instruction list is implemented as a GC
object, this then removes manual memory management of the function body and
it's child nodes. Since the GC and the bytecode go hand in hand, this was
done in one (giant) commit.
As a downside, we've now lost the ability do do list matching on
assignments. I've already started to work on implementing this in the new
architecture, but left it out of this commit, as it's already quite a large
commit :)
