laria/apfl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
apfl/src/eval.c

1278 lines
36 KiB
C
Raw Normal View History

Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
#include <assert.h>
#include "apfl.h"
Introduce allocator abstraction We now no longer call malloc/free/... directly, but use an allocator object that is passed around. This was mainly done as a preparation for a garbage collector: The collector will need to know, how much memory we're using, introducing the collector abstraction will allow the GC to hook into the memory allocation and observe the memory usage. This has other potential applications: - We could now be embedded into applications that can't use the libc allocator. - There could be an allocator that limits the total amount of used memory, e.g. for sandboxing purposes. - In our tests we could use this to simulate out of memory conditions (implement an allocator that fails at the n-th allocation, increase n by one and restart the test until there are no more faked OOM conditions). The function signature of the allocator is basically exactly the same as the one Lua uses.
2022-02-08 21:53:13 +00:00
#include "alloc.h"
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
#include "bytecode.h"
#include "compile.h"
#include "context.h"
apfl_debug_print_val: Use format writer abstraction
2022-06-05 20:06:33 +00:00
#include "format.h"
Implement simple variable support We can now assign to variables and retrieve their values. Not all assignables are implemented yet, but still pretty cool :).
2022-01-06 21:53:26 +00:00
#include "hashmap.h"
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
#include "matcher.h"
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
#include "resizable.h"
Ditch that weird internal.h header
2022-04-22 21:17:28 +00:00
#include "strings.h"
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
#include "value.h"
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
static void evaluate(apfl_ctx ctx, struct func_call_stack_entry *cse);
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
static void evaluate_matcher(apfl_ctx ctx, struct matcher_call_stack_entry *cse);
static void matcher_init_matching(apfl_ctx ctx, struct matcher *matcher);
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
static void
Add apfl_stackidx typedef This makes it clear that a value of that type is meant as an index into the stack.
2022-02-10 21:39:39 +00:00
stack_must_drop(apfl_ctx ctx, apfl_stackidx index)
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
{
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
assert(apfl_stack_drop(ctx, index));
Partially implement pattern matching assignments We're still missing predicates (need to be able to call functions for that one) and assignments into dictionaries. But we now can deconstruct a list and check against constants. So things like this work now: [1 foo ~bar [a b]] = [1 "Hello" 2 3 4 [5 6]] # foo is: "Hello" # bar is: [2 3 4] # a is: 5 # b is: 6 Pretty cool :)
2022-01-14 22:16:19 +00:00
}
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
#define ABSTRACT_GET_ARGUMENT(i, ilist, arg) \
if (*i >= ilist->len) { \
return false; \
} \
\
*arg = ilist->instructions[(*i)++]; \
return true;
#define ABSTRACT_MUST_GET_ARG(get, ctx, i, ilist, arg) \
if (!get(i, ilist, arg)) { \
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode); \
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static bool
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
get_argument(size_t *i, struct instruction_list *ilist, union instruction_or_arg *arg)
Partially implement pattern matching assignments We're still missing predicates (need to be able to call functions for that one) and assignments into dictionaries. But we now can deconstruct a list and check against constants. So things like this work now: [1 foo ~bar [a b]] = [1 "Hello" 2 3 4 [5 6]] # foo is: "Hello" # bar is: [2 3 4] # a is: 5 # b is: 6 Pretty cool :)
2022-01-14 22:16:19 +00:00
{
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
ABSTRACT_GET_ARGUMENT(i, ilist, arg)
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
}
static void
must_get_argument(apfl_ctx ctx, size_t *i, struct instruction_list *ilist, union instruction_or_arg *arg)
{
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
ABSTRACT_MUST_GET_ARG(get_argument, ctx, i, ilist, arg)
}
static bool
get_matcher_argument(size_t *i, struct matcher_instruction_list *milist, union matcher_instruction_or_arg *arg)
{
ABSTRACT_GET_ARGUMENT(i, milist, arg)
}
static void
must_get_matcher_argument(apfl_ctx ctx, size_t *i, struct matcher_instruction_list *milist, union matcher_instruction_or_arg *arg)
{
ABSTRACT_MUST_GET_ARG(get_matcher_argument, ctx, i, milist, arg)
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
static struct func_call_stack_entry *
get_current_func_cse(apfl_ctx ctx)
{
struct call_stack_entry *cse = apfl_call_stack_cur_entry(ctx);
if (cse == NULL) {
return NULL;
}
return cse->type == CSE_FUNCTION
? &cse->func
: NULL;
}
enum scope_type {
SCOPE_LOCAL,
SCOPE_CLOSUE,
SCOPE_GLOBAL,
};
static struct scope *
get_scope(apfl_ctx ctx, enum scope_type type)
{
struct func_call_stack_entry *func_cse;
switch (type) {
case SCOPE_LOCAL:
if ((func_cse = get_current_func_cse(ctx)) != NULL) {
return func_cse->scope;
}
return NULL;
case SCOPE_CLOSUE:
if ((func_cse = get_current_func_cse(ctx)) != NULL) {
return func_cse->closure_scope;
}
return NULL;
case SCOPE_GLOBAL:
return ctx->globals;
}
assert(false);
return NULL;
}
static struct scope *
get_or_create_local_scope(apfl_ctx ctx)
{
struct func_call_stack_entry *func_cse = get_current_func_cse(ctx);
assert(func_cse != NULL);
if (func_cse->scope != NULL) {
return func_cse->scope;
}
if ((func_cse->scope = apfl_scope_new(&ctx->gc)) == NULL) {
apfl_raise_alloc_error(ctx);
}
return func_cse->scope;
}
static bool
try_variable_get_for_scope_type(apfl_ctx ctx, struct apfl_string *name, enum scope_type type)
{
struct apfl_value value;
struct scope *scope;
if ((scope = get_scope(ctx, type)) != NULL) {
if (apfl_scope_get(scope, name, &value)) {
apfl_stack_must_push(ctx, value);
return true;
}
}
return false;
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
variable_get(apfl_ctx ctx, struct apfl_string *name)
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
if (try_variable_get_for_scope_type(ctx, name, SCOPE_LOCAL)) {
return;
}
if (try_variable_get_for_scope_type(ctx, name, SCOPE_CLOSUE)) {
return;
}
if (try_variable_get_for_scope_type(ctx, name, SCOPE_GLOBAL)) {
return;
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.variable_doesnt_exist);
}
static bool
try_variable_update_existing_for_scope_type(
apfl_ctx ctx,
struct apfl_string *name,
struct apfl_value value,
enum scope_type type
) {
struct scope *scope;
if ((scope = get_scope(ctx, type)) != NULL) {
if (apfl_scope_update_existing(scope, name, value)) {
return true;
}
}
return false;
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
variable_set_value(apfl_ctx ctx, struct apfl_string *name, bool local, struct apfl_value value)
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
bool was_set = false;
if (!local) {
was_set = try_variable_update_existing_for_scope_type(ctx, name, value, SCOPE_LOCAL)
|| try_variable_update_existing_for_scope_type(ctx, name, value, SCOPE_CLOSUE);
values: Split lists into read-only lists and editable lists This is analogous to dictionaries and ensures that no circular references can be created when using the exported API in apfl.h. This also changes apfl_value_copy into apfl_value_incref to better reflect what it does and to reflect that it is no longer an operation that can fail.
2022-01-04 22:11:38 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
if (!was_set) {
struct scope *scope = get_or_create_local_scope(ctx);
assert(scope != NULL /*get_or_create_local_scope should never return NULL*/);
if (!apfl_scope_set(&ctx->gc, scope, name, value)) {
apfl_raise_alloc_error(ctx);
}
Implement evaluating dictionary literals We can now have dictionaries with keys and values of arbitrary types. Very cool! :)
2022-01-04 20:51:44 +00:00
}
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
}
static void
variable_set(apfl_ctx ctx, struct apfl_string *name, bool keep_on_stack, bool local)
{
struct apfl_value value = apfl_stack_must_get(ctx, -1);
variable_set_value(ctx, name, local, value);
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
if (keep_on_stack) {
// If the value should be kept on the stack, the value is now in two
// places. We need to set the COW flag to prevent mutations of one copy
// affecting the other one.
value = apfl_value_set_cow_flag(value);
} else {
stack_must_drop(ctx, -1);
Implement evaluating dictionary literals We can now have dictionaries with keys and values of arbitrary types. Very cool! :)
2022-01-04 20:51:44 +00:00
}
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
static bool
variable_exists_by_scope_type(apfl_ctx ctx, struct apfl_string *name, enum scope_type type)
{
struct scope *scope;
if ((scope = get_scope(ctx, type)) != NULL) {
if (apfl_scope_has(scope, name)) {
return true;
}
}
return false;
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
variable_new(apfl_ctx ctx, struct apfl_string *name, bool local)
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
if (!local) {
if (variable_exists_by_scope_type(ctx, name, SCOPE_LOCAL)) {
return;
}
if (variable_exists_by_scope_type(ctx, name, SCOPE_CLOSUE)) {
return;
}
}
struct scope *scope = get_or_create_local_scope(ctx);
if (!apfl_scope_create_var(&ctx->gc, scope, name)) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_alloc_error(ctx);
Implement evaluating dictionary literals We can now have dictionaries with keys and values of arbitrary types. Very cool! :)
2022-01-04 20:51:44 +00:00
}
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
func_inner(apfl_ctx ctx, struct instruction_list *body)
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
struct scope *scope = apfl_closure_scope_for_func(ctx);
if (scope == NULL) {
apfl_raise_alloc_error(ctx);
}
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
if (!apfl_gc_tmproot_add(&ctx->gc, GC_OBJECT_FROM(scope, GC_TYPE_SCOPE))) {
apfl_raise_alloc_error(ctx);
}
struct apfl_value *func_value = apfl_stack_push_placeholder(ctx);
if (func_value == NULL) {
apfl_raise_alloc_error(ctx);
}
if ((func_value->func = apfl_func_new(&ctx->gc, body, scope)) == NULL) {
stack_must_drop(ctx, -1);
apfl_raise_alloc_error(ctx);
}
func_value->type = VALUE_FUNC;
}
static void
func(apfl_ctx ctx, struct instruction_list *body)
{
size_t tmproots = apfl_gc_tmproots_begin(&ctx->gc);
func_inner(ctx, body);
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
}
static void
call_stack_push(apfl_ctx ctx, struct call_stack_entry cse)
{
if (!apfl_resizable_append(
ctx->gc.allocator,
sizeof(struct call_stack_entry),
(void**)&ctx->call_stack.items,
&ctx->call_stack.len,
&ctx->call_stack.cap,
&cse,
1
)) {
apfl_call_stack_entry_deinit(ctx->gc.allocator, &cse);
apfl_raise_alloc_error(ctx);
}
}
static void
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
call_stack_drop(apfl_ctx ctx)
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
{
assert(ctx->call_stack.len > 0);
apfl_call_stack_entry_deinit(ctx->gc.allocator, apfl_call_stack_cur_entry(ctx));
assert(
// We're shrinking the memory here, should not fail
apfl_resizable_resize(
ctx->gc.allocator,
sizeof(struct call_stack_entry),
(void **)&ctx->call_stack.items,
&ctx->call_stack.len,
&ctx->call_stack.cap,
ctx->call_stack.len - 1
)
);
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
}
static void
return_from_function_inner(apfl_ctx ctx)
{
struct apfl_value value;
if (!apfl_stack_pop(ctx, &value, -1)) {
// No return value on the stack. Return nil instead
value = (struct apfl_value) { .type = VALUE_NIL };
}
call_stack_drop(ctx);
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_stack_must_push(ctx, value);
}
static void
return_from_function(apfl_ctx ctx)
{
size_t tmproots = apfl_gc_tmproots_begin(&ctx->gc);
return_from_function_inner(ctx);
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
}
static void
prepare_call(apfl_ctx ctx, size_t tmproots, struct apfl_value args, struct call_stack_entry cse)
{
call_stack_push(ctx, cse);
// Note: This pushes args on the stack of the newly created call stack
apfl_stack_must_push(ctx, args);
// Both the function and the args are now rooted again, we can undo the tmproots early.
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
}
// Keep evaluate instructions until we've returned from the current call stack.
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
// Must not be called with a CSE_CFUNCTION on top of the call stack.
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
static void
evaluate_until_call_stack_return(apfl_ctx ctx)
{
struct call_stack *call_stack = &ctx->call_stack;
size_t depth_started = call_stack->len;
assert(depth_started > 0);
while (call_stack->len >= depth_started) {
struct call_stack_entry *cse = apfl_call_stack_cur_entry(ctx);
assert(cse != NULL);
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
switch (cse->type) {
case CSE_CFUNCTION:
assert(false);
break;
case CSE_FUNCTION:
evaluate(ctx, &cse->func);
break;
case CSE_MATCHER:
evaluate_matcher(ctx, &cse->matcher);
break;
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
}
}
static void
must_tmproot_add_value(apfl_ctx ctx, struct apfl_value value)
{
struct gc_object *obj = apfl_value_get_gc_object(value);
if (obj != NULL) {
if (!apfl_gc_tmproot_add(&ctx->gc, obj)) {
apfl_raise_alloc_error(ctx);
}
}
}
static void
call_inner(apfl_ctx ctx, size_t tmproots, apfl_stackidx func_index, apfl_stackidx args_index, bool call_from_apfl)
{
struct apfl_value func = apfl_stack_must_get(ctx, func_index);
must_tmproot_add_value(ctx, func);
struct apfl_value args = apfl_stack_must_get(ctx, args_index);
must_tmproot_add_value(ctx, args);
assert(apfl_stack_drop_multi(ctx, 2, (apfl_stackidx[]){func_index, args_index}));
if (!VALUE_IS_A(func, APFL_VALUE_FUNC)) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.not_a_function);
}
if (!VALUE_IS_A(args, APFL_VALUE_LIST)) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.not_a_list);
}
switch (func.type) {
case VALUE_FUNC:
prepare_call(ctx, tmproots, args, (struct call_stack_entry) {
.type = CSE_FUNCTION,
.stack = apfl_stack_new(),
.func = {
.pc = 0,
.instructions = func.func->body,
.scope = NULL,
.closure_scope = func.func->scope,
.execution_line = func.func->body->line,
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
.returning_from_matcher = false,
.matcher = NULL,
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
},
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
});
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
if (call_from_apfl) {
// In this case we're already coming from evaluate_until_call_stack_return,
// which will pick up the new stack entry. This way we can avoid doing the recursion in C.
return;
} else {
evaluate_until_call_stack_return(ctx);
}
break;
case VALUE_CFUNC:
prepare_call(ctx, tmproots, args, (struct call_stack_entry) {
.type = CSE_CFUNCTION,
.stack = apfl_stack_new(),
.cfunc = {
.func = func.cfunc,
},
});
func.cfunc->func(ctx);
return_from_function(ctx);
break;
default:
assert(false); // Otherwise the VALUE_IS_A() check for APFL_VALUE_FUNC would have failed
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
}
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
static void
call(apfl_ctx ctx, apfl_stackidx func_index, apfl_stackidx args_index, bool call_from_apfl)
{
size_t tmproots = apfl_gc_tmproots_begin(&ctx->gc);
call_inner(ctx, tmproots, func_index, args_index, call_from_apfl);
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
}
void
apfl_call(apfl_ctx ctx, apfl_stackidx func_index, apfl_stackidx args_index)
{
call(ctx, func_index, args_index, false);
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
}
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
static void
matcher_load(apfl_ctx ctx, struct func_call_stack_entry *cse, struct matcher_instruction_list *milist)
{
assert(cse != NULL);
if ((cse->matcher = apfl_matcher_new(&ctx->gc, milist)) == NULL) {
apfl_raise_alloc_error(ctx);
}
}
static void
matcher_set_val(apfl_ctx ctx, struct matcher *matcher, size_t index)
{
if (matcher == NULL) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode);
}
if (index >= matcher->instructions->value_count) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode);
}
matcher->values[index] = apfl_stack_must_pop(ctx, -1);
}
static void
variable_set_from_matcher_inner(
apfl_ctx ctx,
struct matcher *matcher,
struct apfl_string *varname,
size_t index,
bool local
) {
if (matcher == NULL) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode);
}
if (index >= matcher->instructions->capture_count) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode);
}
struct apfl_value value = apfl_value_move(&matcher->captures[index]);
must_tmproot_add_value(ctx, value);
variable_set_value(ctx, varname, local, value);
}
static void
variable_set_from_matcher(
apfl_ctx ctx,
struct matcher *matcher,
struct apfl_string *varname,
size_t index,
bool local
) {
size_t tmproots = apfl_gc_tmproots_begin(&ctx->gc);
variable_set_from_matcher_inner(ctx, matcher, varname, index, local);
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
evaluate(apfl_ctx ctx, struct func_call_stack_entry *cse)
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
{
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
if (cse->returning_from_matcher) {
assert(cse->matcher != NULL);
if (!cse->matcher->result) {
cse->matcher = NULL;
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.value_doesnt_match);
}
cse->returning_from_matcher = false;
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
union instruction_or_arg arg;
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
union instruction_or_arg arg2;
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
size_t *pc = &cse->pc;
struct instruction_list *ilist = cse->instructions;
while (*pc < cse->instructions->len) {
switch (ilist->instructions[(*pc)++].instruction) {
case INSN_NIL:
apfl_push_nil(ctx);
goto continue_loop;
case INSN_TRUE:
apfl_push_bool(ctx, true);
goto continue_loop;
case INSN_FALSE:
apfl_push_bool(ctx, false);
goto continue_loop;
case INSN_NUMBER:
must_get_argument(ctx, pc, ilist, &arg);
apfl_push_number(ctx, arg.number);
goto continue_loop;
case INSN_STRING:
must_get_argument(ctx, pc, ilist, &arg);
apfl_stack_must_push(ctx, (struct apfl_value) {
.type = VALUE_STRING,
.string = arg.string,
});
goto continue_loop;
case INSN_LIST:
must_get_argument(ctx, pc, ilist, &arg);
apfl_list_create(ctx, arg.count);
goto continue_loop;
case INSN_LIST_APPEND:
apfl_list_append(ctx, -2, -1);
goto continue_loop;
case INSN_LIST_EXPAND_INTO:
apfl_list_append_list(ctx, -2, -1);
goto continue_loop;
case INSN_DICT:
apfl_dict_create(ctx);
goto continue_loop;
case INSN_DICT_APPEND_KVPAIR:
apfl_dict_set(ctx, -3, -2, -1);
goto continue_loop;
case INSN_GET_MEMBER:
apfl_get_member(ctx, -2, -1);
goto continue_loop;
case INSN_VAR_NEW:
must_get_argument(ctx, pc, ilist, &arg);
variable_new(ctx, arg.string, false);
goto continue_loop;
case INSN_VAR_NEW_LOCAL:
must_get_argument(ctx, pc, ilist, &arg);
variable_new(ctx, arg.string, true);
goto continue_loop;
case INSN_VAR_GET:
must_get_argument(ctx, pc, ilist, &arg);
variable_get(ctx, arg.string);
goto continue_loop;
case INSN_VAR_SET:
must_get_argument(ctx, pc, ilist, &arg);
variable_set(ctx, arg.string, true, false);
goto continue_loop;
case INSN_VAR_SET_LOCAL:
must_get_argument(ctx, pc, ilist, &arg);
variable_set(ctx, arg.string, true, true);
goto continue_loop;
case INSN_MOVE_TO_LOCAL_VAR:
must_get_argument(ctx, pc, ilist, &arg);
variable_set(ctx, arg.string, false, true);
goto continue_loop;
case INSN_NEXT_LINE:
cse->execution_line++;
goto continue_loop;
case INSN_SET_LINE:
must_get_argument(ctx, pc, ilist, &arg);
cse->execution_line = arg.count;
goto continue_loop;
case INSN_GET_BY_INDEX_KEEP:
must_get_argument(ctx, pc, ilist, &arg);
apfl_get_list_member_by_index(ctx, -1, arg.index);
goto continue_loop;
case INSN_DROP:
if (!apfl_stack_drop(ctx, -1)) {
apfl_raise_invalid_stackidx(ctx);
}
goto continue_loop;
case INSN_CALL:
call(ctx, -2, -1, true);
// By returning from this function, the newly pushed call stack entry (if any) will get picked up by
// evaluate_until_call_stack_return. In case no new CSE was pushed (when a cfunc was called), we'll the
// simply continue with the current call stack.
return;
case INSN_FUNC:
must_get_argument(ctx, pc, ilist, &arg);
func(ctx, arg.body);
goto continue_loop;
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
case INSN_MATCHER_LOAD:
must_get_argument(ctx, pc, ilist, &arg);
matcher_load(ctx, cse, arg.matcher);
goto continue_loop;
case INSN_MATCHER_SET_VAL:
must_get_argument(ctx, pc, ilist, &arg);
matcher_set_val(ctx, cse->matcher, arg.index);
goto continue_loop;
case INSN_MATCHER_MUST_MATCH:
// matcher_init_matching pushes a new call stack entry for the matcher onto the stack. We rturn from this
// So this new CSE gets executed. By setting returning_from_matcher, we know that we came from the matcher,
// once it returns.
matcher_init_matching(ctx, cse->matcher);
cse->returning_from_matcher = true;
return;
case INSN_MATCHER_DROP:
cse->matcher = NULL;
goto continue_loop;
case INSN_VAR_SET_FROM_MATCHER:
must_get_argument(ctx, pc, ilist, &arg);
must_get_argument(ctx, pc, ilist, &arg2);
variable_set_from_matcher(ctx, cse->matcher, arg.string, arg2.index, false);
goto continue_loop;
case INSN_VAR_SET_LOCAL_FROM_MATCHER:
must_get_argument(ctx, pc, ilist, &arg);
must_get_argument(ctx, pc, ilist, &arg2);
variable_set_from_matcher(ctx, cse->matcher, arg.string, arg2.index, true);
goto continue_loop;
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
}
assert(false);
continue_loop:;
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
return_from_function(ctx);
Implement evaluating container member access
2022-01-04 23:25:41 +00:00
}
Implement evaluating dictionary literals We can now have dictionaries with keys and values of arbitrary types. Very cool! :)
2022-01-04 20:51:44 +00:00
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
static void
matcher_stack_push(apfl_ctx ctx, struct matcher_call_stack_entry *cse, struct matcher_stack_entry entry)
{
if (!apfl_resizable_append(
ctx->gc.allocator,
sizeof(struct matcher_stack_entry),
(void **)&cse->matcher_stack,
&cse->matcher_stack_len,
&cse->matcher_stack_cap,
&entry,
1
)) {
apfl_raise_alloc_error(ctx);
}
}
APFL_NORETURN static void
raise_invalid_matcher_state(apfl_ctx ctx)
{
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.invalid_matcher_state);
}
static void
matcher_stack_drop(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
if (cse->matcher_stack_len == 0) {
raise_invalid_matcher_state(ctx);
}
assert(
// We're shrinking, should not fail
apfl_resizable_resize(
ctx->gc.allocator,
sizeof(struct matcher_stack_entry),
(void **)&cse->matcher_stack,
&cse->matcher_stack_len,
&cse->matcher_stack_cap,
cse->matcher_stack_len-1
)
);
}
static void
matcher_init_matching_inner(apfl_ctx ctx, struct matcher *matcher)
{
struct apfl_value value = apfl_stack_must_get(ctx, -1);
must_tmproot_add_value(ctx, value);
eval: Guard against matcher being NULL when matching
2022-07-31 20:04:59 +00:00
if (matcher == NULL) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode);
}
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
struct matcher_call_stack_entry matcher_cse = {
.pc = 0,
.matcher = matcher,
.matcher_stack = NULL,
.matcher_stack_len = 0,
.matcher_stack_cap = 0,
};
matcher_stack_push(ctx, &matcher_cse, (struct matcher_stack_entry) {
.mode = MATCHER_MODE_VALUE,
});
call_stack_push(ctx, (struct call_stack_entry) {
.type = CSE_MATCHER,
.stack = apfl_stack_new(),
.matcher = matcher_cse,
});
apfl_stack_must_push(ctx, apfl_value_set_cow_flag(value));
}
static void
matcher_init_matching(apfl_ctx ctx, struct matcher *matcher)
{
size_t tmproots = apfl_gc_tmproots_begin(&ctx->gc);
matcher_init_matching_inner(ctx, matcher);
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
}
static void
matcher_check_index(apfl_ctx ctx, size_t count, size_t index)
{
if (index >= count) {
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.corrupted_bytecode);
}
}
static struct matcher_stack_entry *
matcher_cur_stack_entry(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
if (cse->matcher_stack_len == 0) {
raise_invalid_matcher_state(ctx);
}
return &cse->matcher_stack[cse->matcher_stack_len-1];
}
static bool
matcher_current_val_in_mstack(apfl_ctx ctx, struct matcher_stack_entry *mstack, struct apfl_value *value)
{
struct apfl_value cur;
switch (mstack->mode) {
case MATCHER_MODE_VALUE:
case MATCHER_MODE_LIST_REMAINING:
if (!apfl_stack_get(ctx, &cur, -1)) {
raise_invalid_matcher_state(ctx);
}
*value = cur;
return true;
case MATCHER_MODE_STOP:
case MATCHER_MODE_LIST_UNDERFLOW:
return false;
case MATCHER_MODE_LIST_START:
if (!apfl_stack_get(ctx, &cur, -1)) {
raise_invalid_matcher_state(ctx);
}
if (cur.type != VALUE_LIST) {
raise_invalid_matcher_state(ctx);
}
if (mstack->lower >= cur.list->len) {
return false;
}
*value = cur.list->items[mstack->lower];
return true;
case MATCHER_MODE_LIST_END:
if (!apfl_stack_get(ctx, &cur, -1)) {
raise_invalid_matcher_state(ctx);
}
if (cur.type != VALUE_LIST) {
raise_invalid_matcher_state(ctx);
}
if (mstack->upper == 0) {
return NULL;
}
*value = cur.list->items[mstack->upper-1];
return true;
}
raise_invalid_matcher_state(ctx);
}
static bool
matcher_current_val(apfl_ctx ctx, struct matcher_call_stack_entry *cse, struct apfl_value *value)
{
struct matcher_stack_entry *mstack = matcher_cur_stack_entry(ctx, cse);
return matcher_current_val_in_mstack(ctx, mstack, value);
}
static bool
matcher_next(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
again:;
struct matcher_stack_entry *mstack = matcher_cur_stack_entry(ctx, cse);
switch (mstack->mode) {
case MATCHER_MODE_VALUE:
mstack->mode = MATCHER_MODE_STOP;
if (!apfl_stack_drop(ctx, -1)) {
raise_invalid_matcher_state(ctx);
}
return true;
case MATCHER_MODE_STOP:
case MATCHER_MODE_LIST_UNDERFLOW:
raise_invalid_matcher_state(ctx);
return false;
case MATCHER_MODE_LIST_START:
mstack->lower++;
return true;
case MATCHER_MODE_LIST_END:
if (mstack->upper <= mstack->lower) {
mstack->mode = MATCHER_MODE_LIST_UNDERFLOW;
return false;
}
mstack->upper--;
return true;
case MATCHER_MODE_LIST_REMAINING:
if (!apfl_stack_drop(ctx, -1)) {
raise_invalid_matcher_state(ctx);
}
matcher_stack_drop(ctx, cse);
goto again; // We also need to advance the previous stack entry,
// like we would do when doing a MATCHER_LEAVE_LIST
}
raise_invalid_matcher_state(ctx);
}
static bool
matcher_enter_list(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
struct matcher_stack_entry *mstack = matcher_cur_stack_entry(ctx, cse);
struct apfl_value cur;
if (!matcher_current_val_in_mstack(ctx, mstack, &cur)) {
return false;
}
if (cur.type != VALUE_LIST) {
return false;
}
size_t len = cur.list->len;
apfl_stack_must_push(ctx, cur);
matcher_stack_push(ctx, cse, (struct matcher_stack_entry) {
.mode = MATCHER_MODE_LIST_START,
.lower = 0,
.upper = len,
});
return true;
}
static void
matcher_continue_from_end(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
struct matcher_stack_entry *mstack = matcher_cur_stack_entry(ctx, cse);
if (mstack->mode != MATCHER_MODE_LIST_START) {
raise_invalid_matcher_state(ctx);
}
mstack->mode = MATCHER_MODE_LIST_END;
}
static void
matcher_remainding(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
struct matcher_stack_entry *mstack = matcher_cur_stack_entry(ctx, cse);
struct apfl_value cur;
if (!apfl_stack_get(ctx, &cur, -1)) {
raise_invalid_matcher_state(ctx);
}
if (
(mstack->mode != MATCHER_MODE_LIST_START && mstack->mode != MATCHER_MODE_LIST_END)
|| cur.type != VALUE_LIST
) {
raise_invalid_matcher_state(ctx);
}
if (mstack->lower > mstack->upper) {
raise_invalid_matcher_state(ctx);
}
struct list_header *cur_list = cur.list;
assert(cur_list->len >= mstack->upper);
size_t len = mstack->upper - mstack->lower;
apfl_list_create(ctx, len);
struct apfl_value new_val = apfl_stack_must_get(ctx, -1);
assert(new_val.type == VALUE_LIST);
struct list_header *new_list = new_val.list;
assert(new_list->cap == len);
assert(new_list->len == 0);
for (size_t i = mstack->lower; i < mstack->upper; i++) {
new_list->items[new_list->len++] = cur_list->items[i];
}
assert(new_list->len == len);
if (!apfl_stack_drop(ctx, -2)) { // Drop the original list
raise_invalid_matcher_state(ctx);
}
mstack->mode = MATCHER_MODE_LIST_REMAINING;
}
static bool
matcher_leave_list(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
struct matcher_stack_entry *mstack = matcher_cur_stack_entry(ctx, cse);
if (mstack->mode != MATCHER_MODE_LIST_START) {
raise_invalid_matcher_state(ctx);
}
if (mstack->lower < mstack->upper) {
// List was not completely matched
return false;
}
if (!apfl_stack_drop(ctx, -1)) {
raise_invalid_matcher_state(ctx);
}
matcher_stack_drop(ctx, cse);
return matcher_next(ctx, cse);
}
static void
return_from_matcher(apfl_ctx ctx, bool result)
{
struct call_stack_entry *cse = apfl_call_stack_cur_entry(ctx);
assert(cse != NULL);
assert(cse->type == CSE_MATCHER);
cse->matcher.matcher->result = result;
call_stack_drop(ctx);
cse = apfl_call_stack_cur_entry(ctx);
assert(cse != NULL);
assert(cse->type == CSE_FUNCTION);
}
#define RETURN_WITHOUT_MATCH(ctx) \
do { \
return_from_matcher((ctx), false); \
return; \
} while (0)
#define RETURN_WITHOUT_MATCH_ON_FALSE(ctx, x) \
do { \
if (!(x)) { \
RETURN_WITHOUT_MATCH(ctx); \
} \
} while (0)
static void
evaluate_matcher(apfl_ctx ctx, struct matcher_call_stack_entry *cse)
{
union matcher_instruction_or_arg arg;
size_t *pc = &cse->pc;
struct matcher *matcher = cse->matcher;
struct matcher_instruction_list *milist = matcher->instructions;
while (*pc < milist->len) {
struct apfl_value cur;
switch (milist->instructions[(*pc)++].instruction) {
case MATCHER_CAPTURE:
if (!matcher_current_val(ctx, cse, &cur)) {
RETURN_WITHOUT_MATCH(ctx);
}
must_get_matcher_argument(ctx, pc, milist, &arg);
matcher_check_index(ctx, milist->capture_count, arg.index);
matcher->captures[arg.index] = apfl_value_set_cow_flag(cur);
RETURN_WITHOUT_MATCH_ON_FALSE(ctx, matcher_next(ctx, cse));
goto continue_loop;
case MATCHER_IGNORE:
if (!matcher_current_val(ctx, cse, &cur)) {
RETURN_WITHOUT_MATCH(ctx);
}
RETURN_WITHOUT_MATCH_ON_FALSE(ctx, matcher_next(ctx, cse));
goto continue_loop;
case MATCHER_CHECK_CONST:
if (!matcher_current_val(ctx, cse, &cur)) {
RETURN_WITHOUT_MATCH(ctx);
}
must_get_matcher_argument(ctx, pc, milist, &arg);
matcher_check_index(ctx, milist->value_count, arg.index);
RETURN_WITHOUT_MATCH_ON_FALSE(ctx, apfl_value_eq(matcher->values[arg.index], cur));
goto continue_loop;
case MATCHER_CHECK_PRED:
if (!matcher_current_val(ctx, cse, &cur)) {
RETURN_WITHOUT_MATCH(ctx);
}
must_get_matcher_argument(ctx, pc, milist, &arg);
matcher_check_index(ctx, milist->value_count, arg.index);
apfl_raise_const_error(ctx, APFL_RESULT_ERR, apfl_messages.feature_not_implemented); // TODO: Implement this
goto continue_loop;
case MATCHER_ENTER_LIST:
RETURN_WITHOUT_MATCH_ON_FALSE(ctx, matcher_enter_list(ctx, cse));
goto continue_loop;
case MATCHER_LEAVE_LIST:
RETURN_WITHOUT_MATCH_ON_FALSE(ctx, matcher_leave_list(ctx, cse));
goto continue_loop;
case MATCHER_CONTINUE_FROM_END:
matcher_continue_from_end(ctx, cse);
goto continue_loop;
case MATCHER_REMAINDING:
matcher_remainding(ctx, cse);
goto continue_loop;
}
assert(false);
continue_loop:;
}
return_from_matcher(
ctx,
// We've successfully matched everything, if there's only one stack element left and we're in stop mode
cse->matcher_stack_len == 1 && cse->matcher_stack[0].mode == MATCHER_MODE_STOP
);
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
struct apfl_iterative_runner_data {
apfl_ctx ctx;
apfl_tokenizer_ptr tokenizer;
apfl_parser_ptr parser;
enum apfl_result result;
bool with_error_on_stack;
bool end;
struct scope *scope;
};
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
iterative_runner_eval_expr_inner(apfl_iterative_runner runner, struct apfl_expr expr)
Implement simple variable support We can now assign to variables and retrieve their values. Not all assignables are implemented yet, but still pretty cool :).
2022-01-06 21:53:26 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_ctx ctx = runner->ctx;
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
struct instruction_list *ilist = apfl_instructions_new(&ctx->gc, expr.position.line);
if (ilist == NULL) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_alloc_error(ctx);
Partially implement pattern matching assignments We're still missing predicates (need to be able to call functions for that one) and assignments into dictionaries. But we now can deconstruct a list and check against constants. So things like this work now: [1 foo ~bar [a b]] = [1 "Hello" 2 3 4 [5 6]] # foo is: "Hello" # bar is: [2 3 4] # a is: 5 # b is: 6 Pretty cool :)
2022-01-14 22:16:19 +00:00
}
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
if (!apfl_gc_tmproot_add(&ctx->gc, GC_OBJECT_FROM(ilist, GC_TYPE_INSTRUCTIONS))) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_alloc_error(ctx);
Partially implement pattern matching assignments We're still missing predicates (need to be able to call functions for that one) and assignments into dictionaries. But we now can deconstruct a list and check against constants. So things like this work now: [1 foo ~bar [a b]] = [1 "Hello" 2 3 4 [5 6]] # foo is: "Hello" # bar is: [2 3 4] # a is: 5 # b is: 6 Pretty cool :)
2022-01-14 22:16:19 +00:00
}
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
struct apfl_error error;
if (!apfl_compile(&ctx->gc, expr, &error, ilist)) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_error_object(ctx, error);
Implement simple variable support We can now assign to variables and retrieve their values. Not all assignables are implemented yet, but still pretty cool :).
2022-01-06 21:53:26 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
call_stack_push(ctx, (struct call_stack_entry) {
.type = CSE_FUNCTION,
.stack = apfl_stack_new(),
.func = {
.pc = 0,
.instructions = ilist,
.scope = runner->scope,
.closure_scope = NULL,
.execution_line = ilist->line,
Implement matching assignments This allows the destructuring of lists into individual values. We can have arbitrarily nested lists, can check for constant values and can have up to one '~'-prefixed variable per list, that will capture the remaining elements of the list. It is implemented as a second set of bytecode instructions, which define a matcher. These matchers should also enable us to implement the same pattern matching capabiities for function parameters. Not all matching features are implemented yet, predicate matching and matching into a dictionary key is not implemented yet.
2022-07-28 18:46:32 +00:00
.returning_from_matcher = false,
.matcher = NULL,
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
},
});
evaluate_until_call_stack_return(ctx);
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
iterative_runner_eval_expr(apfl_iterative_runner runner, struct apfl_expr expr)
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_ctx ctx = runner->ctx;
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
size_t tmproots = apfl_gc_tmproots_begin(&ctx->gc);
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
iterative_runner_eval_expr_inner(runner, expr);
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
apfl_gc_tmproots_restore(&ctx->gc, tmproots);
Start work on evaluating expressions Right now we're only evaluating bool/nil/number/string/list literals, but it's a start :).
2022-01-02 16:19:54 +00:00
}
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
Use formatter for printing values
2022-04-22 21:13:01 +00:00
bool
apfl_debug_print_val: Use format writer abstraction
2022-06-05 20:06:33 +00:00
apfl_debug_print_val(apfl_ctx ctx, apfl_stackidx index, struct apfl_format_writer w)
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
{
struct apfl_value value;
Implement mark&sweep garbage collection and bytecode compilation Instead of the previous refcount base garbage collection, we're now using a basic tri-color mark&sweep collector. This is done to support cyclical value relationships in the future (functions can form cycles, all values implemented up to this point can not). The collector maintains a set of roots and a set of objects (grouped into blocks). The GC enabled objects are no longer allocated manually, but will be allocated by the GC. The GC also wraps an allocator, this way the GC knows, if we ran out of memory and will try to get out of this situation by performing a full collection cycle. The tri-color abstraction was chosen for two reasons: - We don't have to maintain a list of objects that need to be marked, we can simply grab the next grey one. - It should allow us to later implement incremental collection (right now we only do a stop-the-world collection). This also switches to a bytecode based evaluation of the code: We no longer directly evaluate the AST, but first compile it into a series of instructions, that are evaluated in a separate step. This was done in preparation for inplementing functions: We only need to turn a function body into instructions instead of evaluating the node again with each call of the function. Also, since an instruction list is implemented as a GC object, this then removes manual memory management of the function body and it's child nodes. Since the GC and the bytecode go hand in hand, this was done in one (giant) commit. As a downside, we've now lost the ability do do list matching on assignments. I've already started to work on implementing this in the new architecture, but left it out of this commit, as it's already quite a large commit :)
2022-04-11 20:24:22 +00:00
if (!apfl_stack_pop(ctx, &value, index)) {
apfl_debug_print_val: Use format writer abstraction
2022-06-05 20:06:33 +00:00
FMT_TRY(apfl_format_put_string(w, "apfl_debug_print_val: Invalid stack index "));
FMT_TRY(apfl_format_put_int(w, (int)index));
FMT_TRY(apfl_format_put_string(w, "\n"));
Use formatter for printing values
2022-04-22 21:13:01 +00:00
return true;
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
}
apfl_debug_print_val: Use format writer abstraction
2022-06-05 20:06:33 +00:00
return apfl_value_print(value, w);
Convert to Lua-style stack API Only for evaluating expressions for now and right now the only exposed operation is to debug print a value on the stack, this obviously needs to be expanded. I've done this for two reasons: 1. A Lua-style stack API is much nicer to work with than to manually manage refcounts. 2. We'll soon need a more sophisticated garbage collector (if you even want to count the refcounting as garbage collection). For this, the GC will need root objects to start tracing for live objects, the stack will be one of these roots.
2022-01-20 21:45:09 +00:00
}
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
apfl_iterative_runner
apfl_iterative_runner_new(apfl_ctx ctx, struct apfl_source_reader reader)
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_iterative_runner runner = NULL;
apfl_tokenizer_ptr tokenizer = NULL;
apfl_parser_ptr parser = NULL;
runner = ALLOC_OBJ(ctx->gc.allocator, struct apfl_iterative_runner_data);
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
if (runner == NULL) {
return NULL;
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
tokenizer = apfl_tokenizer_new(ctx->gc.allocator, reader);
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
if (tokenizer == NULL) {
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
goto error;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
parser = apfl_parser_new(ctx->gc.allocator, apfl_tokenizer_as_token_source(tokenizer));
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
if (parser == NULL) {
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
goto error;
}
struct scope *scope = apfl_scope_new(&ctx->gc);
if (scope == NULL) {
goto error;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
}
*runner = (struct apfl_iterative_runner_data) {
.ctx = ctx,
.tokenizer = tokenizer,
.parser = parser,
.result = APFL_RESULT_OK,
.end = false,
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
.scope = scope,
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
};
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
if (!apfl_ctx_register_iterative_runner(ctx, runner)) {
goto error;
}
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
return runner;
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
error:
FREE_OBJ(ctx->gc.allocator, runner);
apfl_tokenizer_destroy(tokenizer);
apfl_parser_destroy(parser);
return NULL;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
handle_parse_error(apfl_ctx ctx, struct apfl_error error)
{
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
enum apfl_result errtype = APFL_RESULT_ERR;
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
if (apfl_error_is_fatal_type(error.type)) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
errtype = APFL_RESULT_ERR_FATAL;
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
}
Push some errors as const strings onto the stack
2022-04-24 14:10:35 +00:00
const char *const_str = apfl_error_as_const_string(error);
if (const_str != NULL) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_const_error(ctx, errtype, const_str);
Push some errors as const strings onto the stack
2022-04-24 14:10:35 +00:00
}
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
struct apfl_string string;
if (!apfl_error_as_string(error, ctx->gc.allocator, &string)) {
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_alloc_error(ctx);
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
if (!apfl_move_string_onto_stack(ctx, string)) {
apfl_raise_alloc_error(ctx);
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
apfl_raise_error_with_type(ctx, -1, errtype);
Push formatted parser error on stack during evaluation This way we can see the parse errors again in evaluation mode Not fully fleshed out yet: We simply use apfl_debug_print_val to dump the top of the stack (the formatted error) to stderr but don't nicely handle if there is nothing on the stack (apfl_debug_print_val will print a rather cryptic "stack index invalid" error). Also the whole dance we need to do to put the formatted error onto the stack feels rather awkward (there should probably a function for this) and we also should probably try to push an error description on the stack in case this moving-string-to-stack business fails. Now "only" all other errors need to be put on the stack as a string :)
2022-04-21 20:40:27 +00:00
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
static void
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
iterative_runner_next_protected(apfl_ctx ctx)
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
{
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_stack_drop(ctx, -1);
apfl_cfunc_self_getslot(ctx, 0);
apfl_iterative_runner runner = apfl_get_userdata(ctx, -1);
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
switch (apfl_parser_next(runner->parser)) {
case APFL_PARSE_OK:
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
iterative_runner_eval_expr(runner, apfl_parser_get_expr(runner->parser));
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
return;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
case APFL_PARSE_ERROR:
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
handle_parse_error(runner->ctx, apfl_parser_get_error(runner->parser));
return;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
case APFL_PARSE_EOF:
runner->end = true;
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
return;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
}
assert(false);
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
}
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
bool
apfl_iterative_runner_next(apfl_iterative_runner runner)
{
if (runner->end) {
return false;
}
apfl_stack_clear(runner->ctx);
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
// TODO: It's a bit silly that we build the cfunc every time. Can we be
// smarter and leave it on the stack somewhere? Or save it in the
// runner? Also, what if the construction of the func fails? We're not
// running protected yet :/
apfl_push_cfunc(runner->ctx, iterative_runner_next_protected, 1);
apfl_push_userdata(runner->ctx, runner);
apfl_cfunc_setslot(runner->ctx, -2, 0, -1);
apfl_list_create(runner->ctx, 0);
apfl_call_protected(runner->ctx, -2, -1, &runner->with_error_on_stack);
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
return !runner->end;
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
}
enum apfl_result
apfl_iterative_runner_get_result(apfl_iterative_runner runner)
{
return runner->result;
}
Implement exceptions-like error handling Most functzions will no longer return an enum apfl_result, but will raise an error that bubbles up.
2022-06-24 21:13:44 +00:00
bool
apfl_iterative_runner_has_error_on_stack(apfl_iterative_runner runner)
{
return runner->with_error_on_stack;
}
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
void
apfl_iterative_runner_destroy(apfl_iterative_runner runner)
{
if (runner == NULL) {
return;
}
apfl_parser_destroy(runner->parser);
apfl_tokenizer_destroy(runner->tokenizer);
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
apfl_ctx_unregister_iterative_runner(runner->ctx, runner);
No longer evaluate exprs directly
2022-04-21 19:15:20 +00:00
FREE_OBJ(runner->ctx->gc.allocator, runner);
}
Implement functions and function calls We can now define and call functions. Lexical closure scopes are also working :). It's limited to simple functions or complex functions with a single argument list of only variable names for now.
2022-07-11 19:41:05 +00:00
void
apfl_iterative_runner_visit_gc_objects(apfl_iterative_runner runner, gc_visitor visitor, void *opaque)
{
// TODO: It's a bit awkward that this function is defined here but the
// prototype lives in context.h... Maybe we should just merge context
// and eval together? The separation is rather arbitrary anyway :/
if (runner->scope != NULL) {
visitor(opaque, GC_OBJECT_FROM(runner->scope, GC_TYPE_SCOPE));
}
}
Reference in a new issue Copy permalink