From 00ab5c3488c02beab5c3f4e371f5196404334e3c Mon Sep 17 00:00:00 2001 From: dec05eba Date: Sat, 2 Mar 2019 00:40:08 +0100 Subject: Fix crash in parser import, optimize tokenizer_consume_if to not reparse if already parsed --- doc/IMPLEMENTATION.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc') diff --git a/doc/IMPLEMENTATION.md b/doc/IMPLEMENTATION.md index 1d86297..0da1c1a 100644 --- a/doc/IMPLEMENTATION.md +++ b/doc/IMPLEMENTATION.md @@ -1,5 +1,7 @@ # Goal 1. In the first stage the parser parses multiple files at the same time using multiple threads. +The tokenization should be done without storing the tokens in a list (streaming) but AST needs to be stored in a list +because the compiler needs to support out of order declarations. 2. In the second stage the ast is handled using multiple threads. In this stage, variables, parameters and types are defined and resolved and if a type is defined after there is a reference to it, then the compiler first resolves that type. There are flags set to make sure there aren't recursive dependencies. -- cgit v1.2.3