eddic 1.2.4: New Boost Spirit X3 parser and minor cleanups

Pros

Let's start with the pros.

First, the runtime performance is definitely better. Parsing all my eddi test cases and samples, takes 42% less time than with the previous parser. It is important to know that the old parser was very optimized, with moves instead of copies and with a static lexer. You can take a look at this post to see what was necessary to optimize the old Qi. I think it's a good result since the new grammar does not use a lexer (x3 does not support it) and does not need these optimizations. This improvement really was my objective. I'll try to push it farther in the future.

Compile-time performance is also much better. It takes 3 times less time to compile the new parser (1 minute to around 20 seconds). Moreover, the new parser is now in only one file, rather than being it necessary to split it all over the place for compile-time performance. Even though it's not really important for me, it's still good to have :)

Especially due to the performance point, I've been able to remove some code, the lexer, the generated static lexer and the special pointers optimizations of the AST.

Cons

Unfortunately, there are some disadvantages of using the new Spirit X3.

First, the AST needs to be changed. For good parsing performance, you need to use x3::variant and x3::forward_ast. This is a major pain in the ass since x3::variant is much less practical to use than boost::variant. Almost everything is explicit, meaning uglier code than before, in my opinion. Moreover, you need to work around x3::forward_ast for boost::get, whereas boost::recursive_wrapper was working better in that matter. I've had to create my own wrapper around boost::get in order to be able to use the new tree. In my opinion, this is clearly a regression.

Secondly, although X3 was also meant to remove the need to use some hacks in the grammar, I ended up having more hacks than before. For instance, many AST node have a fake field in order to make X3 happy. I've still had to use the horrible eps hack at one place. I've had to create a few more rules in order to fix type deduction that is working differently than before (worse for me). And for some reasons, I had to replace some expectations from the grammar to make it parse correctly. This is a really important regression in my opinion, since it may make the parsing slower and will make the error message less nice.

The previous error handling system allowed me to track the file from which an AST node was parsed from. Although the new error handler is a lot nicer than the old system, it does not have this feature, so I had to work around this by using new annotation nodes and a new global handler. Overall, it's probably a bit worse than before, but makes for lighter AST nodes.

Finally, for some reason, I haven't been able to use the debug option of the library (lots of compile time errors). That complicated a bit the debugging of the parser.

Spirit X3 or Spirit Qi ?

Overall, I have to say I'm a bit disappointed by Spirit X3. Even though it's faster at runtime and faster to compile, I was really expecting less issues with it. What I really did not like was all the changes I had to make because of x3::variant and x3::forward_ast. Overall, I really don't think it was worth the trouble porting my parser to Spirit X3.

If you have a new project, I would still consider using Boost Spirit X3.

If you have an existing parser, I would probably not advice porting it to X3. Unless you really have issues with parsing performances (and especially if you have not already optimized QI parser), it's probably not worth the trouble and all the time necessary for all the changes.