CLI for Incremental Batch Builds

We have tasks for compiling grammars and parsing files, but we need to pass file paths and a rule name into these tasks. We will pass this data to the program via command-line arguments. To parse command-line arguments, we will use clap, which is an awesome library for easily parsing command-line arguments. Add clap as a dependency to pie/Cargo.toml:

We’re using the derive feature of clap to automatically derive a full-featured argument parser from a struct. Modify pie/examples/parser_dev/main.rs:

The Args struct contains exactly the data we need: the path to the grammar file, the name of the rule to start parsing with, and paths to program files to parse. We derive an argument parser for Args with #[derive(Parser)]. Then we parse command-line arguments in main with Args::parse().

Test this program with cargo run --example parser_dev -- --help, which should result in usage help for the program. Note that the names, ordering, and doc-comments of the fields are used to generate this help. You can test out several more commands:

• cargo run --example parser_dev --
• cargo run --example parser_dev -- foo
• cargo run --example parser_dev -- foo bar
• cargo run --example parser_dev -- foo bar baz qux

Now let’s use these arguments to actually compile the grammar and parse example program files. Modify pie/examples/parser_dev/main.rs:

In compile_grammar_and_parse, we create a new Pie instance that writes the build log to stderr, and create a new build session. Then, we require a compile grammar task using the grammar_file_path from Args, and write any errors to the errors String. We then require a parse task for every path in args.program_file_paths, using the previously created compile_grammar_task and args.rule_name. Successes are printed to stdout and errors are written to errors. Finally, we print errors to stdout if there are any.

To test this out, we need a grammar and some test files. Create grammar.pest:

num = @{ ASCII_DIGIT+ }

main = { SOI ~ num ~ EOI }

WHITESPACE = _{ " " | "\t" | "\n" | "\r" }

Create test_1.txt with:

42

And create test_2.txt with:

foo

Run the program with cargo run --example parser_dev -- grammar.pest main test_1.txt test_2.txt. This should result in a build log showing that the grammar is successfully compiled, that one file is successfully parsed, and that one file has a parse error.

Unfortunately, there is no incrementality between different runs of the example, because the Pie Store is not persisted. The Store only exists in-memory while the program is running, and is then thrown away. Thus, there cannot be any incrementality. To get incrementality, we need to serialize the Store before the program exits, and deserialize it when the program starts. This is possible and not actually that hard, I just never got around to explaining it in this tutorial. See the Side Note: Serialization section at the end for info on how this can be implemented.

Feel free to experiment a bit with the grammar, example files, etc. before continuing. We will develop an interactive editor next however, which will make experimentation easier!