Tracking Build Events

So far we have had no convenient way to inspect what our build system is doing, apart from println! debugging or attaching a debugger to the program. In this section, we will change that by tracking build events for debugging and integration testing purposes.

We will:

  1. Create a Tracker trait that receives build events through method calls. The Tracker trait can be implemented in different ways to handle build events in different ways.
  2. Implement a NoopTracker that does nothing, removing the tracking overhead.
  3. Make the build system generic over Tracker, such that Context implementations call methods on the tracker to create build events.
  4. Implement a WritingTracker that writes build events to standard output or standard error, for debugging purposes.
  5. Implement an EventTracker that stores build events for later inspection, for integration testing purposes.
  6. Implement a CompositeTracker that forwards build events to 2 other trackers, so we can use multiple trackers at the same time.

Tracker trait

Add the tracker module to pie/src/lib.rs:

Then create the pie/src/tracker directory, create the pie/src/tracker/mod.rs file, and add the following content:

use std::io;

use crate::dependency::{Dependency, FileDependency, Inconsistency, TaskDependency};
use crate::stamp::OutputStamper;
use crate::Task;

/// Trait for tracking build events. Can be used to implement logging, event tracing, progress tracking, metrics, etc.
#[allow(unused_variables)]
pub trait Tracker<T: Task> {
  /// Start: a new build.
  fn build_start(&mut self) {}
  /// End: completed build.
  fn build_end(&mut self) {}

  /// End: created a file `dependency`.
  fn require_file_end(&mut self, dependency: &FileDependency) {}
  /// Start: require `task` using `stamper`.
  fn require_task_start(&mut self, task: &T, stamper: &OutputStamper) {}
  /// End: required a task, resulting in a task `dependency` and `output`, and the task `was_executed`.
  fn require_task_end(&mut self, dependency: &TaskDependency<T, T::Output>, output: &T::Output, was_executed: bool) {}

  /// Start: check consistency of `dependency`.
  fn check_dependency_start(&mut self, dependency: &Dependency<T, T::Output>) {}
  /// End: checked consistency of `dependency`, possibly found `inconsistency`.
  fn check_dependency_end(
    &mut self,
    dependency: &Dependency<T, T::Output>,
    inconsistency: Result<Option<&Inconsistency<T::Output>>, &io::Error>
  ) {}

  /// Start: execute `task`.
  fn execute_start(&mut self, task: &T) {}
  /// End: executed `task` resulting in `output`.
  fn execute_end(&mut self, task: &T, output: &T::Output) {}
}

The Tracker trait is generic over Task.

Trait Bound

Here, we chose to put the Task trait bound on the trait itself. This will not lead to cascading trait bounds, as the Tracker trait will only be used as a bound in impls, not in structs or other traits.

Tracker has methods corresponding to events that happen during a build, such as a build starting, requiring a file, requiring a task, checking a dependency, and executing a task. All but the require_file event have start and end variants to give trackers control over nesting these kind of events. Then end variants usually have more parameters as more info is available when something is has finished.

Tracker methods accept &mut self so that tracker implementations can perform mutation, such as storing a build event. We provide default methods that do nothing so that implementors of Tracker only have to override the methods for events they are interested in. We use #[allow(unused_variables)] on the trait to not give warnings for unused variables, as all variables are unused due to the empty default implementations.

Rust Help: References in Result and Option

The check_dependency_end method accepts the inconsistency as Result<Option<&Inconsistency<T::Output>>, &io::Error>. The reason we accept it like this is that many methods in Result and Option take self, not &self, and therefore cannot be called on &Result<T, E> and &Option<T>.

We can turn &Result<T, E> into Result<&T, &E> with as_ref (same for Option). Since trackers always want to work with Result<&T, &E>, it makes more sense for the caller of the tracker method to call as_ref to turn their result into Result<&T, &E>.

The final reason to accept Result<&T, &E> is that if you have a &T or &E, you can easily construct a Result<&T, &E> with Ok(&t) and Err(&e). However, you cannot construct a &Result<T, E> from &T or &E, so Result<&T, &E> is a more flexible type.

Are these Default Methods Useful?

Adding a method to Tracker with a default implementation ensures that implementations of Tracker do not have to be changed to work with the new method. This is both good and bad. Good because we can add methods without breaking compatibility. Bad because we can forget to handle a new method, which can lead to problems with for example a composite tracker that forwards events to 2 trackers. In this tutorial we chose the convenient option, but be sure to think about these kind of tradeoffs yourself!

Check that the code compiles with cargo test.

No-op tracker

Add a no-op tracker, which is a tracker that does nothing, by adding the following code to pie/src/tracker/mod.rs:

/// [`Tracker`] that does nothing.
#[derive(Copy, Clone, Debug)]
pub struct NoopTracker;
impl<T: Task> Tracker<T> for NoopTracker {}

Due to the default methods that do nothing on Tracker, this implementation is extremely simple.

Rust Help: Removing Tracker Overhead

We will use generics to select which tracker implementation to use. Therefore, all calls to trackers are statically dispatched, and could be inlined. Because NoopTracker only has empty methods, and those empty methods can be inlined, using NoopTracker will effectively remove all tracking code from your binary, thus removing the overhead of tracking if you don’t want it.

In this tutorial, we do not annotate methods with #[inline], meaning that the Rust compiler (and the LLVM backend) will make its own decisions on what to make inlineable and what not. If you care about performance here, be sure to annotate those default empty methods with #[inline].

Using the Tracker trait

Now we will make the build system generic over Tracker, and insert Tracker calls in context implementations.

Make Pie and Session generic over Tracker by modifying pie/src/lib.rs:

We use A as the generic argument for tracker types in the source code. The Pie struct owns the tracker, similarly to how it owns the store. Pie can be created with a specific tracker with with_tracker, and provides access to the tracker with tracker and tracker_mut.

Rust Help: Default Type

We assign NoopTracker as the default type for trackers in Pie, so that no tracking is performed when we use the Pie type without an explicit tracker type. The Default implementation only works with NoopTracker, because we impl Default for Pie<T, T::Output>, which is equivalent to impl Default for Pie<T, T::Output, NoopTracker> due to the default type.

We make Session generic over trackers, and mutibly borrow the tracker from Pie, again like we do with the store. For convenience, Session also provides access to the tracker with tracker and tracker_mut.

Now we make TopDownContext generic over Tracker, and insert calls to tracker methods. Modify pie/src/context/top_down.rs:

We make TopDownContext generic over trackers, and call methods on the tracker:

  • In require_initial we call build_start/build_end to track builds.
  • In require_file_with_stamper we call require_file_end to track file dependencies.
  • In require_file_with_stamper we call require_task_start/require_task_end to track task dependencies.
    • We extract should_execute into a variable, and pull dependency out of the if, so that we can pass them to tracker.required_task.
    • We also call execute_start/execute_end to track execution.
  • In should_execute_task we call check_dependency_start/check_dependency_end to track dependency checking.
    • We extract inconsistency into a variable, and convert it into the right type for check_dependency_end.

Check that the code compiles with cargo test. Existing code should keep working due to the NoopTracker default type in Pie.

We won’t modify NonIncrementalContext to use a tracker, as NonIncrementalContext has no state, so we cannot pass a tracker to it.

Implement writing tracker

Now we can implement some interesting trackers. We start with a simple WritingTracker that writes build events to some writer.

Add the writing module to pie/src/tracker/mod.rs:

Then create the pie/src/tracker/writing.rs file and add:

use std::io::{self, BufWriter, Stderr, Stdout, Write};

use crate::dependency::{Dependency, FileDependency, Inconsistency, TaskDependency};
use crate::stamp::OutputStamper;
use crate::Task;
use crate::tracker::Tracker;

/// [`Tracker`] that writes events to a [`Write`] instance, for example [`Stdout`].
#[derive(Clone, Debug)]
pub struct WritingTracker<W> {
  writer: W,
  indentation: u32,
}

impl WritingTracker<BufWriter<Stdout>> {
  /// Creates a [`WritingTracker`] that writes to buffered standard output.
  pub fn with_stdout() -> Self { Self::new(BufWriter::new(io::stdout())) }
}
impl WritingTracker<BufWriter<Stderr>> {
  /// Creates a [`WritingTracker`] that writes to buffered standard error.
  pub fn with_stderr() -> Self { Self::new(BufWriter::new(io::stderr())) }
}
impl<W: Write> WritingTracker<W> {
  /// Creates a [`WritingTracker`] that writes to `writer`.
  pub fn new(writer: W) -> Self {
    Self {
      writer,
      indentation: 0,
    }
  }

  /// Gets the writer of this writing tracker.
  pub fn writer(&self) -> &W { &self.writer }
  /// Gets the mutable writer of this writing tracker.
  pub fn writer_mut(&mut self) -> &mut W { &mut self.writer }
}

The WritingTracker is generic over a writer W that must implement std::io::Write, which is a standard trait for writing bytes to something. with_stdout and with_stderr are used to create buffered writers to standard output and standard error. new can be used to create a writer to anything that implements Write, such as a File. writer and writer_mut are for retrieving the underlying writer.

Add some utility functions for WritingTracker to pie/src/tracker/writing.rs: 

#[allow(dead_code)]
impl<W: Write> WritingTracker<W> {
  fn writeln(&mut self, args: std::fmt::Arguments) {
    self.write_indentation();
    let _ = writeln!(&mut self.writer, "{}", args);
  }
  fn write(&mut self, args: std::fmt::Arguments) {
    let _ = write!(&mut self.writer, "{}", args);
  }
  fn write_nl(&mut self) {
    let _ = write!(&mut self.writer, "\n");
  }

  fn indent(&mut self) {
    self.indentation = self.indentation.saturating_add(1);
  }
  fn unindent(&mut self) {
    self.indentation = self.indentation.saturating_sub(1);
  }
  fn write_indentation(&mut self) {
    for _ in 0..self.indentation {
      let _ = write!(&mut self.writer, " ");
    }
  }

  fn flush(&mut self) {
    let _ = self.writer.flush();
  }
}

writeln and write will mainly be used for writing text. The text to write is passed into these methods using std::fmt::Arguments for flexibility, accepting the result of format_args!. WritingTracker keeps track of indentation to show recursive dependency checking and execution, which is controlled with indent and unindent. Since we are usually writing to buffers, we must flush them to observe the output.

Failing Writes

Writes can fail, but we silently ignore them in this tutorial (with let _ = ...) for simplicity. You could panic when writing fails, but panicking when writing to standard output fails is probably going a bit too far. You could store the latest write error and give access to it, which at least allows users of WritingTracker check for some errors.

In general, tracking events can fail, but the current Tracker API does not allow for propagating these errors with Result. This in turn because TopDownContext does not return Result for require_task due to the trade-offs discussed in the section on TopDownContext.

Rust Help: Saturating Arithmetic

We use saturating_add and saturating_sub for safety, which are saturating arithmetic operations that saturate at the numeric bounds instead of overflowing. For example, 0u32.saturating_sub(1) will result in 0 instead of overflowing into 4294967295.

These saturating operations are not really needed when calls to indent and unindent are balanced. However, if we make a mistake, it is better to write no indentation than to write 4294967295 spaces of indentation.

Alternatively, we could use standard arithmetic operations, which panic on overflow in debug/development mode, but silently overflow in release mode.

Now we can implement the tracker using these utility methods. Add the Tracker implementation to pie/src/tracker/writing.rs:

impl<W: Write, T: Task> Tracker<T> for WritingTracker<W> {
  fn build_start(&mut self) {
    self.indentation = 0;
  }
  fn build_end(&mut self) {
    self.writeln(format_args!("🏁"));
    self.flush();
  }

  fn require_file_end(&mut self, dependency: &FileDependency) {
    self.writeln(format_args!("- {}", dependency.path().display()));
  }
  fn require_task_start(&mut self, task: &T, _stamper: &OutputStamper) {
    self.writeln(format_args!("→ {:?}", task));
    self.indent();
    self.flush();
  }
  fn require_task_end(&mut self, _dependency: &TaskDependency<T, T::Output>, output: &T::Output, _was_executed: bool) {
    self.unindent();
    self.writeln(format_args!("← {:?}", output));
    self.flush();
  }

  fn check_dependency_start(&mut self, dependency: &Dependency<T, T::Output>) {
    match dependency {
      Dependency::RequireTask(d) => {
        self.writeln(format_args!("? {:?}", d.task()));
        self.indent();
        self.flush();
      },
      _ => {},
    }
  }
  fn check_dependency_end(
    &mut self,
    dependency: &Dependency<T, T::Output>,
    inconsistency: Result<Option<&Inconsistency<T::Output>>, &io::Error>
  ) {
    match dependency {
      Dependency::RequireFile(d) => {
        match inconsistency {
          Err(e) => self.writeln(format_args!("✗ {} (err: {:?})", d.path().display(), e)),
          Ok(Some(Inconsistency::File(s))) =>
            self.writeln(format_args!("✗ {} (old: {:?} ≠ new: {:?})", d.path().display(), d.stamp(), s)),
          Ok(None) => self.writeln(format_args!("✓ {}", d.path().display())),
          _ => {}, // Other variants cannot occur.
        }
      },
      Dependency::RequireTask(d) => {
        self.unindent();
        match inconsistency {
          Ok(Some(Inconsistency::Task(s))) =>
            self.writeln(format_args!("✗ {:?} (old: {:?} ≠ new: {:?})", d.task(), d.stamp(), s)),
          Ok(None) => self.writeln(format_args!("✓ {:?}", d.task())),
          _ => {}, // Other variants cannot occur.
        }
      }
    }
    self.flush()
  }

  fn execute_start(&mut self, task: &T) {
    self.writeln(format_args!("▶ {:?}", task));
    self.indent();
    self.flush();
  }
  fn execute_end(&mut self, _task: &T, output: &T::Output) {
    self.unindent();
    self.writeln(format_args!("◀ {:?}", output));
    self.flush();
  }
}

We implement most tracker methods and write what is happening, using some unicode symbols to signify events:

  • 🏁: end of a build,
  • -: created a file dependency,
  • →: start requiring a task,
  • ←: end of requiring a task,
  • ?: start checking a task dependency,
  • ✓: end of dependency checking, when the dependency is consistent,
  • ✗: end of dependency checking, when the dependency is inconsistent,
  • ▶: start of task execution,
  • ◀: end of task execution.

We flush the writer after every event to ensure that bytes are written out. When a task is required, checked, or executed, we increase indentation to signify the recursive checking/execution. When a task is done being required, checked, or executed, we decrease the indentation again. In check_dependency_end we write the old and new stamps if a dependency is inconsistent.

This tracker is very verbose. You can add configuration booleans to control what should be written, but in this tutorial we will keep it simple like this.

Check that the code compiles with cargo test.

Let’s try out our writing tracker in the incrementality example by modifying pie/examples/incremental.rs:

We remove the println! statements from tasks and create Pie with WritingTracker. Now run the example with cargo run --example incremental, and you should see the writing tracker print to standard output:

   Compiling pie v0.1.0 (/pie)
    Finished dev [unoptimized + debuginfo] target(s) in 0.44s
     Running `target/debug/examples/incremental`
A) New task: expect `read_task` to execute
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input.txt", Modified)
 ▶ ReadStringFromFile("/tmp/.tmpS2I0ZA/input.txt", Modified)
  - /tmp/.tmpS2I0ZA/input.txt
 ◀ Ok("Hi")
← Ok("Hi")
🏁

B) Reuse: expect no execution
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input.txt", Modified)
 ✓ /tmp/.tmpS2I0ZA/input.txt
← Ok("Hi")
🏁

C) Inconsistent file dependency: expect `read_task` to execute
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input.txt", Modified)
 ✗ /tmp/.tmpS2I0ZA/input.txt (old: Modified(Some(SystemTime { tv_sec: 1703256537, tv_nsec: 965189499 })) ≠ new: Modified(Some(SystemTime { tv_sec: 1703256537, tv_nsec: 969189501 })))
 ▶ ReadStringFromFile("/tmp/.tmpS2I0ZA/input.txt", Modified)
  - /tmp/.tmpS2I0ZA/input.txt
 ◀ Ok("Hello")
← Ok("Hello")
🏁

D) Different tasks: expect `read_task_b_modified` and `read_task_b_exists` to execute
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Modified)
 ▶ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Modified)
  - /tmp/.tmpS2I0ZA/input_b.txt
 ◀ Ok("Test")
← Ok("Test")
🏁
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Exists)
 ▶ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Exists)
  - /tmp/.tmpS2I0ZA/input_b.txt
 ◀ Ok("Test")
← Ok("Test")
🏁

E) Different stampers: expect only `read_task_b_modified` to execute
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Modified)
 ✗ /tmp/.tmpS2I0ZA/input_b.txt (old: Modified(Some(SystemTime { tv_sec: 1703256537, tv_nsec: 969189501 })) ≠ new: Modified(Some(SystemTime { tv_sec: 1703256537, tv_nsec: 973189502 })))
 ▶ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Modified)
  - /tmp/.tmpS2I0ZA/input_b.txt
 ◀ Ok("Test Test")
← Ok("Test Test")
🏁
→ ReadStringFromFile("/tmp/.tmpS2I0ZA/input_b.txt", Exists)
 ✓ /tmp/.tmpS2I0ZA/input_b.txt
← Ok("Test")
🏁

Implement event tracker

The writing tracker is great for debugging purposes, but we cannot use it to check whether our build system is incremental and sound. To check incrementality and soundness, we need to be able to check whether a task has executed or not, and check the order of build events. Therefore, we will implement the EventTracker that stores build events for later inspection.

Add the event module to pie/src/tracker/mod.rs:

Then create the pie/src/tracker/event.rs file and add:

use std::ops::RangeInclusive;
use std::path::{Path, PathBuf};

use crate::dependency::{FileDependency, TaskDependency};
use crate::stamp::{FileStamp, FileStamper, OutputStamp, OutputStamper};
use crate::Task;
use crate::tracker::Tracker;

/// [`Tracker`] that stores [events](Event) in a [`Vec`], useful in testing to assert that a context implementation is
/// incremental and sound.
#[derive(Clone, Debug)]
pub struct EventTracker<T, O> {
  events: Vec<Event<T, O>>,
}

impl<T: Task> Default for EventTracker<T, T::Output> {
  fn default() -> Self { Self { events: Vec::new() } }
}

/// Enumeration of important build events.
#[derive(Clone, Debug)]
pub enum Event<T, O> {
  RequireFileEnd(RequireFileEnd),

  RequireTaskStart(RequireTaskStart<T>),
  RequireTaskEnd(RequireTaskEnd<T, O>),

  ExecuteStart(ExecuteStart<T>),
  ExecuteEnd(ExecuteEnd<T, O>),
}

/// End: required file at `path` using `stamper` to create `stamp`.
#[derive(Clone, Debug)]
pub struct RequireFileEnd {
  pub path: PathBuf,
  pub stamper: FileStamper,
  pub stamp: FileStamp,
  pub index: usize,
}
/// Start: require `task` using `stamper`.
#[derive(Clone, Debug)]
pub struct RequireTaskStart<T> {
  pub task: T,
  pub stamper: OutputStamper,
  pub index: usize,
}
/// End: required `task` resulting in `output`, using `stamper` to create `stamp`, and the task `was_executed`.
#[derive(Clone, Debug)]
pub struct RequireTaskEnd<T, O> {
  pub task: T,
  pub output: O,
  pub stamper: OutputStamper,
  pub stamp: OutputStamp<O>,
  pub was_executed: bool,
  pub index: usize,
}
/// Start: execute `task`.
#[derive(Clone, Debug)]
pub struct ExecuteStart<T> {
  pub task: T,
  pub index: usize,
}
/// End: executed `task`, producing `output`.
#[derive(Clone, Debug)]
pub struct ExecuteEnd<T, O> {
  pub task: T,
  pub output: O,
  pub index: usize,
}

The EventTracker stores build events in a Vec. The Event enumeration mimics the relevant Tracker methods, but uses structs with all arguments in owned form (for example task: T instead of task: &T) as we want to store these events. We also store the index of every event, so we can easily check whether an event happened before or after another.

Add the tracker implementation to pie/src/tracker/event.rs:

impl<T: Task> Tracker<T> for EventTracker<T, T::Output> {
  fn build_start(&mut self) {
    self.events.clear();
  }

  fn require_file_end(&mut self, dependency: &FileDependency) {
    let data = RequireFileEnd {
      path: dependency.path().into(),
      stamper: *dependency.stamper(),
      stamp: *dependency.stamp(),
      index: self.events.len()
    };
    self.events.push(Event::RequireFileEnd(data));
  }
  fn require_task_start(&mut self, task: &T, stamper: &OutputStamper) {
    let data = RequireTaskStart { task: task.clone(), stamper: stamper.clone(), index: self.events.len() };
    self.events.push(Event::RequireTaskStart(data));
  }
  fn require_task_end(&mut self, dependency: &TaskDependency<T, T::Output>, output: &T::Output, was_executed: bool) {
    let data = RequireTaskEnd {
      task: dependency.task().clone(),
      stamper: *dependency.stamper(),
      stamp: dependency.stamp().clone(),
      output: output.clone(),
      was_executed,
      index: self.events.len()
    };
    self.events.push(Event::RequireTaskEnd(data));
  }

  fn execute_start(&mut self, task: &T) {
    let data = ExecuteStart { task: task.clone(), index: self.events.len() };
    self.events.push(Event::ExecuteStart(data));
  }
  fn execute_end(&mut self, task: &T, output: &T::Output) {
    let data = ExecuteEnd { task: task.clone(), output: output.clone(), index: self.events.len() };
    self.events.push(Event::ExecuteEnd(data));
  }
}

We implement the relevant methods from Tracker and store the build events as Event instances in self.events. When a new build starts, we clear the events.

Now we will add code to inspect the build events. This is quite a bit of code that we will be using in integration testing to test incrementality and soundness. We’ll add in just two steps to keep the tutorial going, and we will use this code in the next section, but feel free to take some time to inspect the code.

First we add some methods to Event to make finding the right event and getting its data easier for the rest of the code. Add the following code to pie/src/tracker/event.rs:

impl<T: Task> Event<T, T::Output> {
  /// Returns `Some(&data)` if this is a [require file end event](Event::RequireFileEnd) for file at `path`, or `None`
  /// otherwise.
  pub fn match_require_file_end(&self, path: impl AsRef<Path>) -> Option<&RequireFileEnd> {
    let path = path.as_ref();
    match self {
      Event::RequireFileEnd(data) if data.path == path => Some(data),
      _ => None,
    }
  }

  /// Returns `Some(&data)` if this is a [require start event](Event::RequireTaskStart) for `task`, or `None` otherwise.
  pub fn match_require_task_start(&self, task: &T) -> Option<&RequireTaskStart<T>> {
    match self {
      Event::RequireTaskStart(data) if data.task == *task => Some(data),
      _ => None,
    }
  }
  /// Returns `Some(&data)` if this is a [require end event](Event::RequireTaskEnd) for `task`, or `None` otherwise.
  pub fn match_require_task_end(&self, task: &T) -> Option<&RequireTaskEnd<T, T::Output>> {
    match self {
      Event::RequireTaskEnd(data) if data.task == *task => Some(data),
      _ => None,
    }
  }

  /// Returns `true` if this is a task execute [start](Event::ExecuteStart) or [end](Event::ExecuteEnd) event.
  pub fn is_execute(&self) -> bool {
    match self {
      Event::ExecuteStart(_) | Event::ExecuteEnd(_) => true,
      _ => false,
    }
  }
  /// Returns `true` if this is an execute [start](Event::ExecuteStart) or [end](Event::ExecuteEnd) event for `task`.
  pub fn is_execute_of(&self, task: &T) -> bool {
    match self {
      Event::ExecuteStart(ExecuteStart { task: t, .. }) |
      Event::ExecuteEnd(ExecuteEnd { task: t, .. }) if t == task => true,
      _ => false,
    }
  }
  /// Returns `Some(&data)` if this is an [execute start event](Event::ExecuteStart) for `task`, or `None` otherwise.
  pub fn match_execute_start(&self, task: &T) -> Option<&ExecuteStart<T>> {
    match self {
      Event::ExecuteStart(data) if data.task == *task => Some(data),
      _ => None,
    }
  }
  /// Returns `Some(&data)` if this is an [execute end event](Event::ExecuteEnd) for `task`, or `None` otherwise.
  pub fn match_execute_end(&self, task: &T) -> Option<&ExecuteEnd<T, T::Output>> {
    match self {
      Event::ExecuteEnd(data) if data.task == *task => Some(data),
      _ => None,
    }
  }
}

These methods check if the current event is a specific kind of event, and return their specific data as Some(data), or None if it is a different kind of event.

Finally, we add methods to EventTracker for inspecting events. Add the following code to pie/src/tracker/event.rs:

impl<T: Task> EventTracker<T, T::Output> {
  /// Returns a slice over all events.
  pub fn slice(&self) -> &[Event<T, T::Output>] {
    &self.events
  }
  /// Returns an iterator over all events.
  pub fn iter(&self) -> impl Iterator<Item=&Event<T, T::Output>> {
    self.events.iter()
  }

  /// Returns `true` if `predicate` returns `true` for any event.
  pub fn any(&self, predicate: impl FnMut(&Event<T, T::Output>) -> bool) -> bool {
    self.iter().any(predicate)
  }
  /// Returns `true` if `predicate` returns `true` for exactly one event.
  pub fn one(&self, predicate: impl FnMut(&&Event<T, T::Output>) -> bool) -> bool {
    self.iter().filter(predicate).count() == 1
  }

  /// Returns `Some(v)` for the first event `e` where `f(e)` returns `Some(v)`, or `None` otherwise.
  pub fn find_map<R>(&self, f: impl FnMut(&Event<T, T::Output>) -> Option<&R>) -> Option<&R> {
    self.iter().find_map(f)
  }


  /// Finds the first [require file end event](Event::RequireFileEnd) for `path` and returns `Some(&data)`, or `None`
  /// otherwise.
  pub fn first_require_file(&self, path: &PathBuf) -> Option<&RequireFileEnd> {
    self.find_map(|e| e.match_require_file_end(path))
  }
  /// Finds the first [require file end event](Event::RequireFileEnd) for `path` and returns `Some(&index)`, or `None`
  /// otherwise.
  pub fn first_require_file_index(&self, path: &PathBuf) -> Option<&usize> {
    self.first_require_file(path).map(|d| &d.index)
  }

  /// Finds the first require [start](Event::RequireTaskStart) and [end](Event::RequireTaskEnd) event for `task` and
  /// returns `Some((&start_data, &end_data))`, or `None` otherwise.
  pub fn first_require_task(&self, task: &T) -> Option<(&RequireTaskStart<T>, &RequireTaskEnd<T, T::Output>)> {
    let start_data = self.find_map(|e| e.match_require_task_start(task));
    let end_data = self.find_map(|e| e.match_require_task_end(task));
    start_data.zip(end_data)
  }
  /// Finds the first require [start](Event::RequireTaskStart) and [end](Event::RequireTaskEnd) event for `task` and
  /// returns `Some(start_data.index..=end_data.index)`, or `None` otherwise.
  pub fn first_require_task_range(&self, task: &T) -> Option<RangeInclusive<usize>> {
    self.first_require_task(task).map(|(s, e)| s.index..=e.index)
  }

  /// Returns `true` if any task was executed.
  pub fn any_execute(&self) -> bool {
    self.any(|e| e.is_execute())
  }
  /// Returns `true` if `task` was executed.
  pub fn any_execute_of(&self, task: &T) -> bool {
    self.any(|e| e.is_execute_of(task))
  }
  /// Returns `true` if `task` was executed exactly once.
  pub fn one_execute_of(&self, task: &T) -> bool {
    self.one(|e| e.match_execute_start(task).is_some())
  }

  /// Finds the first execute [start](Event::ExecuteStart) and [end](Event::ExecuteEnd) event for `task` and returns
  /// `Some((&start_data, &end_data))`, or `None` otherwise.
  pub fn first_execute(&self, task: &T) -> Option<(&ExecuteStart<T>, &ExecuteEnd<T, T::Output>)> {
    let start_data = self.find_map(|e| e.match_execute_start(task));
    let end_data = self.find_map(|e| e.match_execute_end(task));
    start_data.zip(end_data)
  }
  /// Finds the first execute [start](Event::ExecuteStart) and [end](Event::ExecuteEnd) event for `task` and returns
  /// `Some(start_data.index..=end_data.index)`, or `None` otherwise.
  pub fn first_execute_range(&self, task: &T) -> Option<RangeInclusive<usize>> {
    self.first_execute(task).map(|(s, e)| s.index..=e.index)
  }
}

We add several general inspection methods:

  • slice and iter provide raw access to all stored Events,
  • any and one are for checking predicates over all events,
  • find_map for finding the first event given some function, returning the output of that function.

Then we add methods for specific kinds of events, following the general methods. For example, first_require_task finds the first require task start and end events for a task, and return their event data as a tuple. first_require_task_range finds the same events, but returns their indices as a RangeInclusive<usize>.

Check that the code compiles with cargo test.

Implement composite tracker

Currently, we cannot use both EventTracker and WritingTracker at the same time. We want this so that we can check incrementality and soundness, but also look at standard output for debugging, at the same time. Therefore, we will implement a CompositeTracker that forwards build events to 2 trackers.

Add the following code to pie/src/tracker/mod.rs:

/// [`Tracker`] that forwards build events to 2 trackers.
#[derive(Copy, Clone, Debug)]
pub struct CompositeTracker<A1, A2>(pub A1, pub A2);
impl<T: Task, A1: Tracker<T>, A2: Tracker<T>> Tracker<T> for CompositeTracker<A1, A2> {
  fn build_start(&mut self) {
    self.0.build_start();
    self.1.build_start();
  }
  fn build_end(&mut self) {
    self.0.build_end();
    self.1.build_end();
  }

  fn require_file_end(&mut self, dependency: &FileDependency) {
    self.0.require_file_end(dependency);
    self.1.require_file_end(dependency);
  }
  fn require_task_start(&mut self, task: &T, stamper: &OutputStamper) {
    self.0.require_task_start(task, stamper);
    self.1.require_task_start(task, stamper);
  }
  fn require_task_end(&mut self, dependency: &TaskDependency<T, T::Output>, output: &T::Output, was_executed: bool) {
    self.0.require_task_end(dependency, output, was_executed);
    self.1.require_task_end(dependency, output, was_executed);
  }

  fn check_dependency_start(&mut self, dependency: &Dependency<T, T::Output>) {
    self.0.check_dependency_start(dependency);
    self.1.check_dependency_start(dependency);
  }
  fn check_dependency_end(
    &mut self,
    dependency: &Dependency<T, T::Output>,
    inconsistency: Result<Option<&Inconsistency<T::Output>>, &io::Error>
  ) {
    self.0.check_dependency_end(dependency, inconsistency);
    self.1.check_dependency_end(dependency, inconsistency);
  }

  fn execute_start(&mut self, task: &T) {
    self.0.execute_start(task);
    self.1.execute_start(task);
  }
  fn execute_end(&mut self, task: &T, output: &T::Output) {
    self.0.execute_end(task, output);
    self.1.execute_end(task, output);
  }
}

CompositeTracker is a tuple struct containing 2 trackers that implements all tracker methods and forwards them to the 2 contained trackers. Its tuple fields are pub so it can be constructed with CompositeTracker(t1, t2) and the contained trackers can be accessed with c.0 and c.1.

Check that the code compiles with cargo test.

Now that the build event tracking infrastructure is in place, we can start integration testing!

Download source code