70 %

On-Demand (lazy) inputs for incremental computation in salsa with file watching powered by notify in Rust

This is an example application that shows an example integration with Salsa, an incremental computation library, and notify, a cross-platform file-watcher API written in Rust. The application uses on-demand (or lazy) queries to read from the filesystem and invalidate paths on change. It is an expanded version of the example in the salsa book with additional changes to make it work with v0.15.2.

The code

rust

use notify::{watcher, DebouncedEvent, RecommendedWatcher, RecursiveMode, Watcher};
use std::path::{Path, PathBuf};
use std::sync::mpsc::{channel};
use std::sync::{Arc, Mutex};
use std::time::Duration;

#[salsa::query_group(VfsDatabaseStorage)]
trait VfsDatabase: salsa::Database + FileWatcher {
    fn read(&self, path: PathBuf) -> String;
}

trait FileWatcher {
    fn watch(&self, path: &Path);
    fn did_change_file(&mut self, path: &PathBuf);
}

fn read(db: &dyn VfsDatabase, path: PathBuf) -> String {
    db.salsa_runtime()
        .report_synthetic_read(salsa::Durability::LOW);

    db.watch(&path);
    std::fs::read_to_string(&path).unwrap_or_default()
}

#[salsa::database(VfsDatabaseStorage)]
struct MyDatabase {
    storage: salsa::Storage<Self>,
    watcher: Arc<Mutex<RecommendedWatcher>>,
}

impl<'a> salsa::Database for MyDatabase {}

impl FileWatcher for MyDatabase {
    fn watch(&self, path: &Path) {
        // Add a path to be watched. All files and directories at that path and
        // below will be monitored for changes.
        let mut watcher = self.watcher.lock().unwrap();
        watcher.watch(path, RecursiveMode::Recursive).unwrap();
    }
    fn did_change_file(&mut self, path: &PathBuf) {
        ReadQuery.in_db_mut(self).invalidate(&path.to_path_buf());
    }
}

fn main() {
    let (tx, rx) = channel();
    // Create a watcher object, delivering debounced events.
    // The notification back-end is selected based on the platform.
    let mut watcher = Arc::from(Mutex::new(watcher(tx, Duration::from_secs(1)).unwrap()));
    let mut db = MyDatabase {
        watcher,
        storage: salsa::Storage::default(),
    };
    let file_to_watch = Path::new("./test/something.txt");

    db.read(file_to_watch.to_path_buf());
    loop {
        match rx.recv() {
            Ok(event) => {
                println!("{:?}", event);
                match event {
                    DebouncedEvent::Write(filepath_buf) => {
                        db.did_change_file(&filepath_buf);
                        db.read(Path::new("./test/something2.txt").to_path_buf());
                    }
                    _ => {}
                }
            }
            Err(e) => println!("watch error: {:?}", e),
        }
    }
}

Explanation

fn main

In the main function we create a sender and a receiver channel to bootstrap our file watcher with.

rust

fn main() {
    let (tx, rx) = channel();

Then we bootstrap our watcher using the Sender<DebouncedEvent>. It is important that we wrap this in Arc and Mutex because otherwise the mutation requirement will bubble up to our read query, which can't handle it (it requires the first argument to be &self).

rust

// Create a watcher object, delivering debounced events.
    // The notification back-end is selected based on the platform.
    let mut watcher = Arc::from(Mutex::new(watcher(tx, Duration::from_secs(1)).unwrap()));

We bootstrap the Salsa database with the watcher and our storage, choosing to use the default value for the salsa storage.

rust

let mut db = MyDatabase {
        watcher,
        storage: salsa::Storage::default(),
    };

Then the main function sets up a file to watch that comes from "somewhere else" (exercise left to the reader).

rust

let file_to_watch = Path::new("./test/something.txt");

We use the read query to read the file in and set up the watch on that file

rust

db.read(file_to_watch.to_path_buf());

and finally we loop forever, pulling file watcher event values off the Receiver<DebouncedEvent>. Note that we've specified two files. One is in the test directory named something.txt and is set up earlier. The next is only set up after the original something.txt is changed. This shows usage of the read query again.

rust

loop {
        match rx.recv() {
            Ok(event) => {
                println!("{:?}", event);
                match event {
                    DebouncedEvent::Write(filepath_buf) => {
                        db.did_change_file(&filepath_buf);
                        db.read(Path::new("./test/something2.txt").to_path_buf());
                    }
                    _ => {}
                }
            }
            Err(e) => println!("watch error: {:?}", e),
        }
    }
}

traits and impls

In the rest of the program, we specify that our VfsDatabase must also implement the Supertraits salsa::Database and FileWatcher (we own FileWatcher and salsa owns the salsa::Database).

rust

#[salsa::query_group(VfsDatabaseStorage)]
trait VfsDatabase: salsa::Database + FileWatcher {
    fn read(&self, path: PathBuf) -> String;
}

Our FileWatcher trait requires the implementation of a watch function and a did_change_file function. Only did_change_file requires a mutable reference to the db.

rust

trait FileWatcher {
    fn watch(&self, path: &Path);
    fn did_change_file(&mut self, path: &PathBuf);
}

read is the center of this example. We set up a query that takes a path as a key and returns a String. The salsa runtime thinks this is a HIGH durability action by default, so we override that with a LOW durability.

Then we .watch the relevant path, which triggers the watcher, and return the contents of the file.

rust

fn read(db: &dyn VfsDatabase, path: PathBuf) -> String {
    db.salsa_runtime()
        .report_synthetic_read(salsa::Durability::LOW);

    db.watch(&path);
    std::fs::read_to_string(&path).unwrap_or_default()
}

The database types also need to be set up. This is where we specify the Arc<Mutex<>> that allows us to implement watch.

rust

#[salsa::database(VfsDatabaseStorage)]
struct MyDatabase {
    storage: salsa::Storage<Self>,
    watcher: Arc<Mutex<RecommendedWatcher>>,
}

impl<'a> salsa::Database for MyDatabase {}

and finally, we implement FileWatcher, which pulls the watcher out of the Mutex using lock and watches the additional path.

did_change_file is used as a mechanism to invalidate the path key for the ReadQuery we set up earlier. We need a mutable db for this.

rust

impl FileWatcher for MyDatabase {
    fn watch(&self, path: &Path) {
        // Add a path to be watched. All files and directories at that path and
        // below will be monitored for changes.
        let mut watcher = self.watcher.lock().unwrap();
        watcher.watch(path, RecursiveMode::Recursive).unwrap();
    }
    fn did_change_file(&mut self, path: &PathBuf) {
        ReadQuery.in_db_mut(self).invalidate(&path.to_path_buf());
    }
}