File System Watcher Trigger
File System Watcher Trigger
Process files as they appear (e.g., ingest PDFs, images, or logs; run extraction via SK).
Reliable watcher pattern
var fsw = new FileSystemWatcher("/data/inbox")
{
IncludeSubdirectories = false,
NotifyFilter = NotifyFilters.FileName | NotifyFilters.Size | NotifyFilters.LastWrite,
Filter = "*.*",
EnableRaisingEvents = true
};
var queue = Channel<string>.CreateUnbounded();
fsw.Created += (_, e) => queue.Writer.TryWrite(e.FullPath);
fsw.Changed += (_, e) => queue.Writer.TryWrite(e.FullPath);
_ = Task.Run(async () =>
{
var pending = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
await foreach (var path in queue.Reader.ReadAllAsync())
{
pending.Add(path);
await Task.Delay(500); // debounce
foreach (var p in pending.ToArray())
{
if (IsStable(p)) { await ProcessAsync(p); pending.Remove(p); }
}
}
});
- On startup, do a full directory scan to catch missed files.
- In containers, mount volumes and ensure inotify limits are sufficient.
Pros / Cons
- Pros: Near-real-time ETL; simple local integration.
- Cons: Event storms and partial writes; must debounce and verify stability.