filed
Job queue using FUSE
git clone git://mccd.space/filedcommit 0378369b3b7709033e8e6ce888c332d60c71789e
parent 961ef95ca648cd0dc617c1b5847552a6fe0c7282
Author: Marc Coquand <marc@coquand.email>
Date: Tue, 16 Dec 2025 16:34:36 +0100
Docs
Diffstat:
2 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/README.md b/README.md
@@ -98,7 +98,8 @@ I've tried a few other queue tools: sqs/sns, rabbitmq, bull, systemd-run. The fi
- [ ] "Landlock"-mode, or sandboxed jobs - Requires a design
- [ ] A reusable systemd unit file
- [ ] A reusable openrc unit file
-- [ ] Notification on failure. Unfortunately [inotify does not work with fuse]
+- [ ] Notification on failure. Unfortunately [inotify does not work with fuse], which would have been elegant otherwise.
+- [ ] Notify forget and other updates.
## Alternatives
@@ -112,4 +113,4 @@ I've tried a few other queue tools: sqs/sns, rabbitmq, bull, systemd-run. The fi
[bull]: https://www.npmjs.com/package/bull
[man pages]: https://git.sr.ht/~marcc/filed/tree/main/item/filed.1.scd
[scdoc]: https://git.sr.ht/~sircmpwn/scdoc
-[inotify does not work with fuse]: https://github.com/libfuse/libfuse/wiki/Fsnotify-and-FUSE
+[inotify does not work with fuse]: https://github.com/bazil/fuse/issues/188
diff --git a/filed.1.scd b/filed.1.scd
@@ -10,32 +10,33 @@ filed - queue jobs utility
# DESCRIPTION
-filed (file d'attente) is an inspectable job queue that operates on files
-with retries. It mounts a directory to _mountpoint_, which is where the user
-can add and inspect jobs.
+filed (file d'attente) is an inspectable job queue that operates on
+files. It mounts a directory _mountpoint_, which is where the user can add
+and inspect jobs.
filed exposes 4 directories to _mountpoint_, where each directory contains
zero or more _jobs_. Job names must be unique across all four directories. The
directories are:
- *pending* - jobs to be run. To create a new job, create a file
- here with the command to run.
+ */pending* - jobs to be run. To create a new job, create a file
+ here with the command to run. The filename is the ID, and the content
+ is the command that will be executed with sh.
- *active* - currently running jobs. It is possible to access logs of
+ */active* - currently running jobs. It is possible to access logs of
the running jobs by inspecting the files.
- *failed* - jobs that exceeded retry count. You can retry a job by
+ */failed* - jobs that exceeded retry count. You can retry a job by
moving them back to pending. You can also safely remove jobs here.
- *complete* - jobs that succeeded, with content being the job
+ */complete* - jobs that succeeded, with content being the job
output. Jobs here can safely be removed.
filed exposes 2 files:
- *new-id* contains a short unique id that can be sampled for job
+ */new-id* contains a short unique id that can be sampled for job
name entropy. The id is guaranteed to be unique at creation.
- *config.json* provides various settings. Changes made to this file
+ */config.json* provides various settings. Changes made to this file
are applied immediately. See *CONFIGURATION* for full list of options.
# SECURITY
@@ -52,8 +53,8 @@ principle of least access.
Importantly, the system is intended for only trusted scripts: the job user
has access to the state, and is thus able to rewrite access rights. It is
-recommended for the running scripts to use _namespaces(7)_ or _Landlock(7)_
-to drop privileges. More security features are coming in the future.
+recommended for the running scripts to use _namespaces(7)_ or _Landlock(7)_ to
+drop further drop privileges.
Another aspect to be aware of is that File d'attente stores logs of all jobs.
Care should be taken to ensure that no secrets are printed.
@@ -83,7 +84,7 @@ higher than 20.
## Backoff mult and backoff base
-The time to wait before retrying. The formula is
+The time in seconds to wait before retrying. The formula used is:
```
base * mult^attempts
@@ -91,7 +92,7 @@ The time to wait before retrying. The formula is
## Timeout
-Time before the job will be killed by a signal.
+Time (in seconds) before the job will be killed by a signal.
# ENVIRONMENT
@@ -117,7 +118,9 @@ Inspect a currently running job:
# SEE ALSO
-Periodic jobs can be set up using _CRON(8)_.
+- Periodic jobs can be set up using _CRON(8)_.
+- Monitoring failures can be done with _WATCH(1)_
+- Limiting job privileges can be done with _bwrap(1)_ or _landrun_
# LIMITATIONS