filed
Job queue using FUSE
git clone git://mccd.space/filed
| Log | Files | Refs | README | LICENSE |
commit 0378369b3b7709033e8e6ce888c332d60c71789e parent 961ef95ca648cd0dc617c1b5847552a6fe0c7282 Author: Marc Coquand <marc@coquand.email> Date: Tue, 16 Dec 2025 16:34:36 +0100 Docs Diffstat:
| M | README.md | | | 5 | +++-- |
| M | filed.1.scd | | | 33 | ++++++++++++++++++--------------- |
2 files changed, 21 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md @@ -98,7 +98,8 @@ I've tried a few other queue tools: sqs/sns, rabbitmq, bull, systemd-run. The fi - [ ] "Landlock"-mode, or sandboxed jobs - Requires a design - [ ] A reusable systemd unit file - [ ] A reusable openrc unit file -- [ ] Notification on failure. Unfortunately [inotify does not work with fuse] +- [ ] Notification on failure. Unfortunately [inotify does not work with fuse], which would have been elegant otherwise. +- [ ] Notify forget and other updates. ## Alternatives @@ -112,4 +113,4 @@ I've tried a few other queue tools: sqs/sns, rabbitmq, bull, systemd-run. The fi [bull]: https://www.npmjs.com/package/bull [man pages]: https://git.sr.ht/~marcc/filed/tree/main/item/filed.1.scd [scdoc]: https://git.sr.ht/~sircmpwn/scdoc -[inotify does not work with fuse]: https://github.com/libfuse/libfuse/wiki/Fsnotify-and-FUSE +[inotify does not work with fuse]: https://github.com/bazil/fuse/issues/188 diff --git a/filed.1.scd b/filed.1.scd @@ -10,32 +10,33 @@ filed - queue jobs utility # DESCRIPTION -filed (file d'attente) is an inspectable job queue that operates on files -with retries. It mounts a directory to _mountpoint_, which is where the user -can add and inspect jobs. +filed (file d'attente) is an inspectable job queue that operates on +files. It mounts a directory _mountpoint_, which is where the user can add +and inspect jobs. filed exposes 4 directories to _mountpoint_, where each directory contains zero or more _jobs_. Job names must be unique across all four directories. The directories are: - *pending* - jobs to be run. To create a new job, create a file - here with the command to run. + */pending* - jobs to be run. To create a new job, create a file + here with the command to run. The filename is the ID, and the content + is the command that will be executed with sh. - *active* - currently running jobs. It is possible to access logs of + */active* - currently running jobs. It is possible to access logs of the running jobs by inspecting the files. - *failed* - jobs that exceeded retry count. You can retry a job by + */failed* - jobs that exceeded retry count. You can retry a job by moving them back to pending. You can also safely remove jobs here. - *complete* - jobs that succeeded, with content being the job + */complete* - jobs that succeeded, with content being the job output. Jobs here can safely be removed. filed exposes 2 files: - *new-id* contains a short unique id that can be sampled for job + */new-id* contains a short unique id that can be sampled for job name entropy. The id is guaranteed to be unique at creation. - *config.json* provides various settings. Changes made to this file + */config.json* provides various settings. Changes made to this file are applied immediately. See *CONFIGURATION* for full list of options. # SECURITY @@ -52,8 +53,8 @@ principle of least access. Importantly, the system is intended for only trusted scripts: the job user has access to the state, and is thus able to rewrite access rights. It is -recommended for the running scripts to use _namespaces(7)_ or _Landlock(7)_ -to drop privileges. More security features are coming in the future. +recommended for the running scripts to use _namespaces(7)_ or _Landlock(7)_ to +drop further drop privileges. Another aspect to be aware of is that File d'attente stores logs of all jobs. Care should be taken to ensure that no secrets are printed. @@ -83,7 +84,7 @@ higher than 20. ## Backoff mult and backoff base -The time to wait before retrying. The formula is +The time in seconds to wait before retrying. The formula used is: ``` base * mult^attempts @@ -91,7 +92,7 @@ The time to wait before retrying. The formula is ## Timeout -Time before the job will be killed by a signal. +Time (in seconds) before the job will be killed by a signal. # ENVIRONMENT @@ -117,7 +118,9 @@ Inspect a currently running job: # SEE ALSO -Periodic jobs can be set up using _CRON(8)_. +- Periodic jobs can be set up using _CRON(8)_. +- Monitoring failures can be done with _WATCH(1)_ +- Limiting job privileges can be done with _bwrap(1)_ or _landrun_ # LIMITATIONS