filed

Job queue using FUSE

git clone git://mccd.space/filed

commit dce7517b4bae8ccfaa8b1c64b8bfaebac1ca10f7
parent 51fc05f55c0c94a274182364a446b88f20438202
Author: Marc Coquand <marc@coquand.email>
Date:   Thu, 18 Dec 2025 09:13:17 +0100

Docs should be 5 - filesystems

Diffstat:

M.gitignore | 2+-


MREADME.md | 4++--


Afiled.5.scd | 136+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


3 files changed, 139 insertions(+), 3 deletions(-)
diff --git a/.gitignore b/.gitignore
@@ -1,2 +1,2 @@
 filed
-filed.*
+filed.[1-8]
diff --git a/README.md b/README.md
@@ -20,8 +20,8 @@ $ go install
 To build the docs you need [scdoc]
 
 ```sh
-$ scdoc < filed.8.scd > filed.8
-# mv filed.8 /usr/local/man/man1
+$ scdoc < filed.5.scd > filed.5
+# mv filed.5 /usr/local/man/man1
 ```
 
 ## Getting started
diff --git a/filed.5.scd b/filed.5.scd
@@ -0,0 +1,136 @@
+FILED(8)
+
+# NAME
+
+filed - queue jobs utility
+
+# SYNOPSIS
+
+*filed* _mountpoint_
+
+# DESCRIPTION
+
+filed (file d'attente) is an inspectable job queue that operates on
+files. It mounts a directory _mountpoint_, which is where the user can add
+and inspect jobs.
+
+filed exposes 4 directories to _mountpoint_, where each directory contains
+zero or more _jobs_. Job names must be unique across all four directories. The
+directories are:
+
+	*/pending* - jobs to be run. To create a new job, create a file
+	here with the command to run. The filename is the ID, and the content
+	is the command that will be executed with sh.
+
+	*/active* - currently running jobs. It is possible to access logs of
+	the running jobs by inspecting the files. It is also possible to 
+	remove an active job, which will kill the process.
+
+	*/failed* - jobs that exceeded retry count. You can retry a job by
+	moving them back to pending. You can also safely remove jobs here.
+
+	*/complete* - jobs that succeeded, with content being the job
+	output. Jobs here can safely be removed.
+
+filed exposes 2 files: 
+
+	*/new-id* contains a short unique id that can be sampled for job
+	name entropy. The id is guaranteed to be unique at creation.
+
+	*/config.json* provides various settings. Changes made to this file 
+	are applied immediately. See *CONFIGURATION* for full list of options.
+
+# SECURITY
+
+All commands are executed using the same user as the main process. Access
+is configured the same way you configure other files on Unix systems.
+
+By default, all write operations are allowed by the executing user, and all
+read operations by the executing user's group. All other users have zero 
+access to the system.
+
+It is recommended for the admin to update the users and group to apply
+principle of least access. 
+
+Importantly, the system is intended for only trusted scripts: the job user
+has access to the state, and is thus able to rewrite access rights. It is
+recommended for the running scripts to use _namespaces(7)_ or _Landlock(7)_ to
+drop further drop privileges. 
+
+Another aspect to be aware of is that File d'attente stores logs of all jobs.
+Care should be taken to ensure that no secrets are printed.
+
+Access rights can be modified using _CHOWN(1)_ and _CHMOD(1)_. 
+
+# MAINTENANCE
+
+filed never removes logs of completed and failed jobs. It is therefore recommended
+that the admin adds a cron job to clean up old jobs. The cron job could
+look like this, for example:
+
+	```
+	0 2 * * * find /path/to/filed/{complete,failed} -type f -mtime +7 -delete
+	```
+
+# CONFIGURATION
+
+## Max retries
+
+Maximum amount of retries before moving the job to failed. 
+
+## Max job count
+
+Maximum amount of concurrent jobs. It is recommended to not set this much
+higher than 20.
+
+## Backoff mult and backoff base
+
+The time in seconds to wait before retrying. The formula used is:
+
+	```
+	base * mult^attempts
+	```
+
+## Timeout 
+
+Time (in seconds) before the job will be killed by a signal.
+
+# ENVIRONMENT
+
+## FILED_STATE_FILE
+
+Path to store sqlite state. File is created if it does not
+exist. Defaults to $XDG_DATA_HOME/filed.db, or exit(1) if XDG_DATA_HOME is unset.
+
+# EXAMPLE
+
+Create a new job with a unique id, that echoes hello world:
+
+	$ printf "echo helloworld" > "/var/filed/pending/$(< /var/filed/new-id)"
+
+Retry a job that failed:
+
+	$ mv /var/filed/failed/myjob /var/filed/pending
+
+Inspect a currently running job:
+
+	$ cat /var/filed/active/myjob
+
+
+# SEE ALSO
+
+- Periodic jobs can be set up using _CRON(8)_.
+- Monitoring failures can be done with _WATCH(1)_
+- Limiting job privileges can be done with _bwrap(1)_ or _landrun_
+
+# LIMITATIONS
+
+File d'attente does not work with _inotify(7)_.
+
+# AUTHORS
+
+Maintained by Marc Coquand <marc@coquand.email>. Up-to-date sources can be
+found at https://git.sr.ht/~marcc/filed and bugs/patches can be submitted by
+email to ~marcc/public-inbox@lists.sr.ht.
+
+