filed
Job queue using FUSE
git clone git://mccd.space/filed
| Log | Files | Refs | README | LICENSE |
commit a86f4a3c55e164499c39a42c39ff0f8bdfa67a39 parent d9731d2ef169b058023f9c4e0c875d1c15332a26 Author: Marc <marc@coquand.email> Date: Mon, 29 Jun 2026 15:55:37 +0200 * Diffstat:
| M | README.md | | | 49 | +++++++++++++++++++++++++++++++++++++------------ |
1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md @@ -1,14 +1,21 @@ # File d'attente -*File d'attente* (queue in French) is a concurrent, file-based job queue, written in Go. +*File d'attente* (queue in French) is a concurrent, file-based job +queue, written in Go. -File d'attente uses files and directories for queue manipulation. Create a job with "`printf cmd > /pending/$id`", view running jobs with "`ls /active`", and restart a failed job with "`mv /failed/$id /pending`". +File d'attente uses files and directories for queue manipulation. +Create a job with "`printf cmd > /pending/$id`", view running jobs +with "`ls /active`", and restart a failed job with "`mv /failed/$id +/pending`". -The tool is intended for single-server workloads, as a companion queue to another application. File d'attente comes with sandboxing, automatic retries, timeout, and backoff built-in. +The tool is intended for single-server workloads, as a companion queue +to another application. File d'attente comes with sandboxing, +automatic retries, timeout, and backoff built-in. ## Installation -File d'attente is built in Go and depends on sqlite and fuse (make sure fusermount is available in path). +File d'attente is built in Go and depends on sqlite and fuse (make +sure fusermount is available in path). ```sh $ git clone https://sr.ht/~marcc/filed/ @@ -30,18 +37,24 @@ done ## Getting started -It is recommended to read the [man pages] for more complete documentation and security considerations, but below is a small example to get you started. +It is recommended to read the [man pages] for more complete +documentation and security considerations, but below is a small +example to get you started. -`filed` requires a job directory and a state file location (defaulting to `XDG_DATA_HOME`). Afterward, you can start the daemon: +`filed` requires a job directory and a state file location (defaulting +to `XDG_DATA_HOME`). Afterward, you can start the daemon: ```sh $ mkdir /tmp/filed-jobs $ filed -rof "/usr/bin/echo" -ro "/lib" /tmp/filed-jobs ``` -`filed` mounts the directory `filed-jobs` and exposes a few files and directories. With the above script, each job will launch in a sandboxed-mode and only have access to `echo` and `lib`. +`filed` mounts the directory `filed-jobs` and exposes a few files and +directories. With the above script, each job will launch in a +sandboxed-mode and only have access to `echo` and `lib`. -A job can then be added by creating a file in the newly available pending directory: +A job can then be added by creating a file in the newly available +pending directory: ```sh $ printf "echo 'hello world'" > /tmp/filed-jobs/pending/1 @@ -55,7 +68,8 @@ $ cat /tmp/filed-jobs/complete/1 hello world ``` -By default, a job retries 3 times, and if unsuccessful, gets moved to `/failed`. You can inspect the logs to see what went wrong: +By default, a job retries 3 times, and if unsuccessful, gets moved to +`/failed`. You can inspect the logs to see what went wrong: ```sh $ printf "ech this-will-fail" > /tmp/filed-jobs/pending/2 @@ -90,9 +104,19 @@ Available in the manpages: ## Design & Motivation -I wanted to create a queue that would be easy to use for self-hosted web applications, that could be used by any programming language. I also wanted to make it easy for admins to understand why a job fails, and to rerun jobs if there is an error. +I wanted to create a queue that would be easy to use for self-hosted +web applications, that could be used by any programming language. I +also wanted to make it easy for admins to understand why a job fails, +and to rerun jobs if there is an error. -I was inspired by 9p, and files proved to be a great abstraction since directories model state transitions quite well. File d'attente makes it very easy to inspect the state, without needing to build an admin portal with separate sign in. Instead, all admin operations can be done by just SSHing into the server, and the operations for manipulating, securing and automating the system become very intuitive. The source code can then be very slimmed down, while still packing a lot of features. +I was inspired by 9p, and files proved to be a great abstraction since +directories model state transitions quite well. File d'attente makes +it very easy to inspect the state, without needing to build an admin +portal with separate sign in. Instead, all admin operations can be +done by just SSHing into the server, and the operations for +manipulating, securing and automating the system become very +intuitive. The source code can then be very slimmed down, while still +packing a lot of features. ## TODO @@ -119,7 +143,8 @@ bugs/patches can be submitted by email to ~marcc/public-inbox@lists.sr.ht ## Status -File d'attente is tested, but not battle-tested. There are probably quite a few warts and inefficiencies. +File d'attente is tested, but not battle-tested. There are probably +quite a few warts and inefficiencies. ## Alternatives