Technical guides on jneidel

Completion of hugo links in Emacs

web@jneidel.com (Jonathan Neidel) — Fri, 13 Feb 2026 00:00:00 +0000

When writing an article in hugo I want to link to previous articles I published. In markdown I can use:

[Hugo link completions](dev/hugo-links)

Which will resolve to either:

content/dev/hugo-links.md or
content/dev/hugo-links/index.md.

As my site grew, so did I grow tired of trying to remember or copy-pasting these links. Completion to the rescue. Fully integrated in your favorite editor: Emacs.

Hugo provides the completion candidates; the content files it knows about:

#! /bin/sh

# matches NAME/index.md files, excludes _index.md and NAME.md
cd $WEB_HOME
hugo list all --noBuildLock | tail -n+2 | cut -d, -f1 | grep -v "_index" | cut -d/ -f2- |
 rev | cut -d/ -f2- | rev

Multi-language script

This script supports both English (.md) and German (.de.md) content files.

#! /bin/sh

lang=$1

list_files() {
 # excludes _index.md and NAME.md files
 cd $WEB_HOME
 case $lang in
 en)
 hugo list all --noBuildLock | tail -n+2 | cut -d, -f1 | grep -v "_index" | cut -d/ -f2- |
 grep -v ".de.md" | rev | cut -d/ -f2- | rev
 break;;
 de)
 hugo list all --noBuildLock | tail -n+2 | cut -d, -f1 | grep -v "_index" | cut -d/ -f2- |
 grep ".de.md" | rev | cut -d/ -f2- | rev
 break;;
 -|both)
 hugo list all --noBuildLock | tail -n+2 | cut -d, -f1 | grep -v "_index" | cut -d/ -f2- |
 rev | cut -d/ -f2- | rev | sort | uniq
 break;;
 *)
 echo "Invalid language"
 exit 1
 esac
}

You can find an expanded version of the script in my dotfiles.

The elisp function for this completion can be as simple as:

(defun hugo-md-web-complete ()
 "Completion for links to my website."
 (interactive)
 (let* ((articles (shell-command-to-string "pick-hugo-article")) ; the shell script from above
 (link (completing-read "link: " (split-string articles "\n" t) nil t)))
 (insert (format "[%s](%s)" (read-string "link text: ") link))))
;; inserts: [link text](link)

Now, I use multiple languages and ox-hugo (hugo org-mode content files), so my solution needed to account for that.

Local hugo links are not very useful outside of the hugo directory. Instead the script should output a complete website link: “https://jneidel.com/…”

(defun jn/org-web-complete (lang)
 "Completion for links to my website."
 (let* ((articles (shell-command-to-string (format "pick-hugo-article %s" lang)))
 (link (completing-read "web: " (split-string articles "\n" t) nil t))
 (current-file (when buffer-file-name
 (file-truename buffer-file-name)))
 (is-editing-website (and
 current-file
 (string-prefix-p (substitute-in-file-name "$WEB_HOME") current-file)))
 (website-root (if (equal lang "de")
 "https://jneidel.de/"
 "https://jneidel.com/")))
 (if is-editing-website
 link
 (concat website-root link))))

(defun jn/org-web-de-complete ()
 (jn/org-web-complete "de"))
(defun jn/org-web-en-complete ()
 (jn/org-web-complete "en"))

(with-eval-after-load 'org
 (org-link-set-parameters "de" :complete #'jn/org-web-de-complete)
 (org-link-set-parameters "en" :complete #'jn/org-web-en-complete))

(Latest version in my dotemacs.)

At the end I am registering the completions to be called through org-insert-link using de: or en:. This integration is great. No new keybinding are needed and I can use it through my full-featured org-insert-link-dwim.

That’s it :) Have a great day!

References #

See this post for an alternative approach using read-file-name

Automatic Backups of my GitHub Repositories

web@jneidel.com (Jonathan Neidel) — Sat, 25 Oct 2025 00:00:00 +0000

I have all of my code on GitHub. Microsoft could at any time decide to just ban me and I would loose access to it all. It unlikely, but it could happen.

There are many obvious solutions to this:

take my code to a more trustworthy vendor
self-host my git
backup/mirror github repositories

For the moment I went with the backup option.

Requirements #

Backups should happen automatically, for every repo in my account. I don’t want to think about it.

Setup #

I picked the tool gickup for the job. I have it running on my homeserver, so the repos are part of it’s backup scheme.

I installed the latest release:

wget https://github.com/cooperspencer/gickup/releases/download/v0.10.39/gickup_0.10.39_linux_386.tar.gz
tar -xzf gickup_0.10.39_linux_386.tar.gz

Created and tested the config (docs):

# gickup.yml
source:
 github:
 - user: jneidel
 token: github_pat_...
 issues: true
 filter:
 excludeforks: true
destination:
 local:
 - path: /root/backup/github-repos
 bare: true

Invoke with gickup gickup.yml.

For authentication I created a personal access token with the permissions shown below. I tried using my git ssh key, which worked for cloning, but ran into API rate limits. The token is more secure also, since it’s read-only.

I want this to run once a week, so I setup execution with cron: 0 16 * * 0.

Conclusions #

The one-time setup was easier than expected. It does not require maintenance so I don’t have to think about this again.

Consistent Keybindings Across Linux and macOS

web@jneidel.com (Jonathan Neidel) — Fri, 26 Sep 2025 00:00:00 +0000

This article describes how I set up consistent keybindings between my Linux and macOS devices. Basically, I adjust the macOS bindings with Karabiner Elements to mirror the Linux bindings. I detail the approach and include my configs so you can replicate it.

Why setup consistent keybindings? #

I have two machines with different operating system. Linux on my personal computer and MacOS on my work machine.

For copy-paste and browser navigation there are many differences in the bindings not addressable at an OS/application level. Even if you develop individual muscle memory for both sets of keybindings, a switch will always trip you up or slow you down. Especially so if you connect the same external keyboard to both devices. Ideally an external keyboard should work the same with both computers. Copy-paste on the same key. Open new browser tab on the same key.

Solution concepts #

A software solution is the clean way. Options were 1) keyboard specific firmware (QMK/Vial) or 2) keycode translation software like Xmodmap and Karabiner-Elements) or a combination of 1) and 2).

The problem at hand is not just a “turn caps lock into control” kind of situation. Specifically the macOS command key needs to be split up by capability.

An example: to copy you have macOSs cmd-c as ctrl-c on Linux, but to navigate to the first browser tab you use cmd-1 and alt-1 respectively. On top of that I want to populate the now empty cmd-1 to bring me to the first workspace of my window manager.

1) Firmware: separate keyboard by OS #

Reddit had some ideas about how to do this, namely building different layers for each OS into the keyboards firmware and changing the default layer (DF()) depending on the environment.

It would work, but the downsides of the solution would be:

mental overhead (“Am I in the right layer for this OS?”)
keyboard modification need to be (manually) synchronized between the layers
does not apply to the laptop-internal keyboard
capabilities might need their own keys, think OS-agnostic copy and paste buttons

Another thing I came up with was programming the left/right microcontroller differently and using one for Linux and the other for macOS. This would have similar consequences as the approach above.

2) Translate keys on the Mac #

Karabiner Elements is capable of doing the translations described above, e.g.:

“alt-1” triggers “cmd-1”
“cmd-1” triggers something to go to workspace 1

Karabiner Elements elegantly solves the problem at hand, without any of the downsides presented by the keyboard firmware solution.

The Linux machine stays the same and acts as the source to truth to be replicated.

Solution implementation #

With Karabiner Elements installed on the Mac, let’s head to “Complex Modifications”.

Copy-paste #

Most obviously I want copy, paste and cut to work (ctrl-c, ctrl-v, ctrl-x.)

Bascially that would look like the below. Translate the keypress “ctrl-c” to “cmd-c”.

 {
 "from": {
 "key_code": "c",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "c",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 }

If you look at my whole config below, you will see that I do more than that. “ctrl-c” is used to interrupt the current process in the terminal. To preserve it in the terminal I add an exception and an additional rule that adds: “ctrl-shift-c” to “cmd-c” for inside a terminal (relevant docs.)

Also the modifiers include a left/right prefix, so I add a copy that triggers for all the modifiers combinations I use.

Full config for copy-paste functionality

{
 "description": "Enable copy-paste on ctrl (shift-ctrl in the terminal)",
 "manipulators": [
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "c",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "c",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "c",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "c",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "c",
 "modifiers": { "mandatory": ["right_control", "left_shift"] }
 },
 "to": [
 {
 "key_code": "c",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "c",
 "modifiers": { "mandatory": ["left_control", "left_shift"] }
 },
 "to": [
 {
 "key_code": "c",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "v",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "v",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": { "mandatory": ["right_control", "left_shift"] }
 },
 "to": [
 {
 "key_code": "v",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": { "mandatory": ["left_control", "left_shift"] }
 },
 "to": [
 {
 "key_code": "v",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "x",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "x",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "x",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "x",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 }
 ]
}

One of my most widely used applications, the browser, has terrible keybinding customization. Both chromium and firefox have their built-in keybindings (which differ between OSs) that are pratically unchangable. At least they are consistent between the two browser families :slightly_smiling_face:

I translate:

Reload (“ctrl-r”)
New tab (“ctrl-t”)
Close tab (“ctrl-w”)
Focus address bar (“ctrl-l”)
Focus n-th tab (“alt-1,2,3,…”)

Full config for browser navigation

{
 "description": "Match Linux browser navigation",
 "manipulators": [
 {
 "from": {
 "key_code": "t",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "t",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "t",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "t",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "t",
 "modifiers": { "mandatory": ["right_control", "left_shift"] }
 },
 "to": [
 {
 "key_code": "t",
 "modifiers": ["left_command", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "t",
 "modifiers": { "mandatory": ["left_control", "left_shift"] }
 },
 "to": [
 {
 "key_code": "t",
 "modifiers": ["left_command", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "w",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "w",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "w",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "w",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "l",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "l",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "r",
 "modifiers": { "mandatory": ["left_control"] }
 },
 "to": [
 {
 "key_code": "r",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "r",
 "modifiers": { "mandatory": ["right_control"] }
 },
 "to": [
 {
 "key_code": "r",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "1",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "1",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "2",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "2",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "3",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "4",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "5",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "6",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "6",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "7",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "7",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "8",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "8",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "9",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "9",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "0",
 "modifiers": { "mandatory": ["left_option"] }
 },
 "to": [
 {
 "key_code": "0",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 }
 ]
}

Window management #

I use aerospace for window management on MacOS. I wanted to mirror my i3 config as closely as possible. The keycode sent by the macOS command key is the same as super on Linux. After remapping the many of the macOS key combinations that utilize command, we can now add window management commands.

command is first converted to “meh” (cmd-alt-ctrl"), which I then use in my ~/.config/aerospace/aerospace.toml to bind to actions:

cmd-ctrl-alt-1 = 'workspace 1'
cmd-ctrl-alt-2 = 'workspace 2'
# …

cmd-ctrl-alt-shift-1 = 'move-node-to-workspace 1'
cmd-ctrl-alt-shift-2 = 'move-node-to-workspace 2'
# …

Full config for window management

{
 "description": "Map aerospace bindings to cmd (via meh)",
 "manipulators": [
 {
 "from": {
 "key_code": "1",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "1",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "1",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "1",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "2",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "2",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "2",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "2",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "3",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "3",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "4",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "4",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "5",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "5",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "6",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "6",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "6",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "6",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "7",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "7",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "7",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "7",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "8",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "8",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "8",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "8",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "9",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "9",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "9",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "9",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "0",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "0",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "0",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "0",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "p",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": [
 "left_command",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "p",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": [
 "left_command",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "p",
 "modifiers": {
 "mandatory": [
 "command",
 "control"
 ]
 }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": [
 "left_command",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "h",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "h",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "j",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "j",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "k",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "k",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "l",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "h",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "h",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "j",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "j",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "k",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "k",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "l",
 "modifiers": {
 "mandatory": [
 "command",
 "shift"
 ]
 }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control",
 "left_shift"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "f",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "f",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "s",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "s",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "w",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "w",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "e",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "e",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "r",
 "modifiers": {
 "mandatory": [
 "command"
 ]
 }
 },
 "to": [
 {
 "key_code": "r",
 "modifiers": [
 "left_command",
 "left_option",
 "left_control"
 ]
 }
 ],
 "type": "basic"
 }
 ]
}

Restrict translations to external keyboards #

The configurations I supplied above apply the translation not only to any externally plugged-in keyboards, but also to the internal laptop keyboard.

If you only want it to affect external ones, you can add this restrictive condition to every rule:

"conditions": [
 {
 "identifiers": [
 { "is_built_in_keyboard": true },
 { "vendor_id": 76 }
 ],
 "type": "device_unless"
 }
]

The next step #

One does not have to stop there. I use Emacs, which has keybindings like ctrl-y to paste that predate the convention of ctrl-v. That creates another mental switch. Emacs and other apps. This difference can be eliminated with the same solution. Using Emacs bindings everywhere.

This is my karabiner config for that on macOS (though it can be much better):

Karabiner config for Emacs bindings everywhere

{
 "global": { "check_for_updates_on_startup": false },
 "profiles": [
 {
 "complex_modifications": {
 "rules": [
 {
 "description": "Emacs bindings: Browser only",
 "manipulators": [
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "q",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "w",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 }
 ]
 },
 {
 "description": "Emacs bindings: common (cmd+f, c, v, x; not in Emacs/Terminal)",
 "manipulators": [
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "b",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "left_arrow" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "f",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "right_arrow" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "p",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "up_arrow" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "n",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "down_arrow" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "e",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "right_arrow",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": {
 "mandatory": ["option"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "page_up" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "page_down" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "w",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "x",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "w",
 "modifiers": {
 "mandatory": ["option"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "c",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "z",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "v",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "d",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "delete_forward" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "k",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "right_arrow",
 "modifiers": ["command", "shift"]
 },
 {
 "key_code": "x",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "s",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [
 {
 "key_code": "f",
 "modifiers": ["command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "g",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "escape" }],
 "type": "basic"
 }
 ]
 },
 {
 "description": "Emacs Bindings: Terminal only",
 "manipulators": [
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": {
 "mandatory": ["option"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "page_up" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "v",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "page_down" }],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "g",
 "modifiers": {
 "mandatory": ["control"],
 "optional": ["any"]
 }
 },
 "to": [{ "key_code": "escape" }],
 "type": "basic"
 }
 ]
 },
 {
 "description": "Map aerospace bindings to cmd (via meh)",
 "manipulators": [
 {
 "from": {
 "key_code": "1",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "1",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "1",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "1",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "2",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "2",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "2",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "2",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "3",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "3",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "4",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "4",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "5",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "5",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "6",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "6",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "6",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "6",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "7",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "7",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "7",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "7",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "8",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "8",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "8",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "8",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "9",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "9",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "9",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "9",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "0",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "0",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "0",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "0",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "p",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": ["left_command", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "p",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": ["left_command", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "p",
 "modifiers": { "mandatory": ["command", "control"] }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": ["left_command", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "h",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "h",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "j",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "j",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "k",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "k",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "l",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "h",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "h",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "j",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "j",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "k",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "k",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "l",
 "modifiers": { "mandatory": ["command", "shift"] }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": ["left_command", "left_option", "left_control", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "f",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "f",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "s",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "s",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "w",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "w",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "e",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "e",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 },
 {
 "from": {
 "key_code": "r",
 "modifiers": { "mandatory": ["command"] }
 },
 "to": [
 {
 "key_code": "r",
 "modifiers": ["left_command", "left_option", "left_control"]
 }
 ],
 "type": "basic"
 }
 ]
 },
 {
 "description": "Move common bindings from cmd to ctrl (r, t, z, T, l)",
 "manipulators": [
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "t",
 "modifiers": { "mandatory": ["control"] }
 },
 "to": [
 {
 "key_code": "t",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "t",
 "modifiers": { "mandatory": ["control", "shift"] }
 },
 "to": [
 {
 "key_code": "t",
 "modifiers": ["left_command", "left_shift"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "l",
 "modifiers": { "mandatory": ["control"] }
 },
 "to": [
 {
 "key_code": "l",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "r",
 "modifiers": { "mandatory": ["control"] }
 },
 "to": [
 {
 "key_code": "r",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "a",
 "modifiers": { "mandatory": ["control"] }
 },
 "to": [
 {
 "key_code": "a",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^net\\.kovidgoyal\\.kitty$",
 "^com\\.apple\\.Terminal$",
 "^com\\.googlecode\\.iterm2$",
 "^org\\.gnu\\.Emacs$"
 ],
 "type": "frontmost_application_unless"
 }
 ],
 "from": {
 "key_code": "z",
 "modifiers": { "mandatory": ["control"] }
 },
 "to": [
 {
 "key_code": "z",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 }
 ]
 },
 {
 "description": "Browser only: option+number -> cmd+number",
 "manipulators": [
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "1",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "1",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "2",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "2",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "3",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "3",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "4",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "4",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "5",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "5",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "6",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "6",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "7",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "7",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "8",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "8",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "9",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "9",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 },
 {
 "conditions": [
 {
 "bundle_identifiers": [
 "^org\\.chromium\\.Chromium$"
 ],
 "type": "frontmost_application_if"
 }
 ],
 "from": {
 "key_code": "0",
 "modifiers": { "mandatory": ["option"] }
 },
 "to": [
 {
 "key_code": "0",
 "modifiers": ["left_command"]
 }
 ],
 "type": "basic"
 }
 ]
 },
 {
 "description": "ö mod-tap hyper (via right_command)",
 "manipulators": [
 {
 "from": {
 "key_code": "semicolon",
 "modifiers": { "optional": ["any"] }
 },
 "to": [
 {
 "key_code": "right_command",
 "modifiers": []
 }
 ],
 "to_if_alone": [{ "key_code": "semicolon" }],
 "type": "basic"
 }
 ]
 },
 {
 "description": "caps_lock -> control/return mod-tap",
 "manipulators": [
 {
 "from": {
 "key_code": "caps_lock",
 "modifiers": { "optional": ["any"] }
 },
 "to": [{ "key_code": "left_control" }],
 "to_if_alone": [{ "key_code": "return_or_enter" }],
 "type": "basic"
 }
 ]
 }
 ]
 },
 "name": "Default profile",
 "selected": true,
 "virtual_hid_keyboard": { "keyboard_type_v2": "iso" }
 }
 ]
}

On Linux I use xremap to achieve the same thing.

xremap config for Emacs bindings everywhere

My ~/.config/xremap.yml:

keymap:
 - name: Emacs
 application:
 not: [Emacs, Nyxt, kitty]
 remap:
 # Cursor
 C-b: { with_mark: left }
 C-f: { with_mark: right }
 C-p: { with_mark: up }
 C-n: { with_mark: down }
 # Forward/Backward word
 M-b: { with_mark: C-left }
 M-f: { with_mark: C-right }
 # Beginning/End of line or page
 # C-a: { with_mark: home }, C-x h does not work in input fields as a replacement
 C-e: { with_mark: end }
 Alt-KEY_102ND: { with_mark: home } # M-<
 Alt-Shift-KEY_102ND: { with_mark: end } # M->
 # Page up/down
 M-v: { with_mark: pageup }
 C-v: { with_mark: pagedown }
 # Copy & paste
 C-w: [C-x, { set_mark: false }]
 M-w: [C-c, { set_mark: false }]
 C-z: [C-v, { set_mark: false }]
 Super-z: [C-v, { set_mark: false }] # I use this accidentally all the time
 # Delete
 C-d: [delete, { set_mark: false }]
 M-d: [C-delete, { set_mark: false }]
 # Kill line
 C-k: [Shift-end, C-x, { set_mark: false }]
 # Kill word backward
 Alt-backspace: [C-backspace, {set_mark: false}]
 # Undo
 Alt-u: [C-y, { set_mark: false }] # not working
 # Mark
 C-space: { set_mark: true }
 C-x:
 remap:
 h: C-a # C-x h
 # Search
 C-s: C-f
 # Cancel
 C-g: [esc, { set_mark: false }]

 - name: Browser only
 application:
 only: [firefox, Brave-browser]
 remap:
 # Kill tab
 C-q: C-w
 C-x:
 remap:
 k: C-w # C-x k

 - name: Terminal
 application:
 only: [kitty]
 remap:
 # Page up/down
 M-v: { with_mark: pageup }
 C-v: { with_mark: pagedown }
 # Cancel
 C-g: [esc, { set_mark: false }]
 # Copy & paste are directly configured in kitty.conf

Conclusions #

I showed how I use Karabiner Elements for turning macOS keybindings into their Linux counterparts. This is the most elegant solution for providing consistent keybinding across both platforms and can be adapted to your needs.

Enjoy :)

References #

Karabiner Elements docs for modifiers
Karabiner Elements docs for application specific rules
Going further and turning everything into Emacs keybindings

How I add automatic updates to my now page

web@jneidel.com (Jonathan Neidel) — Sat, 30 Aug 2025 00:00:00 +0000

I have a now page. I like the idea of having one, but I don’t like the idea of regularly updating it:laughing:

So, here is my approach to automating those regular updates.

Why not update it by hand? #

I tried this and it reads nicely. But this is worth little if I don’t update it regularly. I don’t want to establish a routine for this or put it into my calendar. I want to minimize those kinds of infrequent tasks, as they tend to pile up and take me out of what would otherwise do.

Contents of my now page #

On my now page I want to show what I’m currently working on. This information is (mostly) present in my note-taking system.

Design #

Extract active project information from note-taking system
Provide it to the website in json format
Include it in /now page

For 3. I had two ideas:

Commit changes to the repo and rebuild static source
Use Javascript to fetch and include the data dynamically

I went with the Javascript option for the simplicity. My commit history stays clean, the now source empty and I’m doing less rebuilds of my site.

Implementation #

Extract and provide data #

Extraction from the note-taking system and providing the data as json happens in one script, which is called daily via cron.

The script below:

gets project names by listing subdirectories of my projects dir
excludes projects with the file .now-ignore
uses a projects #+status: keyword, if the index.org file contains it
a paragraph between the keywords and the first outline (*) is urlencoded as the project description
uploads the json as a publicly accessible static file to some server

#! /bin/sh

. $HOME/.zsh/org.env

if [ "$1" = "--help" ] || [ "$1" = "-h" ] || [ "$1" = "help" ] || [ -z "$ORG_PROJECTS" ]; then
 cat <<EOF
$ org-projects-to-now-json
Transform org project directories into a json for jneidel.com/now
EOF
 exit
fi

hash jq urlencode || exit 127

get_project_list() {
 potential_projects=$(mktemp)
 find "$ORG_PROJECTS" -maxdepth 1 -mindepth 1 -type d -not -path '*/.*' | sort -V >$potential_projects

 ignored_projects=$(mktemp)
 find "$ORG_PROJECTS" -mindepth 2 -type f -name ".now-ignore" | rev | cut -d/ -f2- | rev | sort -V >$ignored_projects

 diff $potential_projects $ignored_projects --old-line-format='%L'
}

get_description() {
 index_file="$1"

 description_without_headers="$(head -n30 "$index_file" 2>/dev/null | grep -ve '^#+')"
 description_without_sections="$description_without_headers"
 if grep '^\*' -m1 "$index_file" >/dev/null 2>&1; then
 description_without_sections="$(echo "$description_without_headers" | grep -m1 '^\*' -B30 | head -n-1)"
 fi
 trimmed_oneline_description="$(echo "$description_without_sections" | sed 's/^[[:space:]]*//; s/[[:space:]]*$//')"
 echo $description_without_sections | urlencode | sed 's/^%0A$//'
}

assemble_json() {
 json=$(mktemp)
 cat <<EOF >$json
{
"date_updated": "$(date '+%b%d at %H:%M')",
"projects": [
EOF

 get_project_list | while read -r project; do
 index_file="${project}/$ORG_INDEX_FILE_NAME"
 title="$(basename "$project")"
 status=""
 if echo $project | grep -ve "🟨" >/dev/null; then
 status="$(grep -Po '#\+(status|STATUS): \K.*' "$index_file" 2>/dev/null)"
 fi
 description="$(get_description "$index_file")"

 printf ' { "title": "%s", "status": "%s", "description": "%s" },\n' "$title" "$status" "$description" >>$json
 done

 sed '$s/},/}/' -i $json # valid json: last line in array has no trailing ,
 echo "]}" >>$json

 if ! cat $json | jq >/dev/null; then # validate json
 echo "org-projects-to-now-json json validation failed at $(date) in $json" >>"$ORG_INBOX/org-projects-to-now-json errors"
 exit 1
 fi

 cat $json
}

upload_json() {
 json=$(mktemp)
 assemble_json >$json

 echo "Uploading to jneidel.*/now/now.json" >&2
 ~/scripts/cron/waitforinternet && scp -q $json u:~/html/jneidel.com/now/now.json
 ~/scripts/cron/waitforinternet && scp -q $json u:~/html/jneidel.de/now/now.json
}
upload_json

After running the script my json is available on /now/now.json.

Include data in /now #

The inclusion is straight-forward. I’m using hugo, where I can just add html to my markdown files. This is the relevant section (full source) which:

overwrite the “Updated” date from the json
turns json array into html
inserts html after the h2

## Active projects

<script>
(async () => {
const json = await fetch("now.json").then(res => res.json());
document.querySelector("time").innerText = "Updated: " + json.date_updated;

let html = "";
for (const { title, status } of json.projects) {
 html += `
 <p>
 <strong>${title}</strong>
 ${description && "<br>" + decodeURIComponent(description)}
 ${status && "<br><strong>Status</strong>: " + status}
 </p>
 `;
}

document.getElementById("active-projects").insertAdjacentHTML("afterend", html);
})()
</script>

That’s it. You can see the live results here.

Run script on USB-C Dock connect/disconnect under Linux

web@jneidel.com (Jonathan Neidel) — Sun, 24 Aug 2025 00:00:00 +0000

I have a USB-C dock with my monitor, speakers, ethernet, etc. I’m running GNU/Linux and I want to run some scripts when the dock is plugged in or unplugged. Here is how to write the udev rules and script to make that happen.

TL;DR: short demo, my udev rules, my script

Udev rule to match device #

To detect when devices are added (plugged-in) or removed (unplugged) we use udev.

Like this you can see what kind of devices are being added and removed when you do a certain action, like unplugging the dock:

# to monitor device changes
udevadm monitor --environment --udev >udev-unplug
# now unplug your device
# ^C to stop monitor
less udev-unplug
# explore logs

The first entry is already what I want:

UDEV [12639.852887] remove /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2.2/1-2.2:1.0 (usb)
ACTION=remove
DEVPATH=/devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2.2/1-2.2:1.0
SUBSYSTEM=usb
DEVTYPE=usb_interface
PRODUCT=17ef/3060/0
TYPE=0/0/0
INTERFACE=17/0/0
MODALIAS=usb:v17EFp3060d0000dc00dsc00dp00ic11isc00ip00in00
SEQNUM=7716
USEC_INITIALIZED=12618475392
ID_VENDOR_FROM_DATABASE=Lenovo
ID_MODEL_FROM_DATABASE=ThinkPad Dock
ID_PATH_WITH_USB_REVISION=pci-0000:00:14.0-usbv2-0:2.2:1.0
ID_PATH=pci-0000:00:14.0-usb-0:2.2:1.0
ID_PATH_TAG=pci-0000_00_14_0-usb-0_2_2_1_0

ID_MODEL_FROM_DATABASE=ThinkPad Dock identifies it as my dock.

The udev rule that matches this criteria would be:

ACTION=="remove", ENV{ID_MODEL_FROM_DATABASE}=="ThinkPad Dock", RUN+="/home/jneidel/scripts/dock disconnect"

Add this rule and reload udev:

sudo vim /etc/udev/rules.d/thinkpad-dock.rules
# add rule and exit
sudo udevadm control --reload-rules

To check that the rules matches, you take the DEVPATH from the above monitor log entry and run:

udevadm test --action=remove <DEVPATH>
# the DEVPATH looks like this: /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2.2/1-2.2:1.0

This shows me that my rule was matched, because the line at the end about “Queued commands” includes the script that I added as using RUN+= as part of the rule:

Queued commands:
 RUN{program} : /home/jneidel/scripts/dock disconnect

A rule for adding can be created from this by copying the rule and changing ACTION== to "add".

More specific rule criteria #

Maybe you don’t have the ID_MODEL_FROM_DATABASE or something similar. Or maybe you need to have a more specific rule, that unsures that it is only run once.

Specifying a SUBSYSTEM=="usb" can narrow what is matched. Also you can use the line PRODUCT=17ef/3060/0 and match those ids like this ENV{ID_VENDOR_ID}=="17ef", ENV{ID_MODEL_ID}=="3060". By selecting a component like the ethernet driver you can ensure this only matches one entry (so the command is only run once.)

RUN vs starting a systemd service #

With ENV{SYSTEMD_USER_WANTS}="dock-plugged.service" a rule can trigger service, which sets up the environment and runs the script. This works well for ACTION=="add", but not at all for ACTION=="remove".

RUN+="<command>" works consistently for “add” and “remove” actions, so I went with that.

My rules #

These are the udev rules I am using:

ACTION=="add", SUBSYSTEM=="hidraw", ENV{ID_MODEL_FROM_DATABASE}=="ThinkPad Dock", RUN+="/home/jneidel/scripts/dock connect"
ACTION=="remove", ENV{ID_MODEL_FROM_DATABASE}=="ThinkPad Dock", RUN+="/home/jneidel/scripts/dock disconnect"

The script to run #

With RUN+="/home/jneidel/scripts/dock disconnect" I can run a script of mine.

The script I use is this:

#! /bin/sh

if [ "$1" = "--help" ] || [ "$1" = "-h" ] || [ "$1" = "help" ] || [ -z "$1" ]; then
 cat <<EOF
$ dock [connect|disconnect]
Trigger some actions upon connect/disconnect of the dock.
EOF
 exit
fi

# setup env for udev to interact with Xorg (xrandr) for monitor script
export DISPLAY=:0
export XAUTHORITY=/run/user/1000/Xauthority

case "$1" in
 connect)
 /home/jneidel/scripts/monitor -m on
 sleep 4s; /home/jneidel/scripts/sxhkd/backlight set 100
 ;;
 disconnect)
 /home/jneidel/scripts/monitor off
 /home/jneidel/scripts/sxhkd/backlight max
 ;;
esac

Here are the monitor and backlight scripts.

I use sleep 4s, to delay turing off the laptop backlight until the monitor is activated.

Demo #

Here is a short demo to see my setup in action:

References #

Add Hyper Modifier Key for Emacs on Linux

web@jneidel.com (Jonathan Neidel) — Sat, 16 Aug 2025 00:00:00 +0000

Modern keyboards don’t include a hyper key anymore. Emacs still supports it and here is how and why to add it.

What make adding a hyper key useful? #

Emacs only has so many modifier keys. Control and meta are heavily used by the default Emacs keybindings. Super is already filled with my window managers bindings. Leaves only the alt and hyper modifiers, for which I don’t have keys on my laptop.

Using one of these two empty modifier keys allows me to add a lot of directly accessible keybindings in an unobstrusive way. C-x … is fine for most commands, but what I use all the time I want more directly accessible, like H-b for buffer selection. And especially commands that repeat, like undo don’t work well with non-modifier prefixes like C-x, because you can’t keep holding it throughout.

Add your own hyper key #

What you are going to need is:

A key you don’t care about. I picked “End”. It can even be a key that does not exist on your keyboard, like “F13”.

An easy to reach key you want to turn into “Hyper”. I picked “ö”. On my german keyboard this is on the home row under my right pinkie. You will not loose this key. On tap it will work as normal, only on hold will it turn into “Hyper”.

In the configuration below I will reference the keys that I chose. Just replace them with the keys of your choosing when you set this up.

The tools and the logic #

I use kmonad for outputing different keys when I (1.) press a key or (2.) hold it for 200ms. This is technique is called “mod tap”.

The only problem with kmonad is, that it doesn’t know the hyper key. The way we get around this is by having kmonad output a key that we don’t care about (“End” in my case) and turning that into “Hyper” with Xmodmap.

Implementation #

~/.config/kmonad/mod-tap-mappings.kbd

(defcfg
 input (device-file "/dev/input/by-path/platform-i8042-serio-0-event-kbd")
 output (uinput-sink "kmonad-output")

 ;; tell KMonad to let unconfigured keys act normal
 fallthrough true
)

(defsrc
 semicolon ;; this is the evdev version of ö
)

(defalias
 ;; ö mod-tap: tap = ö (semicolon), hold = hyper (end)
 ö_hyper (tap-hold-next 200 semicolon end)
 ;; end will be transformed into hyper by xmodmap
)

(deflayer base
 @ö_hyper
)

Now run sudo kmonad ~/.config/kmonad/mod-tap-mappings.kbd &

~/.config/Xmodmap

! unmap Num_Lock as a modifier
! see modifier mappings: $ xmodmap -pm
remove mod2 = Num_Lock

! have End key output Hyper instead
! see keycodes: $ xmodmap -pke
! mapping both for good measure 🙂
keycode 115 = Hyper_R
keycode 87 = Hyper_R

! activate Hyper as a modifer key
add mod2 = Hyper_R

Now run xmodmap ~/.Xmodmap

You can now test that your newly-added hyper key works in Emacs:

(use-package consult
 :ensure t
 :bind ("H-b" . consult-buffer))

The last thing is to configure this to run at startup. Observe the order of execution, as it is matters. Add this to your autostart script, in my case ~/.xinit

# ö mod-tap: tap = ö, hold = hyper
sudo kmonad ~/.config/kmonad/mod-tap-mappings.kbd &
sleep 3s && xmodmap ~/.Xmodmap &

Enjoy :slightly_smiling_face:

References #

The issue on the kmonad Github contains more details if you want to dive deeper.

Utilizing git rebase for a clean commit history

web@jneidel.com (Jonathan Neidel) — Fri, 18 Apr 2025 00:00:00 +0000

This article describes how I do use git rebase to maintain a clean commit history in my projects. With this manipulating your git history will become second nature.

A working knowledge of git in the terminal is assumed.

Definitions #

When I say “a clean commit history”, what do I mean? To me, a clean history contains:

well described commit messages
no superfluous commits (a là “fix conflicts”, “fix linting”)
no merge commits

Rebasing by itself does not achieve these goals, but without the tool it will be hard to address them.

An example of a clean history:

An example of a messy history:

Both screenshots were taken of git log --oneline --graph --decorate (my prefered log style.)

Avoid merge commits #

To not clutter up the history with useless merge commits:

We don’t create merge commits when merging a pull request.
We don’t create merge commits locally.

The first is configure where ever your repo is hosted or by selecting the correct option on the “merge” button:

The merge options on Github.

I use the rebase pull strategy, to not create merge commits while pulling in changes from a remote or when fixing conflicts. It is set up like this:

git config --global pull.rebase true

Implications of rebasing #

A commit history contains a sequence of commits. Each commit is identified by a hash over it’s contents.

When we use git rebase, we rewrite that history. The order of commits might be changed. The contents or description of a commit might be altered. Thus their identifiers (hashes) change.

The result is a history incompatible with the remote server you might have pulled from. To use our version of the history, we have to overwrite what is on the server. We git push --force. Better yet, we use git push --force-with-lease to make sure we are not overwriting the work of someone else, who has pushed in the meantime.

Interactive rebase #

You have probably seen this long list of available commands in an interactive rebase session:

Half of those I have never used. What I do is:

Remove a commit with “drop”
Change a commit; to “reword” the commit message or “edit” the contents
Combine commits together with “squash”
Reorder commits by moving a line up or down

A word on aliases

I strongly recommend that you set up your own (shell¹) aliases for these commands, if you haven’t. Here are mine.

They create a great user experience. Type gri instead git rebase -i. In this article I use the full forms of commands for you to copy, but in real use I have aliases for all of them.

Aliases allow you to configure your preferences. I like my git status with -sb, but I don’t have to remeber those flags, because my alias contains them.

Get into a rebase #

The common method to start an interactive rebase is: git rebase -i COMMIT_HASH. This includes all commits up-to, but not including the referenced commit. To have the rebase include a specific commit, we need to use the hash of the previous commit (N+1).

In red: the commit we want to rebase.
In green: the commit hash we need to use.

Most of the time, it’s good enough for me to just get a rebase with the last N commits. Then I don’t need to check the git log. Should the commit I’m looking for not be included, I can quickly exit and increase the N.

With my aliases I do this to rebase the 5 most recent commits: gri H5.

# in zsh config
alias gri="git rebase -i"
for i in $(seq 0 10); do
 alias -g H$i="HEAD~$i"
done
# global aliases (-g), might be unavailable in other shells

Remove commits #

Change “pick” (aka keep this commit) to a “drop” to remove it from the history.

Change commits #

You can either “edit” the contents of a commit (including the commit message) or just the commit message with “reword”.

Here is what happens on “edit”:

To make amending easier I have aliases for amending with or without modifying the commit message (“no-edit”):

alias amend="git commit --amend"
alias amendd="git commit --amend --no-edit"
alias addam="git add -A; git commit --amend --no-edit"

I also recommended to also have aliases for git rebase --continue and --abort to be able to easily weave through these rebase steps.

Combine commits #

Use “squash” to combine multiple commits into one.

The commit marked with “squash” will be merge into the commit visibly above it. Commits are listed in chronological order (oldest at the top.) That is why the squash command descriptions says “meld into previous commit” (“previous commit” being the one above.)

Vim highlights that the commit at the top can’t be squashed (because no previous commits are available above.)

After a squash you will get a prompt to rewrite the commit message for the new combined commit.

Reorder commits #

To change the order of commits, simply move the line up or down:

Automatic stashing #

If you have uncommitted local changes, git will usually not allow you to start a rebase. It asks you to commit or stash your changes. I found myself constantly stashing and unstashing my changes before and after a rebase.

Git can do this automatically for you:

git config --global rebase.autostash true

See man git-config under autostash for more information.

Oh No, something went wrong #

In case anything goes wrong with your rebase, git reflog is your friend. You can revert your actions (e.g. go back to before the rebase) with git reset --hard, just like you are used to with commits.

Fixup #

Fixup it is like amend for a commit that is further down in the history (amend works on the most recent commit.)

I don’t use the “fixup” command provided in a interactive rebase session. Instead I wrote a script for myself, that works similar to my amend alias. I say which commit I want to “fixup” and the script does the rest.

Simple usage of the fixup script.

Code for the script

In your zsh config:

gri() {
 hash=$1
 if [[ "$hash" =~ "\+$" ]]; then # matches "HASH+"
 echo "Using the commit hash prior to the one given: $prevHash"
 git rebase -i $prevHash
 else
 git rebase -i "$@"
 fi
}

gcff() { # fast fixup
 hash=$1
 git commit --fixup $hash
 GIT_SEQUENCE_EDITOR=: gri $hash+
}

To have the fixup automatically be applied when running git rebase -i, this setting is needed:

git config --global rebase.autosquash true

See man git-rebase under autosquash for more information.

This is a good article to understand how fixup works. (Not needed if you just want to use the above script.)

Conclusions #

With this, you should have all the tools available to you to manipulating your git history at will.

You can use shell aliases, but you can also use aliases defined in git. A shell alias for git log could be gl and a git alias git l.
A git alias can be defined like this: git config --global alias.l 'log --oneline'. Here is my config file. ↩︎

Migrating the technical documentation of a legacy project to Arc42

web@jneidel.com (Jonathan Neidel) — Tue, 31 Dec 2024 00:00:00 +0000

This article will give a rough overview of how I introduced structured technical documentation to our project as a regular developer. You should walk away with a good understanding of how to pitch your stakeholders and how to see the implementation through.

This is not meant as a step-by-step guide, but as a description of what we went through, so that it might serve as an example for you. It might give you a better idea of the work that would be involved to implement something similar in your own project.

More on the project #

I work on a 9 year old PHP project. It has three smaller scrum teams on it. The project used to be on a fixed-price contract, but is not anymore. We actively work on maintenance: refactorings to increase the PHP version, performance improvements, increasing test coverage and much more.

Status quo

This is where we were at in terms of documentation: We had some technical documentation, which was scattered around in multiple locations (in git, confluence, meeting recordings.) It was incomplete, unstructured, not so easily unsearchable and outdated (by up-to 8 years in some places.) Developers rarely created new documentation. If they did, they would not know where to put it (they just chose what seemed like the right place.) There were no processes for reviewing new and updating existing documentation.

Motivation #

After that description of the status quo, it should be pretty clear why we, the developers, want a proper system for our documentation. But why should the company invest into it? Here are the main arguments that applied to our case.

ROI for the company #

Save money onboarding new developers.
Before: You have hours and hours of meetings, where you pay for the time of everybody involved.
After: The person being onboarded reads the technical documentation and gains an overview and a detailed understanding of the system.

Protect yourself from a catastrophic loss of business and domain knowledge when a key person leaves.
Any time only one person has deep knowledge of something you’re at a risk of losing that knowledge for one of many conceivable reasons: The person has an accident, quits, becomes unwilling to share it, etc. Can you afford that loss?

Save money by speeding up development.
In large systems or with new people there will always be areas a developer is unfamiliar with. Being able to familiarize yourself quickly with the part of the system you’re supposed to work on and having all your questions answered, which would otherwise need digging, can save a lot of time and frustration. This become more complicated as the system grows larger.

Requirements #

Ok, so we want better technical documentation. What would it look like in a perfect world?

The ideal documentation would be:

all in one place
easily searchable
structured in a way that makes sense
everything has a clear place (can be deducted by reason)
up-to-date
complete to the degree of detail needed
reviewed by the team
integrated into other processes
everybody has access

arc42 #

Part of the requirements just laid out necessitate a structure to the information. One can come up with their own, but I looked into arc42 and was convinced that this would be better and quicker than rolling my own.

What is arc42? #

arc42 is a way to structure technical documentation. It provides you twelve logical buckets (sections) for your documentation. Every piece of information you have goes into one of the twelve sections. Thus, arc42 makes it clear where to read some information and where to add information.

The sections in more details #

Expand

Introduction and Goals
Short description of the product requirements, quality goals and stakeholders.
Constraints
Anything that constraints decisions about design, implementation and processes.
Context and Scope
Delimits your system from its (external) communication partners (neighboring systems and users). From a business/domain perspective.
Solution Strategy
Summary of the fundamental decisions and solution strategies that shape the architecture.
Building Block View
Static decomposition of the system across multiple levels of abstraction.
Runtime View
Behavior of building blocks as scenarios, covering important use cases or features and interactions at critical external interfaces.
Deployment View
Technical infrastructure with environments and topologies. Mapping of (software) building blocks to infrastructure elements.
Crosscutting Concepts
Overall solution approaches relevant in multiple parts of the system.
Architectural Decisions
Important architecture decisions including their rationales.
Quality Requirements
Quality requirements as scenarios.
Risks and Technical Debt
Known technical risks or technical debt. What potential problems exist within or around the system? What does the development team feel miserable about?
Glossary
Important domain and technical terms that stakeholders use when discussing the system.

Visit arc42s website for more detail and examples of what a section might look like.

Why we chose arc42 #

Arc42 elegantly solves a part of the puzzle. It provides a structural foundation that is a complex as you need it to be and flexible enough for our customization. It is also not bound to any specific platform.

Implementation #

Above we defined a proposal (requirements, ROI, some details). How did I take this proposal and made it a reality inside of the project?

While this whole article is very structured and linear, the real implementation was messy. My thinking evolved over time as the proposal grew, through discussion and negotiation with relevant stakeholders. What you read is the clearest version, written with the benefit of hindsight.

Talking to stakeholders #

The first step was to gather more information and get the buy-in from the stakeholders, who would effected by the change.

Below are the people I brought the proposal to. Throughout, I continuously improved the proposal, adjusted it to newly discovered requirements and made it more concrete.

Tech Lead
In our 1-on-1 we discussed the feasibility (timing, scope, ROI) of the proposal and the solution itself.
Developers
In one of our bi-weekly meetings I presented the proposal to the whole group, collected feedback and held a vote on contentious design decisions. I incorporated their suggestions and got their OK.
PM/PO (Client stand-in)
With the developers behind me, I brought this to my PO/PM. They welcomed the (at this point) well thought-out maintenance initiative and brought it to the client for approval.

After getting approval and me writing the Jira tickets, it was up to the PMs to schedule the epic. It would be a few months before I could continue.

Agreed upon implementation #

We landed on:

arc42 as the structural framework
GitLab as the place for the documentation

Finding a place for the documentation

Among the developers we had a debate about where and how the documentation would be stored. The options we considered were:

Confluence
GitLab (in monorepo or in separate repo?)

Besides providing search, the individual pros for the platforms were:

Conflunence pros:

PO/QA documentation is already there. The projects documentation would be all in one place.
Everybody (including externals) already has access.
A few diagrams can be created in Confluence.

GitLab pros:

Devs and many more already have access and it can be easily provided.
Many types of diagrams can be done with mermaid and remain easily adjustable.
Easily integrates into the existing review process.
Developers don’t have to leave their dev environment to write documentation. It’s not a separate process.
Documentation can be submitted alongside relevant code changes.
Written in the ubiquitous markdown format (not whatever Confluence uses.)
URLs are static and reliable, based on filenames not titles (Confluence URLs are based on the title of the page, i.e. they will change dynamically.)
Can be browsed locally and offline.

To simplify code review we decided to include the technical documentation in the monorepo.

Structuring and migration #

This part made up the main body of work. Applying the structure of arc42 to our project and migrating in anything and everything that already existed in terms of technical documentation.

Gathering existing documentation #

We had documentation spread around a few different places:

multiple sections, with many pages across our Confluence
markdown files in the git
sections in the main README.md
onboarding videos in Sharepoint

I extracted everything and put it in my note-taking system. Skimming the notes, I grouped them into rough buckets (e.g. architecture, deployment, by feature/component, etc.), while also filtering out the irrelevant and outdated.

Grouping into sections #

Arc42 provides well-defined sections for structuring your documentation. (See the overview for a list.) Now it was the time to go through the sections one-by-one and for each:

write an introduction (if necessary)
migrate the contents of relevant notes into that section
gather information to complete or update parts (where necessary)
describe missing aspects (if too large a task, I created a ticket with detailed expectations and questions I expected to be answered)
have it reviewed by colleagues and revise where necessary

With that the new technical documentation stood.

Cleanup and post-processing #

The source of the migrated data should be cleaned up. I marked the old confluence pages as deprecated with a big notice at the top and a link to the new documentation. Content deletion or moving it into a dedicated space would also be possibilities.

I presented the technical documentation to the team and directed them toward creating new documentation inside of the new structure. Any new documentation that was created in the meantime I migrated or wrote a ticket for.

Processes and future work #

Part of the requirements is to keep the docs up-to-date and this can only be achieved through integration into our processes. Without that it would soon become just another outdated dump.

Measures we put into place #

New technical documentation is to be created within the newly created structure.
New components and features need to be described (to some extent.)
Architecture decisions need to be described as ADRs¹.
Changes to existing component need to be reflected in the existing documentation.

These expectations were described in the contribution guidelines and communicated to the developers.

Concrete details #

The above was intentionally light on concrete, technical details of our implementation. I have them collected here in one place for those who are interested.

Show details

All files related to technical documentation live inside of the main GitLab monorepo under docs/.
As part of the main repo: Any change to the documentation follows the normal review process (looked into by at least two other devs.)
Each arc42 section is it’s own markdown file.
- docs/1. Introduction and Goals.md, etc.
docs/README.md contains a list of links to all sections. GitLab directly renders this list as a convenient overview if you click into the docs directory on the web.
Diagrams are preferably included as mermaid source code inside of a mermaid code block, which GitLab (and GitHub) renders as an inline diagram.
Source code is directly linked to, instead of being verbosely quoted: [Abc](../src/Abc.php).
You can make the GitLab docs searchable through a custom bang: In your browser settings you add it here:

The URL with %s looks something like this: GITLAB_HOST/search?group_id=379&nav_source=navbar&project_id=236&repository_ref=docu&scope=blobs&search=+path%3Adocs+%s&search_code=true

Then you can search for e.g. !docs block:

This turns into:

It will direct you to the result in GitLab:

Conclusions #

For our project it was well worth investing in proper technical documentation. Arc42 was a major help in this endeavor.

An Architecture Decision Record describes the reason behind important (architecture) decisions inside of the project. This is so that if the team in the future wants to change the something they can understand why in the past it was designed like that. ADRs are part of section 9: Architecture Decisions. ↩︎

How to make use of Audio and Video Transcoding

web@jneidel.com (Jonathan Neidel) — Wed, 06 Mar 2024 00:00:00 +0000

This is a comprehensive and pragmatic overview of everything I know about audio and video transcoding.

Relevance #

Why should you learn about audio and video codecs?

For me it all started with this:

That’s my homeservers CPU jumping to 80% load when I played a movie in Jellyfin. I was informed of the reason: some transcoding was happening.

I would hear the fan roar every time my wife was watching her series. And it doesn’t scale either. Watching a second movie simultaneously would max out the CPU and decrease playback quality since the server can’t keep up.

I had to figure this codec issue out :slightly_smiling_face:

For you the reason to learn about encodings might be:

Reduce the file sizes of your media and conserving disk space
To understand torrent labels
Because you want to combine together a bunch of clips you collected
Or simply out of curiosity.

Let’s start with analysing and picking the codecs that are right for you:

Codecs #

A codec is the spec for how audio or video data is stored. What differentiates them is their different approaches to compressing the data (getting rid of information ideally without you noticing¹.) I will speak to how this effects perceptible quality and file size.

What codecs you choose depends on your circumstances:

Where are you watching? Laptop, monitor, TV, home cinema?
How much storage capacity do you have and want to use?
What codecs does your video playback application support?

In my case I’m making my media available to be streamed in the browser (via Jellyfin.) The only browsers I care to support are the latest Firefox and Chromium. With that in mind I have chosen these as my preferred codecs: AV1 for video and Opus for audio.

Here is an overview of the most widely used codecs and why I decided on AV1 and Opus:

Picking a video codec #

I started with this table. You can see the codec support Jellyfin has in the browsers I picked.

Jellyfin support for video codecs (Source)

VP9 can be excluded as it’s the predecessor of AV1. Let’s look into the other three in some detail.

H.264 (AVC) #

The standard video codec of old. Has the best possible browser support and is widely used.

H.265 (HEVC) #

The successor of H.264 capable of compressing up-to 50% more (while retaining the same perceptible quality.) Thus saving a significant amount of storage space.

Looking at the compatibility table above, I had to exclude H.265 for my use-case though. It has no Firefox support and I haven’t even managed to get it running for myself in Brave on Linux. I can’t expect this to work for most people. It would mean constant headaches and maintenance work on my end. It’s not an option for me.

AV1 #

AV1 boasts a compression rate even better than H.265 and with that better quality at the same bitrate.

On top of that AV1 is licensed as royalty-free unlike the commercial H.26X family.

With this, and great support in modern browsers, AV1 is the obvious choice for me.

Bit depth #

The compatibility table also mentions bit depth. 10bit will be smaller size and better quality. In my eyes the difference is so slight that I pretty much ignore bit depth.

Picking an audio codec #

With Jellyfin audio transcoding is not nearly as resource-intensive as video transcoding. And I am not using surround sound capable speakers, so I don’t really care. But if I’m already transcoding something I might as well put it into the best available audio codec while I’m at it.

Jellyfin support for audio codecs (Source)

Looking at the table we see that we can pretty much use whatever codec we want (with the exception of DTS and AC3.)

The one I went with was Opus, since it’s open-source and boasts the same quality as other codecs at higher compression rates².

Transcoding #

Now that we have the codecs we want to use the obvious question is: How do we encode a media file with that codec?

That requires some understanding of FFmpeg, so let’s have a look at it’s syntax.

FFmpeg syntax #

With FFmpeg you have one or more input files and a single output. You specify inputs with -i and the output as the last argument without any flags.

ffmpeg -i in.mp4 out.mkv

Before you can tell ffmpeg what to do, you first need to tell it what you want to work on. You do this with -map. Your inputs are zero indexed. To make use the whole first input you can do:

ffmpeg -i in.mp4 -map 0 out.mkv

I said the whole first input because what -map 0 does is that it picks up all the available input streams.

You can also target these separately: -map 0:a for audio, -map 0:v for video and -map 0:s for subtitles. The advantage being that you can easily combine the video stream from one file with the audio stream from another and add in the subtitles from an .srt file. Please refer to FFmpeg’s excellent -map documentation if you want to go deeper on this.

To see what streams are included in a file and on what index they are available, pass the input ffprobe using the same order and -i flags.

For my transcoding I want to make use of all the video, audio and subtitle streams of my single input file. By using -map 0 I ran into problems. From time to time I came across weird embedded streams that FFmpeg didn’t know what do with, throwing an error. So I always specify that I want to use audio, video and subtitles separately. To only use them if the stream is available (and avoid error à la “this file has not subtitle stream”) you can append a ?, like so:

ffmpeg -i in.mp4 -map 0:v -map 0:a -map 0:s? out.mkv

Now that we have declared what parts of the input file we want to use, we can finally tell FFmpeg what we want to do with them.

The simplest statement is copy. So to just copy over the subtitles without doing anything to them we can say -c:s copy.

To transcode our video to AV1 we specify the library to use: -c:v libsvtav1. To see the available encoders and their options see FFmpeg’s AV1 page³. More on presets and quality later.

For transcoding our audio to Opus we specify: -c:a libopus.

And with that have our first working ffmpeg transcoding command:

ffmpeg -i in.mp4 -map 0:v -map 0:a -map 0:s? -c:v libsvtav1 -c:a libopus -c:s copy out.mkv

This will use the encoding library’s defaults. We will want to tweak them depending on our situation.

Let’s look at Opus first. You can specify the bitrate. FFmpeg recommends at least 64Kbps⁴. I go with either 128Kbps or 192Kbps (specified as -b:a 192k.)

AV1 encoding options will have a major impact on file size, picture quality and encoding duration. Please see this article on see how the -crf and -preset play together and how they influence those three factors.

Based on the conclusions from the article and my own requirements I have chosen a preset of 8 for decent file size and encoding time⁵. For -crf I vary from 38 – for old cartoons where less detail actually looks better to me – to 20, when I want to retain higher quality.

With this we have the final version of our command and are ready to start transcoding:

ffmpeg -i in.mp4 -map 0:v -map 0:a -map 0:s? -c:v libsvtav1 -crf 30 -preset 8 -c:a libopus -b:a 192k -c:s copy out.mkv

Examples for the effect of -crf #

Encoding at a crf of 30 and a preset of 8 I get file sizes like these:

All seasons of Seinfeld in 1080p at 51GiB (average episode is 300MiB)
All seasons of The Office in 1080p at 45GiB, (average episode is 240MiB)

Transcoding speed was about 2-2.5 seasons in a 8h session while nearly maxing out my CPU with a combined processing power of 16.20GHz.

Pausing on some shots you will definitely see some artifacts, but in motion it looks fine to me.

Here are two transcodes of the same file at different quality levels:

-crf 38 -preset 8

-crf 26 -preset 8

With a lower preset you get achieve an even smaller file size (and slightly better quality.) It will just take longer to encode⁵.

Common transcoding problems #

Our last assembled command will fail if there is 5.1 surround audio. Until they adjust the default setting to not crash you fix it with -af 'channelmap=channel_layout=5.1'. If you get an error complaining about stereo audio, you will have to remove this modifier though.

I already mentioned marking input streams as optional, so the process does not error out if they are not available: -map 0:s?.

My transcoding flow #

This is very specific to my circumstances but I wanted to share it as an example. Feel free to skip it.

I move all the files I want to transcode into a specific directory on my server. Then this script generates the encoding commands for me, which I can adjust and reorder if needed.

ssh home 'find ~/media/transcode' -name "*.mkv" -or -name "*.mp4" | grep -ve "/out/" -e "/later/" -e "/done/" | cut -d/ -f6- | while read file; do
 name="$(basename "$file" | sed "s/HEVC//; s/x265//; s/'//g")"
 echo "nice ffmpeg -loglevel error -i '$file' -map 0:v -map 0:a -map 0:s? -c:v libsvtav1 -crf 35 -preset 8 -c:a libopus -b:a 192k -af 'channelmap=channel_layout=5.1' -c:s copy 'out/$name'"
done

This script ensures that mkv files stay mkv (they would complain about becoming mp4s) and that that are no ' in the output file name (which would mess up the command execution.)

This list of commands I then copy into another script I use for running arbitrary code on my server:

#! /bin/sh

run_command() {
cat <<EOF
cd '/transcode'
mkdir -p out

nice ffmpeg -loglevel error -i 'Dave Chappelle Deep in the Heart of Texas (2017) (1080p NF WEB-DL x265 HEVC 10bit AC3 5.1 t3nzin)/Dave Chappelle Deep in the Heart of Texas (2017) (1080p NF WEB-DL x265 10bit t3nzin).mkv' -map 0:v -map 0:a -map 0:s -c:v libsvtav1 -crf 30 -preset 8 -c:a libopus -b:a 128k -af 'channelmap=channel_layout=5.1' -c:s copy 'out/Dave Chappelle Deep in the Heart of Texas (2017) (1080p NF WEB-DL 10bit t3nzin).mkv'
nice ffmpeg -loglevel error -i 'All.Light.Everywhere.2021.1080p.BluRay.x265-RARBG/All.Light.Everywhere.2021.1080p.BluRay.x265-RARBG.mp4' -map 0:v -map 0:a -c:v libsvtav1 -crf 26 -preset 8 -c:a libopus -b:a 192k -af 'channelmap=channel_layout=5.1' 'out/All.Light.Everywhere.2021.1080p.BluRay.-RARBG.mp4'
EOF
}

echo "#!/bin/bash" >/tmp/run
run_command >>/tmp/run
chmod +x /tmp/run
scp -q /tmp/run home:
ssh home 'docker run --rm --volume /root:/root --volume "/mnt/user/media/transcode":/transcode -t worker "./run"; rm run' &

The docker stuff at the bottom is very specific to Unraid where I don’t really have the ability to just install the required dependencies on the system itself. So I have setup an Arch container where I just install whatever I need and then have this my script run in there.

Torrent Quality #

Another place where encodings come up are when browsing for torrents. For many years I saw these arcane combinations of letters and numbers without knowing what they meant. I just looked for what I knew, which was resolution and picked whatever was smallest. Now though, with our new-found knowledge we can understand what those labels mean!

Picking a random movie, we look at a few example titles and decipher them together:

Anatomy of a Fall (2023) (1080p BluRay x265 HEVC 10bit EAC3 5.1 French Silence) [QxR]

We got the name, year and resolution. So far so obvious. Next is the encoding source: BluRay. Then the video codec: x265/HEVC with 10bit depth. And audio codec: EAC3 (aka DDP/DD+) in 5.1 surround sound. Lastly we got French to let us know that the only included audio track is french and lastly the tags of the encoding group.

Anatomy Of A Fall (2023) 1080p H264 iTA Fre AC3 5.1 Sub iTA EnG-MIRCrew

This one is encoded in H264 instead. It also includes an Italian (iTA) audio track on-top of the French (Fre) one. The audio encoding is different: AC3 (DD) and it specifies the subtitles which are included.

What is notably missing is the encoding source. It’s not mentioned in the description either, so we can only guess.

Anatomy.of.a.Fall.[Anatomie.D.Une.Chute].2023.1080p.WEB-DL.x264.AC3-HORiZON-ArtSubs

Next we have the original french title included and WEB-DL as the encoding source. The x264 encoder was use to encode to h.264.

Anatomy of a Fall 2023-FR-1080p-HD-WEBRip-2.76GiB-AAC-x264 [PortalGoods]

Our last example has WEBRip as the encoding source and AAC as the audio codec.

With that we have seen the most common labels. AV1 can of course also show up as the video codec, but it’s still kind of rare. With video and audio codecs you should already know what you are and aren’t looking for.

The most important label relating to quality is the:

Encoding source #

Here are the different types, ranked in order of quality from best to worst:

BluRay, also called BR and BRip
WEB-DL, also known as WEB. Sometimes the platform it was downloaded will be specified: AMZN WEB-DL, NF, iTunes, etc.
WEBRip
HDRip
DVDRip, resolution is always less than 720p
CAM

BluRay and DVDRip are obvious. Straight encodings from the physical medium.

WEB-DL is the result of a group downloading the desired media from a streaming service and breaking the encryption. With WEBRip they are not breaking the encryption, but are instead recording their screen while streaming the media. There is a noticeable quality difference between WEB-DL and WEBRip, since the former has access to the raw data.

Below you can see an example of what the different between WEBRip (first picture) and WEB-DL (second picture) can look like. Take a look at the roof of the car on the right. It looks much more crisp in the WEB-DL version.

WEBRIP Example

WEB-DL Example

HDRip are screen captures of a show/movie on TV and commonly include the stations logos, announcements and banner ads.

CAM are camera recordings of the screen in the cinema and are not an option in my mind. The quality is that terrible. Just wait for the digital release and a WEB encoding or go to the cinema.

There are also torrents with the REMUX label. It means the data was not encoded, that the source streams were just repackaged into a container without any compression. These will be huge 60GB files per movie, usually copied directly from the BluRay. With those kinds of sizes they are not an option for me. They are the best quality you can find available.

Bitrate #

There is another thing that has a notable impact on media quality: the bitrate. You will be able to find it in the description or the mediainfo dump of most – but not all – torrents.

…
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Duration : 1 h 42 min
Bit rate : 1 955 kb/s
…

The range I’m looking for is from at least 1.500 kbps up-to 4.500 kbps. Some people would call that bitrate-starved, I call it diminishing returns. I’m also watching on a laptop. In a still frame I might be able to spot some minimal differences, but it’s a moving picture.

You can check the bitrate on any given file with: mediainfo file.mp4

Yify releases are good enough for me with most movies (and they are always in H264.) There are only a few with bombastic pictures like Avatar, Blade Runner or Dune where I will go for higher bitrate releases to get a nicer looking movie.

At low compression rates the difference will be imperceptible. The higher you turn up the compression rate the smaller the file size and the lower the perceived quality. Compression is about finding a balance that is acceptable to the end user with the trade-off being quality vs file size. Some compression algorithms are better than others and will results in better perceptible quality at the same file size (the amount of information.) ↩︎
see ↩︎
Do not use libaom for real-world use. It is the reference implementation and significantly inferior in performance to the implementations that came after it. ↩︎
Recommended minimum bitrates ↩︎
I would consider going down to a preset of 6 if I really cared about the transcode to get a slightly smaller file size and slightly better quality. Usually that’s not worth the doubling the encoding time though. ↩︎ ↩︎

Secure Torrenting with Docker: Transmission behind WireGuard

web@jneidel.com (Jonathan Neidel) — Mon, 02 Oct 2023 00:00:00 +0000

A thorough guide of my torrenting setup. You will find everything here to set this up for yourself. It includes step by step instructions, as well as the complete scripts I use.

It’s not specific to my VPN provider¹ or server OS, though I will use them as examples.

WireGuard #

Obtain the config #

Go to the website of your VPN provider website and generate a WireGuard configuration file.

For Mullvad

Go into the WireGuard configuration section inside your account and select these options:

For AirVPN

Go to the AirVPN client area and into the config generator, select these options:

Save the result as wg0.conf. It will look a little like this:

[Interface]
Address = 10.152.173.151/32,fd7d:76ee:e68f:a993:d065:246a:a61f:1dcf/128
PrivateKey = uOHkXPjTDL9W0KCOh/DjZRlaa+AORPin/A7e40Fc8G8=
Address = 10.66.170.182/32
MTU = 1320
DNS = 10.128.0.1, fd7d:76ee:e68f:a993::1

[Peer]
PublicKey = nAF0wrLG2+avwQfqxnXhBGPUBCvc3QCqWKH4nK5PfEU=
PresharedKey = nAF0wrLG2+avwQfqxnXhBGPUBCvc3QCqWKH4nK5PfEU=
Endpoint = europe3.vpn.airdns.org:1637
AllowedIPs = 0.0.0.0/0,::/0
PersistentKeepalive = 15

Docker container #

Before we start the container, we need to think about where to put the configuration. In most OS’s you can put them in /opt/docker/volumes/wireguard/config, but on Unraid that directory will not survive a reboot, which is why I put it in /mnt/cache/appdata/wireguard. Choose a directory and put your wg0.conf in it.

Then start the container² (replace the volume accordingly):

docker run -d --name="wireguard" \
 --cap-add NET_ADMIN \
 --cap-add SYS_MODULE \
 --sysctl net.ipv4.conf.all.src_valid_mark=1 \
 --volume /mnt/cache/appdata/wireguard/wg0.conf:/etc/wireguard/wg0.conf \
 --publish 51820:51820/udp \
 --publish 9091:9091 \
 --restart=unless-stopped \
 --pull=always \
 jordanpotter/wireguard

If it tells you that the named container already exists, you can remove it like this:

docker stop wireguard 2>/dev/null
docker rm wireguard 2>/dev/null

Debugging #

Look the logs:

docker logs wireguard

Success would look something like this:

docker logs wireguard
[#] ip link add dev wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.152.173.151/32 dev wg0
[#] ip -6 address add fd7d:76ee:e68f:a993:d065:246a:a61f:1dcf/128 dev wg0
[#] ip link set mtu 1320 up dev wg0
[#] resolvconf -a wg0 -m 0 -x
could not detect a useable init system
[#] wg set wg0 fwmark 51820
[#] ip -6 rule add not fwmark 51820 table 51820
[#] ip -6 rule add table main suppress_prefixlength 0
[#] ip -6 route add ::/0 dev wg0 table 51820
[#] ip6tables-restore -n
[#] ip -4 rule add not fwmark 51820 table 51820
[#] ip -4 rule add table main suppress_prefixlength 0
[#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820
[#] echo Skipping setting net.ipv4.conf.all.src_valid_mark
Skipping setting net.ipv4.conf.all.src_valid_mark
[#] iptables-restore -n
IPv6 interface not found, skipping IPv6 kill switch

If it says ‘Could not resolve host’, try restarting the container:

docker restart wireguard

Try to generate another config or remove the ipv6 entries from your config.

Test connectivity #

We want to make sure the your wireguard container masks any of your requests by routing them through the configured VPN connection. We can test this with the service ipinfo. They will tell you what public IP made the request. Make a request against the service through the wireguard network:

printf "ifconfig.io: "
docker run --rm --net=container:wireguard docker.io/curlimages/curl -Ss ifconfig.io
printf "ipinfo.io: "
docker run --rm --net=container:wireguard docker.io/curlimages/curl -Ss ipinfo.io

The output will look like this:

{
 "ip": "185.209.196.210",
 "city": "Frankfurt am Main",
 "region": "Hesse",
 "country": "DE",
 "loc": "50.1155,8.6842",
 "org": "AS39351 31173 Services AB",
 "postal": "60306",
 "timezone": "Europe/Berlin",
 "readme": "https://ipinfo.io/missingauth"
}

The IP you get should be different from you home address:

curl -Ss https://ipinfo.io

Enable port forwarding #

If your VPN provider supports port forwarding (most do not - AirVPN does), activate it on a port of your choosing:

Activate port forwarding on AirVPN

In the client area under Ports you can add a new port:

Put these lines in your wireguard startup script to let your containers use the ports configured with your VPN provider:

--publish 29142:29142/udp \
--publish 29142:29142 \

Transmission #

Transmission, as your torrent client, will route all of it’s traffic through the previously established --network=container:wireguard.

Docker container #

Decide where to store configs and downloaded files. I use the Unraid conventions.

Then start the container:

docker run -d --name="transmission" \
 --volume /mnt/cache/appdata/transmission:/config \
 --volume /mnt/user/media/torrents:/downloads \
 --network=container:wireguard \
 --env PUID=99 \
 --env PGID=100 \
 --env USER=transmission \
 --env PASS=1234 \
 --env TZ="Europe/Berlin" \
 --restart=unless-stopped \
 linuxserver/transmission

The line --network=container:wireguard is the crux of this whole setup. All of transmissions traffic is routed through that container, which in turn routes it through your VPN provider. Your local IP will not be exposed should the VPN connection fails for whatever reason.

You can test this by making a request through the container. You will observe that the same IP is being used as for requests made through the wireguard container:

docker exec transmission sh -c 'curl -Ss https://ipinfo.io'

If you have configured port forwarding earlier, activate it for transmission by adding --env PEERPORT=29142 to the startup script.

Install a web UI: flood #

I use flood as my web interface for transmission. See the list of browser-based remote controls for other options. Flood comes in its own container:

docker run -d --name=tflood \
 -p 3000:3000 \
 -e HOME=/config \
	--volume /mnt/user/appdata/flood:/config \
 jesec/flood

Flood is now available on the port 3000. For flood to be able to access the transmission behind wireguard you will need:

the wireguard container to be available on the (local) network
have transmissions interface port be exposed by the wireguard container

For 1. have a look at the wireguard container documentation section about just that. For my part I bind the app to an ip on the local network with: --net='br0' --ip='192.168.178.7'.

As for 2., this was already taken care of in previous section by: --publish 9091:9091.

Upon reaching floods initial authentication page use these values:

client: Transmission (duh)
url: http://localhost:9091/transmission/rpc (or your configured ip instead of localhost)
transmission user: transmission (or your choice in transmissions --env USER=)
transmission password: 1234 (definitely change this in --env PASS=)
flood user: any name
flood password: any password

You can now test transmissions ability to download and seed. For example by adding the archlinux iso.

Putting it all together #

On my server I have scripts to (re)start each container and a script that (re)starts them in order. These scripts include a lot of quirky little steps that might not be necessary, but were added over time to make it run more smoothly.

Below is exactly what I am running minus the secrets. It’s just here so you have an example of what the whole thing looks like for me.

wireguard:

#! /bin/sh

docker stop wireguard 2>/dev/null
docker rm wireguard 2>/dev/null
docker run -d --name="wireguard" \
 --cap-add NET_ADMIN \
 --cap-add SYS_MODULE \
 --sysctl net.ipv4.conf.all.src_valid_mark=1 \
 --volume /mnt/cache/appdata/wireguard/wg_confs/airvpn_eu.conf:/etc/wireguard/wg0.conf \
 --publish 51820:51820/udp \
 --publish 29142:29142/udp \
 --publish 29142:29142 \
 --publish 9091:9091 \
 --net='br0' --ip='192.168.178.7' \
 --label net.unraid.docker.icon="https://raw.githubusercontent.com/NightMeer/Unraid-Docker-Templates/main/images/wireguard.png" \
 --restart=unless-stopped \
 --pull=always \
 jordanpotter/wireguard

sleep 5
printf "ifconfig.io: "
docker run --rm --net=container:wireguard docker.io/curlimages/curl -Ss ifconfig.io
printf "ipinfo.io: "
docker run --rm --net=container:wireguard docker.io/curlimages/curl -Ss ipinfo.io

transmission:

#! /bin/sh

docker stop transmission 2>/dev/null
docker rm transmission 2>/dev/null
docker run -d --name="transmission" \
 --volume /mnt/cache/appdata/transmission:/config \
 --volume /mnt/user/media/torrents:/downloads \
 --volume /mnt/user/media/torrents/complete:/done \
 --volume /mnt/user/syncthing/media/torrents:/downloads/watch/sync \
 --volume /mnt/user/media/movies/torrents:/movie \
 --volume /mnt/user/media/documentaries/torrents:/docu \
 --volume /mnt/user/media/series:/series \
 --volume /mnt/user/media/myseries:/docuseries \
 --volume /mnt/user/media/standup:/standup \
 --volume /mnt/user/media/requests:/req \
 --volume /mnt/user/media/requests-series:/reqs \
 --volume "/mnt/user/media/audiobooks/non-fiction audio":/nonfic \
 --volume "/mnt/user/media/audiobooks/fiction audio":/fic \
 --volume /mnt/user/media/books/fiction:/booksfic \
 --volume /mnt/user/media/books/books:/books \
 --volume /mnt/user/syncthing/media/_inbox:/dl \
 --network=container:wireguard \
 --env PUID=99 \
 --env PGID=100 \
 --env USER=transmission \
 --env PASS=__mypasshere__ \
 --env TRANSMISSION_WEB_HOME=/config/web/transmission-web-control \
 --env PEERPORT=29142 \
 --env TZ="Europe/Berlin" \
 --label net.unraid.docker.icon="https://raw.githubusercontent.com/linuxserver/docker-templates/master/linuxserver.io/img/transmission-logo.png" \
 --restart=unless-stopped \
 linuxserver/transmission

sleep 5
docker exec transmission sh -c 'curl -Ss https://ipinfo.io'

Nginx proxy manager provides a local domain for the rarely used transmission web control on “192.168.178.7:9091”. I can also access transmission through the command line: transmission-remote 192.168.178.7:9091 --auth=$TRANSMISSION_USER:$TRANSMISSION_PASS. See my related aliases and scripts.

flood:

#! /bin/sh

docker stop tflood 2>/dev/null
docker rm tflood 2>/dev/null
docker run -d --name=tflood \
	-p 3000:3000 \
	-e HOME=/config \
	--volume /mnt/user/appdata/flood:/config \
	--net='br0' --ip='192.168.178.10' \
	--label net.unraid.docker.icon="https://hotio.dev/webhook-avatars/flood.png" \
	jesec/flood:4.9.3
# latest (4.9.5 was broken)

Nginx proxy manager provides a local domain for flood running on “192.168.178.10:3000”.

Now all together. start:

#!/bin/sh

./wireguard
./transmission && ./mam-ip
./flood

mam-ip is a script of mine, that lets a private tracker know of my new IP address.

Conclusions #

That’s it folks. This should give you all you need to reproduce my setup for yourself. Enjoy:slightly_smiling_face:

References #

Originally this setup was based on this reddit post. Many adjustments were made.

Other torrent daemons

You can easily replace transmission with any other containerized torrent daemon. The wireguard container network allows for it and you should have everything here that you need.

Other ways of testing connectivity:

In the comments of reddit post there are also other ways of testing connectivity described. Though, I’m just fine with ipinfo.

AirVPN has a proven track record of not handing over user data (they can’t hand over something they don’t have.) They are one of the few providers of that caliber, which still supports port forwarding. You can pay in crypto and test it for three days (2€). ↩︎
The image I use is jordanpotter/wireguard. Unlike linuxserver/wireguard it has a kill switch (see linuxserver issue), has working ipv6 and is easier to configure. ↩︎

Technical guides on jneidel

Completion of hugo links in Emacs

References #

Automatic Backups of my GitHub Repositories

Requirements #

Setup #

Conclusions #

Consistent Keybindings Across Linux and macOS

Why setup consistent keybindings? #

Solution concepts #

1) Firmware: separate keyboard by OS #

2) Translate keys on the Mac #

Solution implementation #

Copy-paste #

Browser navigation #

Window management #

Restrict translations to external keyboards #

The next step #

Conclusions #

References #

How I add automatic updates to my now page

Why not update it by hand? #

Contents of my now page #

Design #

Implementation #

Extract and provide data #

Include data in /now #

Run script on USB-C Dock connect/disconnect under Linux

Udev rule to match device #

More specific rule criteria #

RUN vs starting a systemd service #

My rules #

The script to run #

Demo #

References #

Add Hyper Modifier Key for Emacs on Linux

What make adding a hyper key useful? #

Add your own hyper key #

The tools and the logic #

Implementation #

References #

Utilizing git rebase for a clean commit history

Definitions #

Avoid merge commits #

Implications of rebasing #

Interactive rebase #

Get into a rebase #

Remove commits #

Change commits #

Combine commits #

Reorder commits #

Automatic stashing #

Oh No, something went wrong #

Fixup #

Conclusions #

Migrating the technical documentation of a legacy project to Arc42

More on the project #

Motivation #

ROI for the company #

Requirements #

arc42 #

What is arc42? #

The sections in more details #

Why we chose arc42 #

Implementation #

Talking to stakeholders #

Agreed upon implementation #

Structuring and migration #

Gathering existing documentation #

Grouping into sections #

Cleanup and post-processing #

Processes and future work #

Measures we put into place #

Concrete details #

Conclusions #

How to make use of Audio and Video Transcoding

Relevance #

Codecs #

Picking a video codec #

H.264 (AVC) #