gKeyMapper/docs/daemon-plan.md

gKeyMapper daemon — implementation plan

Status: implemented (M0–M4 done, see "Implementation status") Scope: a native C/C++ background service that detects G1–G9 (and later M1–M3/MR and media keys) on the Logitech G910 and replays the mappings stored by the GUI.

Implementation status

• M0 done — shared/mapping.{h,cpp} (file format + shortcut parsing) and shared/keys.{h,cpp} (canonical names → Linux codes); the GUI uses them. Round-trip and shortcut parsing are covered by a small standalone test.
• M1 done — daemon/ builds (gkeymapper-daemon): device discovery, kernel-driver detach, F-key binding disable, 64-byte interrupt reads and G1–G9 decoding. Verified on hardware: all nine keys decode correctly (press and release), in order.
• M2 done — action execution: key shortcuts are replayed through a /dev/uinput virtual keyboard, command runs via fork/exec (with a shell fallback so arguments work), disabled does nothing. Verified on hardware.
• M3 done — config watching and lifecycle: daemon/config_watch watches the config directory with inotify, so a save in the GUI reloads within about half a second; SIGHUP/SIGUSR1 stay as the fallback. Save mappings now writes atomically (temp file + rename), so the daemon never reads a half-written file. make install ships the udev rule, the modules-load.d fragment and the systemd user unit (data/).
• Layout handling (added during M2) — key names are resolved through the active layout (XKBLAYOUT from /etc/default/keyboard, via xkbcommon). This matters because the US-position code KEY_Z (44) is y on a German keyboard; z is now correctly resolved to evdev 21.
• AltGr (added after M2) — level-3 combinations are representable as altgr+7; the base (level 0) key is stored plus the altgr modifier.
• M4 done — D-Bus control channel: daemon/ipc_dbus registers dev.rager.gkeymapper.Daemon on the session bus and serves Reload, GetStatus and TestMapping, plus the KeyPressed, ConfigReloaded and DeviceChanged signals. src/daemonclient.{h,cpp} is the GUI side: the footer shows Daemon: running/not reachable, the device and the last G key that fired, and Save mappings calls Reload() (the file watch stays the backstop). make install ships a D-Bus activation file, so the GUI can start the daemon on demand; the unit is Type=dbus with BusName=. A second instance refuses to start instead of fighting over the keyboard. Verified with gdbus (methods, signals, activation) and over AT-SPI (live footer).

1. Goals and non-goals

Goals

• Detect G-key presses in C/C++ and execute the configured action.
• Run with a very small, stable resource footprint (idle ≈ 0 % CPU, few MB RSS, no interpreter, no polling when nothing happens).
• Reuse the existing mappings.conf format as the single source of truth.
• Let the GUI and the daemon run independently; either can start first.
• No Python at runtime.

Non-goals (v1)

• Profiles (M1–M3/MR), per-locale text typing (typeout), embedded scripting.
• GUI redesign beyond a status indicator / reload / test button.

2. What the Python reference does (and what we keep)

Reference: https://github.com/JSubelj/g910-gkey-macro-support

• G-keys are not exposed as distinct evdev key codes. The driver therefore opens the keyboard's vendor HID interface (USB interface 1) directly via libusb, detaches the kernel driver from that interface, and reads raw interrupt-IN reports.
• On start it sends a control transfer to disable the firmware F-key→G-key binding, so pressing G1..G9 no longer also emits F1..F9 on the normal keyboard interface.
• Raw report byte patterns are looked up in tables (events.macroKeys, memoryKeys, mediaKeys, releaseEvents) and turned into actions.
• A uinput virtual keyboard replays shortcut (press, later release), typeout (per-char), uinput (direct KEY_*); run uses /bin/bash -c.
• Config is JSON at ~/.config/g910-gkeys/config.json, reloaded on directory change (inotify via fcntl), plus SIGUSR1 reload.
• Ship: a systemd --user service and udev rules with TAG+="uaccess" for the device and /dev/uinput, so no root is needed.

Relevant USB IDs (from the udev rules): 046d:c335, c33f, c33e, c32b, c24d, c22d. The G910 in this project is 046d:c32b.

What we keep: libusb raw HID access, disabling the F-key binding, uinput output, config-dir watching. What we change: JSON→our line format, single-threaded event-driven loop instead of a Python loop with fixed sleeps, fork/exec reaping, and a real IPC channel to the GUI.

3. Architecture overview

+---------------------+        mappings.conf (shared truth)      +----------------------+
|  gkeymapper (GUI)   | <------------ write / inotify ---------> |  gkeymapper-daemon   |
|  GTK4 + GIO/GDBus   |                                          |  libusb + uinput     |
|                     | <===== D-Bus (session bus) ============> |  GDBus + inotify     |
+---------------------+        status / reload / events          +----------+-----------+
                                                                            | raw HID (iface 1)
                                                                            v
                                                                  +----------------------+
                                                                  | Logitech G910        |
                                                                  +----------+-----------+
                                                                             | uinput
                                                                             v
                                                                  +----------------------+
                                                                  | /dev/uinput keyboard |
                                                                  +----------------------+

Key property: the hot path never crosses IPC. G-key press → decode → uinput happens entirely inside the daemon. IPC (D-Bus) is only used for control, status and notifications, so a slow/absent GUI never delays a keypress.

4. Daemon design

4.1 Process model

• One process, single-threaded, event loop.
• No threads in v1. run/command actions are launched with fork() + execve(); children are reaped via a SIGCHLD/signalfd handler so no zombies accumulate (the reference leaks a Popen per invocation).
• python type is intentionally dropped.

4.2 Event loop

Use one poll()/epoll() loop over:

• the libusb file descriptor (from libusb_get_pollfds(); use libusb's async API or libusb_interrupt_transfer on a worker, see below),
• an inotify fd for the config file/directory,
• a signalfd for SIGTERM/SIGINT/SIGUSR1,
• the D-Bus connection (GDBus integrates with the same GMainContext).

Preferred: drive libusb asynchronously with libusb_submit_transfer() and fold the reported poll fds into the same loop (libusb has an official "poll with another event loop" pattern). That yields exactly 0 wakeups while idle (excluding the kernel's own HID interrupts). If that is too much work for v1, fall back to a blocking libusb_interrupt_transfer(..., timeout=1000) inside a dedicated thread that posts events into the main loop — still cheap (1 wake/s).

To keep GDBus and the fd loop together, the loop is easiest on the GLib main loop (GMainLoop): GDBus, GFileMonitor and GSocket/GSource cover inotify and signalfd, and libusb poll fds can be wrapped in a custom GSource. GLib is already a dependency of the GUI, so this adds no new library. If minimal footprint is the priority over convenience, a raw epoll loop is the alternative; decision point in §11.

4.3 Device layer (libusb)

• Find the device by vendor/product from the supported-device table; pick interface 1, endpoint from the same table.
• libusb_detach_kernel_driver() on interface 1; remember to libusb_attach_kernel_driver() on shutdown (reference does this).
• Send the "disable F-key to G-key binding" control transfer (bmRequestType=0x21, bRequest=0x09, wValue=0x0212, wIndex=1) and consume the confirmation reports; ignore the known LED-return patterns (\x11\xff\x0f, \x11\xff\x10, \x11\xff\xff).
• Hot-plug / re-plug: on LIBUSB_ERROR_NO_DEVICE/IO re-run discovery with a small backoff (reference re-inits in place).
• Optional (later): libusb_hotplug_register_callback to react to device add/remove without polling.

The device table in supported_devices.py (byte patterns per model) is ported to a generated C++ header (constexpr tables) — keep it a build-time artifact so the runtime does zero parsing.

4.4 Decoding

• The report is a fixed-length byte blob. Build a compile-time lookup: a sorted array of {report bytes, action id} with binary search, or a small open-addressed hash keyed on the first 8 bytes. This replaces the Python dict + list(...).index() O(n) scans with O(1)/O(log n) and no allocation.
• Categories: macro (G1..G9), memory (profiles, later), media (later), release (stops a held shortcut), and ignored/known-noise.

4.5 Action layer (uinput)

• Open /dev/uinput once; UI_SET_EVBIT for EV_KEY, register the key set used by the mappings, UI_DEV_CREATE. Keep the fd for the daemon's lifetime.
• Emit events by write()-ing struct input_event (from <linux/uinput.h> / <linux/input-event-codes.h> — no libevdev needed).
• key mapping = shortcut grammar ctrl+c,ctrl+v:
  • single combo: press modifiers+key in order, remember the keys, release them when the keyboard's release event arrives (mirror the reference's press_release_fifo stack).
  • comma list: emit_combo() each group (press+release).
• command mapping: fork(); in the child setsid(), redirect stdio to /dev/null, execve("/bin/sh", "-c", cmd). Reap via SIGCHLD.
• disabled: no-op.
• Give the uinput device a clear name/IDs (e.g. gkeymapper virtual keyboard) so libinput/users can identify it.

4.6 Config

• Parse the existing mappings.conf: G<n>=<type>:<target> with type ∈ {key, command, disabled}.
• Parse at load and on change only; the runtime holds a small pre-parsed action per G-key (enum + fixed target buffer / std::string), so no parsing in the hot path.
• Watch the config file (and, for atomic-save editors, its directory) with inotify/GFileMonitor; reload on IN_CLOSE_WRITE/IN_MOVED_TO. Debounce (e.g. 100 ms) to coalesce editor save bursts.
• Also reload on SIGUSR1 (manual/test) and on Reload() D-Bus call.

4.7 Privileges and packaging

• udev rule with TAG+="uaccess" for the supported USB IDs and for KERNEL=="uinput" (mirrors the reference's 60-g910-gkeys.rules).
• uinput kernel module must be present; ship /etc/modules-load.d/ fragment or document modprobe uinput (here it is built in).
• systemd --user unit gkeymapper-daemon.service: Type=dbus, BusName=dev.rager.gkeymapper.Daemon, ExecStart=gkeymapper-daemon, Restart=on-failure, ExecReload=SIGHUP. Type=dbus + BusName make the unit count as started only once the name is owned — the state D-Bus activation and the GUI see. make install also ships the D-Bus activation file (share/dbus-1/services/dev.rager.gkeymapper.Daemon.service, with SystemdService=gkeymapper-daemon.service), so the bus starts the unit when the GUI asks for the daemon's status and the daemon runs exactly once either way.
• Do not require root. Commands run with the user's privileges (the reference's "executed as root" note is a wiki artifact, not a design goal).

5. GUI ↔ daemon interaction

Three layers, from loosest to tightest:

1. Shared state = the config file (~/.config/gKeyMapper/mappings.conf). The GUI is the writer, the daemon is a watcher/reader. This alone is enough for basic operation and is robust: whichever side starts first, the other converges. The daemon reloads via inotify; no coupling to the GUI process.
2. Control/status = D-Bus on the session bus (GDBus; part of GIO, already linked by the GUI). Used for status, explicit reload, and notifications.
3. Lifecycle: the daemon is a systemd --user service; the GUI can start it implicitly by calling a D-Bus method (activation) and can show whether it is running.

5.1 D-Bus interface (implemented)

• App id: dev.rager.gkeymapper (replaces the placeholder com.example.gkeymapper); it is also the GApplication id.
• Bus name: dev.rager.gkeymapper.Daemon — not the plain app id. GtkApplication owns the app id itself, in order to detect a second GUI instance; a daemon on the same name would make every GUI start look like a duplicate and exit. (The original plan had <app-id> here; that clash was found while implementing M4.)
• Object path: /dev/rager/gkeymapper/Daemon
• Interface: dev.rager.gkeymapper.Daemon

Methods

Method	Signature	Purpose
Reload	() -> ()	Re-read mappings.conf now (GUI "Save" also calls this). Result via the ConfigReloaded signal.
GetStatus	() -> (s version, b running, s device, s configPath)	Status for the GUI footer/indicator. device is "" before the keyboard is open, so the GUI can omit that part.
TestMapping	(u index) -> ()	Fire the mapping of one G key without hardware; same code path as a real press. index is 0-based (0 = G1 … 8 = G9). Fails with a D-Bus error for an out-of-range index or a disabled:/empty row.

Signals

Signal	Signature	Purpose
KeyPressed	(u index, s action)	A G-key fired (GUI highlight/history; also emitted by TestMapping).
ConfigReloaded	(b ok)	Daemon finished reloading.
DeviceChanged	(b present, s device)	Keyboard attached/detached.

Notes

• All are control-plane calls; none is on the keypress path. DaemonService::pump() dispatches once per daemon loop iteration, so a reply can take up to one read timeout (≤500 ms) — fine for status/reload, and the GUI calls asynchronously.
• Reload is best-effort: inotify is the backstop if the call is lost.
• If D-Bus is unavailable, the daemon still works; the GUI degrades to "no status" and "save only". Signals SIGUSR1 (reload) / SIGTERM remain for scripted use, matching the reference.
• SetProfile was dropped: it is reserved for the M-key profiles and would have been a no-op today.
• The daemon requests its name with DO_NOT_QUEUE and exits (code 0, with a message) when another instance owns it, before it touches the device.

5.2 Interaction sequences

Save in the GUI:

GUI: write mappings.conf (atomic: temp + rename)
GUI: call Daemon.Reload()   [optional, for immediacy]
Daemon: inotify IN_MOVED_TO -> (re)parse -> apply
Daemon: emit ConfigReloaded(ok)

Key press:

G910 --raw HID--> Daemon -> decode -> uinput emit / fork+exec
Daemon: emit KeyPressed(index, action)   [GUI only, non-blocking]

Start-up orders:

• Daemon first, GUI later: GUI reads status via D-Bus; no config write needed.
• GUI first, daemon later: GUI writes config; on daemon start it reads the file.

5.3 GUI-side changes (as implemented)

• On "Save mappings": writes the file, then calls Reload() best-effort; the file watch is the backstop, so a lost call is harmless (a save therefore triggers two reloads, both idempotent).
• Footer status line: Daemon: running/not reachable + Device: … + the last G key that fired, fed by GetStatus, the DeviceChanged/ConfigReloaded signals and NameOwnerChanged (so it follows the daemon starting/stopping). The Device: part is omitted while the device is not open yet.
• No "Test" button per row: pressing the real key is the better test while the hardware is at hand, and TestMapping stays available over gdbus/busctl without widening the fixed-width rows.
• The key-name vocabulary matches because the capture dialog already writes lowercase GDK key names and the canonical list lives in shared/keys (§6).

6. Code layout

src/                     # GUI (existing)
  daemonclient.{h,cpp}   # D-Bus client (status/reload/signals)
daemon/                  # as implemented (see §6 above)
  ipc_dbus.{h,cpp}       # GDBus service (control plane)
shared/
  mapping.{h,cpp}        # parse/format mappings.conf (used by GUI and daemon)
  keys.{h,cpp}           # canonical names <-> gdk keyval <-> linux KEY_*
  keymap.{h,cpp}         # layout-aware name <-> input code (xkbcommon)
  ipc_ids.h              # app id, version, D-Bus names
data/
  60-gkeymapper.rules    # udev: uaccess for the device and /dev/uinput
  gkeymapper-uinput.conf # modules-load.d fragment (uinput)
  gkeymapper-daemon.service # systemd user unit (@BINDIR@ filled in on install)
  dev.rager.gkeymapper.Daemon.service # D-Bus activation for the same unit
  icons.gresource.xml    # GUI icons compiled into the binary (GResource)
po/                      # shared catalogs (GUI strings)

Build: the Makefile gets a second target gkeymapper-daemon and install rules for the unit, udev rule, modules-load fragment and activation file. New build dependency: libusb-1.0-0-dev plus libglib2.0-dev (gio-2.0) for the daemon's D-Bus channel.

7. Resource-efficiency measures (summary)

• No Python interpreter; native binary, tables in .rodata.
• Event-driven loop, no idle polling (libusb async + inotify + signalfd; or a 1 s blocking USB read in the simple variant).
• O(1)/O(log n) decode via compile-time tables instead of dict/linear scan.
• One uinput fd for the process lifetime; fixed-size event writes; no per-event heap allocation.
• Config parsed only on load/reload; hot path is table lookups + write().
• fork+exec for programs, children reaped (no zombies), stdio to /dev/null; no shell parsing for key/uinput actions.
• Target footprint: idle 0 % CPU, wakeups only on real HID/DBus/file events, RSS in the low single-digit MB.

Measured after M4 (systemd user unit, daemon idle): ~780 KB RSS, 14 ms CPU since start. The D-Bus control channel adds no wakeups of its own: pump() only dispatches what the loop's read timeout already woke up for.

8. Milestones

• M0 – Shared core. shared/mapping + shared/keys; refactor the GUI to use them (still file-only). Unit tests for parse/format and key tables.
• M1 – Device bring-up. daemon opens the G910 via libusb, disables the F-key binding, logs G1..G9 presses. No output yet.
• M2 – Action layer. uinput keyboard; key (shortcut, incl. release) and command work end-to-end. disabled no-op.
• M3 – Config & lifecycle (done). config reload via inotify (config_watch, directory watch + atomic save) and SIGHUP/SIGUSR1; systemd --user unit + udev rule + modules-load, installed by make install.
• M4 – IPC (done). GDBus service (Reload, GetStatus, TestMapping, signals); GUI status line + Reload-on-save; D-Bus activation file; unit Type=dbus + BusName; single-instance guard.
• M5 – Polish. Packaging/install rules, docs, translations, hardening (hot-plug, debounce, error reporting to the GUI).

9. Testing strategy

• Unit: mapping parser round-trip, key-name tables, decoder against recorded byte sequences from the reference tables.
• Device: replay captured reports into the decoder (no hardware needed).
• Integration: evtest / libinput debug-events to observe the virtual device; compare xev output for a configured shortcut between reference and daemon.
• D-Bus (M4): gdbus call for GetStatus/Reload/TestMapping, gdbus monitor for the signals, and a TestMapping observed with xinput test <keyboard-id> to confirm the emitted keycodes. The GUI's footer line is read back over AT-SPI to check the live status.
• Permission tests: verify the udev rule grants access as a normal user.
• Resource check: powertop/perf or simple wakeup counting while idle.

10. Risks / open points

• Device table size: supported_devices.py is large; porting/generating the byte tables is real work and model-specific.
• Interface detach may momentarily disturb the device; must re-attach on exit and handle crashes gracefully.
• Permissions: relies on TAG+="uaccess" + a user service; must never fall back to requiring root silently.
• command environment: DISPLAY/WAYLAND_DISPLAY/XAUTHORITY and PATH differ between a user service and an interactive shell; decide whether to inherit the user manager environment (systemd --user import-environment) or document run snippets setting DISPLAY=:0 explicitly.
• Key-name vocabulary drift between GUI (GDK names) and daemon (Linux KEY_*) is the most likely source of silent mismatches — that is why the shared table in M0 is mandatory before the daemon consumes the file.
• uinput device identity: a second keyboard appears; choose stable name/vendor/product so the desktop does not confuse it with the real one.
• GUI does not load the config at startup (found while testing M4, GUI-side, not a daemon issue): build_ui() fills the rows empty and only Import configuration... reads the file, so pressing Save mappings in a freshly opened window overwrites the user's mappings with empty ones. Loading ~/.config/gKeyMapper/mappings.conf on start (or refusing to save unchanged rows) should be the next GUI fix.

11. Decisions to make

1. Bus name / app-id: replace com.example.gkeymapper with a real reverse-DNS id (suggested dev.rager.gkeymapper)? Affects the .desktop name too.
2. Event loop: GLib GMainLoop (simplest, reuses GIO) vs. raw epoll (leanest).
3. IPC: D-Bus (idiomatic, GIO already present) vs. a Unix socket (no bus dependency). Recommendation: D-Bus, with signals as a fallback.
4. Scope of v1: keep key/command/disabled only, or also port typeout/media?

Resolved

• Bus name / app id (done in M4). App id (and GApplication id) dev.rager.gkeymapper; the daemon uses dev.rager.gkeymapper.Daemon because GtkApplication owns the plain app id to detect a second GUI instance. The installed .desktop file keeps its name gkeymapper.desktop — both old and new app id yield the X11 WM_CLASS gkeymapper, which is what StartupWMClass matches (verified with xprop).
• IPC (done in M4). D-Bus with GDBus: libdbus and sd-bus dev files are not installed on this machine, GIO is; the daemon links gio-2.0.
• IPC event integration (done in M4). The daemon keeps its simple blocking loop instead of moving to a GMainLoop: DaemonService::pump() calls g_main_context_iteration(ctx, FALSE) once per iteration, so D-Bus is serviced without a thread and without blocking, at the cost of up to one read timeout of latency on the control plane.
• Status line widget (evaluated). The daemon status stays a plain GtkLabel in the window footer. GtkStatusbar does exist in GTK4 and is themed, but it keeps a message stack (pop/push per context id) and GTK4 exposes it over AT-SPI as a generic filler, not as a "statusbar" role — so it was built, compared side by side and dropped.
• Event loop (as implemented, deviates from §4.2). The daemon is single-threaded with a blocking libusb_interrupt_transfer and a 500 ms timeout; the config watch is drained once per iteration. That is ~2 idle wakeups/s at 0 % CPU (the "simple variant" of §4.2) rather than the preferred async libusb + GMainLoop design. Switching to libusb's async API with its poll fds in a poll() loop stays open if idle wakeups ever become a concern — the config watch would fold into the same set.
• Layout handling (done in M2). Key names in the file are layout independent; the daemon resolves them through the active layout (XKBLAYOUT from /etc/default/keyboard, via xkbcommon) and only falls back to the US positions from shared/keys. This closes the vocabulary-drift risk for non-US layouts — a German z is evdev 21, not the US position 44.
• Reload without restart (done in M2, completed in M4). SIGHUP/SIGUSR1 re-read the config file; since M4 the GUI's Save mappings also calls the D-Bus Reload, and the inotify watch applies anything else.
• Config watching (done in M3). The daemon watches the config directory with inotify rather than the file, so an atomic save (temp file + rename) and a recreated file are caught too. The loop iteration (≤ 500 ms) is the coalescing window, so one save yields exactly one reload; SIGHUP/SIGUSR1 stay as the fallback and there is no extra fd in the loop. Save mappings in the GUI now writes atomically, so the daemon can never read a partial file. A failed reload keeps the previous mappings instead of disabling every key.
• Lifecycle (done in M3, extended in M4). A systemd user unit (not a system service): the daemon needs the session for /dev/uinput and for the DISPLAY a command action inherits. Since M4 it is Type=dbus with BusName=dev.rager.gkeymapper.Daemon, and a D-Bus activation file lets the bus start it on demand, so enabling the unit is optional. The udev rule and the modules-load.d fragment live in data/ and are installed by make install (under PREFIX/lib/..., which udev and systemd both search).
• Single instance (done in M4). The daemon requests its bus name with DO_NOT_QUEUE and exits with a message when it is already taken — two instances would detach the kernel driver from each other and create two virtual keyboards. The check happens before the device is opened.