Skip to content

Agent Internals ​

Subsystems ​

These subsystems are initialized and started by plexd up:

SubsystemPackageReference
Control Plane Clientinternal/apiControl Plane Client
Registrationinternal/registrationRegistration
Event Verificationinternal/api (Ed25519Verifier)Event Verification
WireGuardinternal/wireguardWireGuard Tunnel Management
NAT Traversalinternal/natNAT Traversal via STUN
Peer Exchangeinternal/peerexchangePeer Endpoint Exchange
Network Policyinternal/policyNetwork Policy Enforcement
Secure Tunnelinginternal/tunnelSecure Access Tunneling
SSE Managerinternal/api (SSEManager)Control Plane Client
Reconcilerinternal/reconcileConfiguration Reconciliation
Heartbeatinternal/agent (HeartbeatService)Heartbeat Service
Integrityinternal/integrityIntegrity Verification
Actionsinternal/actionsRemote Actions and Hooks
Node APIinternal/nodeapiLocal Node API
Metricsinternal/metricsMetrics Collection & Reporting
Log Forwardinginternal/logfwdLog Forwarding
Audit Forwardinginternal/auditfwdAudit Forwarding
Bridge (bridge mode)internal/bridgeBridge Mode

Startup Sequence (plexd up) ​

The runUp function in cmd/plexd/cmd/up.go performs initialization before entering steady state:

Initialization:

  1. Parse config — read YAML, apply CLI flag overrides, apply PLEXD_* env overrides
  2. Set up logger — structured slog logger with configured level
  3. Create control plane client — api.NewControlPlane() with API config and build version
  4. Register — registrar.Register() loads or creates node identity (fatal on failure)
  5. Create Ed25519 verifier — decode the control plane's signing public key from identity (the key registration returned, or a newer one a state pull carried and the agent persisted); signingKeyDispatchHandler, registered first among the reconcile dispatch handlers, installs the key every state pull carries 5a. Initialize WireGuard — measure the uplink's path MTU with the STUN servers, create WireGuard interface, configure address and MTU, bring interface up. The interface MTU is sized to the uplink unless wireguard.mtu pins it or nat.enabled is false (see MTU detection) 5b. Initialize NAT + Peer Exchange — create STUN client, NAT discoverer, and peer exchanger 5c. Initialize Network Policy — create policy engine, firewall controller, and enforcer 5d. Initialize Tunnel — load/generate host key, create JWT verifier and mesh server 5e. Initialize Bridge (bridge mode only) — create bridge manager, ACME, ingress, user access, site-to-site managers; run Setup on each
  6. Create SSE manager — api.NewSSEManager() with handlers for signing key rotation, WireGuard peer events, tunnel events, policy events, and bridge events (bridge mode)
  7. Create reconciler — reconcile.NewReconciler() with handlers for WireGuard, policy, and bridge reconciliation
  8. Create heartbeat service — agent.NewHeartbeatService() with subsystem status enrichment and key-rotation callback; its credential-refused callback is set in step 12, once the node API server exists
  9. Create integrity store + verifier — integrity.NewStore() and integrity.NewVerifier() for hook checksums
  10. Create action executor — actions.NewExecutor(), register 11 built-in actions, register actions.NewDispatcher() on the reconciler as a dispatch handler, report capabilities to control plane
  11. Create hook watcher — actions.NewHookWatcher() for filesystem hook scanning
  12. Create node API server — nodeapi.NewServer(), wire action provider, hook reloader, and reconcile handler
  13. Create metrics collectors + manager — system collector, agent stats collector, metrics.NewManager()
  14. Create log sources + forwarder — platform log source (journald on Linux, the launchd log file on macOS, the Event Log on Windows), file sources from file_patterns, logfwd.NewForwarder()
  15. Create audit sources + forwarder — process source, auditfwd.NewForwarder()

Goroutines (10 in node mode, 11 in bridge mode):

After initialization, goroutines are started via a sync.WaitGroup:

  1. SSE Manager — sseMgr.Start() — event stream connection
  2. Heartbeat — heartbeat.Run() — periodic heartbeats
  3. Reconciler — reconciler.Run() — periodic state reconciliation
  4. Node API — nodeAPISrv.Start() — local endpoint (a Unix socket, or a named pipe on Windows) + optional HTTP server
  5. Hook Watcher — hookWatcher.Watch() — filesystem watching for hook changes
  6. Metrics Manager — metricsMgr.Run() — collect and report metrics
  7. Log Forwarder — logForwarder.Run() — collect and forward logs
  8. Audit Forwarder — auditForwarder.Run() — collect and forward audit events
  9. Peer Exchange — exchanger.Run() — STUN discovery and endpoint exchange loop
  10. Mesh Server — meshServer.Start() — tunnel SSH server
  11. Bridge Relay (bridge mode only) — bridgeMgr.StartRelay() — UDP relay listener

Shutdown Sequence ​

On SIGTERM or SIGINT:

  1. Context cancel — the signal-notify context is cancelled, which signals all goroutines to stop
  2. sseMgr.Shutdown() — close the SSE connection
  3. executor.Shutdown() — drain running actions
  4. meshServer.Shutdown() — stop tunnel SSH server and drain sessions
  5. ingressMgr.Teardown() — close ingress listeners (bridge mode)
  6. userAccessMgr.Teardown() — remove user access peers and interface (bridge mode)
  7. s2sMgr.Teardown() — remove site-to-site tunnels and interfaces (bridge mode)
  8. acmeMgr.Teardown() — stop ACME certificate management (bridge mode)
  9. bridgeMgr.Teardown() — remove bridge routes and forwarding (bridge mode)
  10. enforcer.Teardown() — remove firewall chains and rules
  11. wgMgr.Teardown() — delete WireGuard interface (last, others depend on it)
  12. wg.Wait() with 30s timeout — wait for all goroutines to exit; force exit if timeout exceeded

Authentication Flow ​

  1. Bootstrap — node registers using a bootstrap token (file, env var, or metadata service)
  2. Node Secret Key — registration returns a NodeSecretKey used for all subsequent API authentication
  3. Heartbeat Auth — each heartbeat uses the node secret key; three heartbeats answered 401 Unauthorized (unauthorized, nsk_invalid or nsk_revoked) with no accepted one between them park the agent, which then stops calling the control plane and probes the credential with the heartbeat (see Credential Park)
  4. Signing Key — the Ed25519 verifier checks SSE envelopes and ssh session tokens against the Domain signing key. Every successful state pull installs the key its signing_key block carries (verifier.Install) and persists it into the identity files, so a restart starts from it. A null block keeps the installed key, and a key the verifier refuses changes nothing. A signing_key_rotated SSE event installs a new key too (verifier.Rotate) and can keep the previous key trusted until the rotation's transition_expires; a pulled key that differs from the current one ends that grace window, because a node trusts exactly one key of its Domain. The mesh SSH server's JWT verifier (step 5d) is separate: it keeps the key decoded at start and picks up a pulled key only after a restart

SSE Event Types ​

The SSE manager processes events in two tiers. Payloads of the reconcile-driving types are opaque — the reconciler's state pull is authoritative, so those events only request a reconcile. session_revoked is the exception: its payload names the session to close before the reconcile.

EventTierHandler
node_state_updatedcontractTriggers a reconcile
policy_updatedcontractpolicy.HandlePolicyUpdated — triggers a reconcile
bridge_config_updatedcontractbridge.HandleBridgeConfigUpdated — triggers a reconcile (bridge mode)
action_requestcontractTriggers a reconcile — the dispatch rides the executions block of the resulting pull
session_setupcontractTriggers a reconcile — the session rides the sessions block of the resulting pull
session_revokedcontractCloses the live session the payload names with the reason it carries (tunnel.Dispatcher.SessionRevokedHandler), then triggers a reconcile; the block drain stays the teardown when the event is missed
node_secrets_updatedcontractTriggers a reconcile; the inventory rides the secrets block of the resulting pull, which feeds the node API's secret index
peer_registereddocumented-comingTriggers a reconcile
peer_psk_assigneddocumented-comingTriggers a reconcile
peer_deregistereddocumented-comingTriggers a reconcile
peer_endpoint_changeddocumented-comingTriggers a reconcile
peer_key_rotateddocumented-comingTriggers a reconcile
rotate_keysdocumented-comingStarts a key rotation (rotator.RotateNow)
signing_key_rotateddocumented-comingUpdates the Ed25519 verifier key (verifier.Rotate); every state pull also installs the key its signing_key block carries (verifier.Install)

Contract types are emitted by the control plane today. Documented-coming types are named so the agent can subscribe once the platform's 14-type taxonomy starts emitting them.

See Also ​