Skip to content

Agent Internals

Subsystems

These subsystems are initialized and started by plexd up:

SubsystemPackageReference
Control Plane Clientinternal/apiControl Plane Client
Registrationinternal/registrationRegistration
Event Verificationinternal/api (Ed25519Verifier)Event Verification
WireGuardinternal/wireguardWireGuard Tunnel Management
NAT Traversalinternal/natNAT Traversal via STUN
Peer Exchangeinternal/peerexchangePeer Endpoint Exchange
Network Policyinternal/policyNetwork Policy Enforcement
Secure Tunnelinginternal/tunnelSecure Access Tunneling
SSE Managerinternal/api (SSEManager)Control Plane Client
Reconcilerinternal/reconcileConfiguration Reconciliation
Heartbeatinternal/agent (HeartbeatService)Heartbeat Service
Integrityinternal/integrityIntegrity Verification
Actionsinternal/actionsRemote Actions and Hooks
Node APIinternal/nodeapiLocal Node API
Metricsinternal/metricsMetrics Collection & Reporting
Log Forwardinginternal/logfwdLog Forwarding
Audit Forwardinginternal/auditfwdAudit Forwarding
Bridge (bridge mode)internal/bridgeBridge Mode

Startup Sequence (plexd up)

The runUp function in cmd/plexd/cmd/up.go performs initialization before entering steady state:

Initialization:

  1. Parse config — read YAML, apply CLI flag overrides, apply PLEXD_* env overrides
  2. Set up logger — structured slog logger with configured level
  3. Create control plane clientapi.NewControlPlane() with API config and build version
  4. Registerregistrar.Register() loads or creates node identity (fatal on failure)
  5. Create Ed25519 verifier — decode the control plane's signing public key from identity 5a. Initialize WireGuard — create WireGuard interface, configure address and MTU, bring interface up 5b. Initialize NAT + Peer Exchange — create STUN client, NAT discoverer, and peer exchanger 5c. Initialize Network Policy — create policy engine, firewall controller, and enforcer 5d. Initialize Tunnel — load/generate host key, create JWT verifier and mesh server 5e. Initialize Bridge (bridge mode only) — create bridge manager, ACME, ingress, user access, site-to-site managers; run Setup on each
  6. Create SSE managerapi.NewSSEManager() with handlers for signing key rotation, WireGuard peer events, tunnel events, policy events, and bridge events (bridge mode)
  7. Create reconcilerreconcile.NewReconciler() with handlers for WireGuard, policy, and bridge reconciliation
  8. Create heartbeat serviceagent.NewHeartbeatService() with subsystem status enrichment, auth-failure callback, and key-rotation callback
  9. Create integrity store + verifierintegrity.NewStore() and integrity.NewVerifier() for hook checksums
  10. Create action executoractions.NewExecutor(), register 11 built-in actions, register action_request SSE handler, report capabilities to control plane
  11. Create hook watcheractions.NewHookWatcher() for filesystem hook scanning
  12. Create node API servernodeapi.NewServer(), wire action provider, hook reloader, and reconcile handler
  13. Create metrics collectors + manager — system collector, agent stats collector, metrics.NewManager()
  14. Create log sources + forwarder — journald source, file sources from file_patterns, logfwd.NewForwarder()
  15. Create audit sources + forwarder — process source, auditfwd.NewForwarder()

Goroutines (10 in node mode, 11 in bridge mode):

After initialization, goroutines are started via a sync.WaitGroup:

  1. SSE ManagersseMgr.Start() — event stream connection
  2. Heartbeatheartbeat.Run() — periodic heartbeats
  3. Reconcilerreconciler.Run() — periodic state reconciliation
  4. Node APInodeAPISrv.Start() — Unix socket + optional HTTP server
  5. Hook WatcherhookWatcher.Watch() — filesystem watching for hook changes
  6. Metrics ManagermetricsMgr.Run() — collect and report metrics
  7. Log ForwarderlogForwarder.Run() — collect and forward logs
  8. Audit ForwarderauditForwarder.Run() — collect and forward audit events
  9. Peer Exchangeexchanger.Run() — STUN discovery and endpoint exchange loop
  10. Mesh ServermeshServer.Start() — tunnel SSH server
  11. Bridge Relay (bridge mode only) — bridgeMgr.StartRelay() — UDP relay listener

Shutdown Sequence

On SIGTERM or SIGINT:

  1. Context cancel — the signal-notify context is cancelled, which signals all goroutines to stop
  2. sseMgr.Shutdown() — close the SSE connection
  3. executor.Shutdown() — drain running actions
  4. meshServer.Shutdown() — stop tunnel SSH server and drain sessions
  5. ingressMgr.Teardown() — close ingress listeners (bridge mode)
  6. userAccessMgr.Teardown() — remove user access peers and interface (bridge mode)
  7. s2sMgr.Teardown() — remove site-to-site tunnels and interfaces (bridge mode)
  8. acmeMgr.Teardown() — stop ACME certificate management (bridge mode)
  9. bridgeMgr.Teardown() — remove bridge routes and forwarding (bridge mode)
  10. enforcer.Teardown() — remove firewall chains and rules
  11. wgMgr.Teardown() — delete WireGuard interface (last, others depend on it)
  12. wg.Wait() with 30s timeout — wait for all goroutines to exit; force exit if timeout exceeded

Authentication Flow

  1. Bootstrap — node registers using a bootstrap token (file, env var, or metadata service)
  2. Node Secret Key — registration returns a NodeSecretKey used for all subsequent API authentication
  3. Heartbeat Auth — each heartbeat uses the node secret key; on 401 Unauthorized, the onAuthFailure callback triggers re-registration
  4. Key Rotation — on signing_key_rotated SSE events or heartbeat RotateKeys flag, the Ed25519 verifier keys are updated

SSE Event Types

The SSE manager processes these event types:

EventHandlerDescription
signing_key_rotatedUpdates Ed25519 verifier keysFired when the control plane rotates its signing key pair
action_requestDispatches to action executorRequests execution of a built-in action or hook
peer_addedwireguard.HandlePeerAddedAdds a new WireGuard peer
peer_removedwireguard.HandlePeerRemovedRemoves a WireGuard peer
peer_key_rotatedwireguard.HandlePeerKeyRotatedRotates a peer's public key
peer_endpoint_changedwireguard.HandlePeerEndpointChangedUpdates a peer's endpoint (via peer exchange)
policy_updatedpolicy.HandlePolicyUpdatedTriggers policy reconciliation
ssh_session_setuptunnel.HandleSSHSessionSetupCreates a new tunnel session
session_revokedtunnel.HandleSessionRevokedCloses a revoked tunnel session
bridge_config_updatedbridge.HandleBridgeConfigUpdatedTriggers bridge reconciliation (bridge mode)
relay_session_assignedbridge.HandleRelaySessionAssignedAssigns a relay session (bridge mode)
relay_session_revokedbridge.HandleRelaySessionRevokedRevokes a relay session (bridge mode)
ingress_rule_assignedbridge.HandleIngressRuleAssignedAssigns an ingress rule (bridge mode)
ingress_rule_revokedbridge.HandleIngressRuleRevokedRevokes an ingress rule (bridge mode)
ingress_config_updatedbridge.HandleIngressConfigUpdatedTriggers ingress reconciliation (bridge mode)
user_access_peer_assignedbridge.HandleUserAccessPeerAssignedAssigns a user access peer (bridge mode)
user_access_peer_revokedbridge.HandleUserAccessPeerRevokedRevokes a user access peer (bridge mode)
user_access_config_updatedbridge.HandleUserAccessConfigUpdatedTriggers user access reconciliation (bridge mode)
site_to_site_tunnel_assignedbridge.HandleSiteToSiteTunnelAssignedAssigns a site-to-site tunnel (bridge mode)
site_to_site_tunnel_revokedbridge.HandleSiteToSiteTunnelRevokedRevokes a site-to-site tunnel (bridge mode)
site_to_site_config_updatedbridge.HandleSiteToSiteConfigUpdatedTriggers site-to-site reconciliation (bridge mode)

See Also