Kubernetes DaemonSet Deployment Reference

Reference documentation for deploying plexd as a Kubernetes DaemonSet. Covers environment detection, configuration, CRD controller, audit log reader, authentication, and manifest structure.

KubernetesEnvironment

KubernetesEnvironment holds metadata about the Kubernetes environment in which plexd is running. A nil value from Detect() indicates the process is not running inside a pod.

type KubernetesEnvironment struct {
    InCluster           bool
    Namespace           string
    PodName             string
    NodeName            string
    ServiceAccountToken string
}

Field	Source
InCluster	true if KUBERNETES_SERVICE_HOST env var is set
Namespace	Read from /var/run/secrets/kubernetes.io/serviceaccount/namespace
PodName	HOSTNAME env var
NodeName	MY_NODE_NAME env var (set via downward API)
ServiceAccountToken	/var/run/secrets/kubernetes.io/serviceaccount/token

EnvironmentDetector interface

type EnvironmentDetector interface {
    Detect() *KubernetesEnvironment
}

DefaultDetector implements this interface using real environment variables and filesystem paths.

Config

Configuration for Kubernetes integration. Passed as a constructor argument; no file I/O in this package.

Field	Type	Default	Description
Enabled	bool	false	Must be explicitly enabled
CRDEnabled	bool	true (when Enabled)	Whether CRD management is active
Namespace	string	(empty — auto-detected)	Override namespace
AuditLogPath	string	/var/log/kubernetes/audit/audit.log	Path to Kubernetes audit log
CRDSyncInterval	time.Duration	10s	Interval for CRD state reconciliation
TokenPath	string	/var/run/secrets/kubernetes.io/serviceaccount/token	Path to service account token

Methods

• ApplyDefaults(env *KubernetesEnvironment) — Sets default values for zero-valued fields. When env is non-nil and InCluster is true, sets Enabled=true, CRDEnabled=true, and auto-detects Namespace.
• Validate() error — Skips validation when Enabled is false. Validates Namespace non-empty, AuditLogPath non-empty, CRDSyncInterval >= 1s, TokenPath non-empty.

PlexdNodeState CRD

The PlexdNodeState custom resource definition exposes node state to the Kubernetes API.

type PlexdNodeState struct {
    Name            string               `json:"name"`
    Namespace       string               `json:"namespace"`
    UID             string               `json:"uid,omitempty"`
    ResourceVersion string               `json:"resourceVersion,omitempty"`
    Labels          map[string]string    `json:"labels,omitempty"`
    Spec            PlexdNodeStateSpec   `json:"spec"`
    Status          PlexdNodeStateStatus `json:"status,omitempty"`
    LastUpdate      time.Time            `json:"lastUpdate"`
}

type PlexdNodeStateSpec struct {
    NodeID     string            `json:"nodeId"`
    MeshIP     string            `json:"meshIp,omitempty"`
    Metadata   map[string]string `json:"metadata,omitempty"`
    Data       []DataEntry       `json:"data,omitempty"`
    SecretRefs []SecretRef       `json:"secretRefs,omitempty"`
}

type PlexdNodeStateStatus struct {
    Report []DataEntry `json:"report,omitempty"`
}

CRD manifest: deploy/kubernetes/crds/plexdnodestate-crd.yaml

Property	Value
API group	plexd.plexsphere.com
Kind	PlexdNodeState
Plural	plexdnodestates
Short name	pns
Scope	Namespaced
Version	v1alpha1

Printer columns: Node ID, Mesh IP, Age.

KubeClient interface

Abstracts Kubernetes API interactions for testability. All state-modifying methods are idempotent.

type KubeClient interface {
    GetNodeState(ctx context.Context, namespace, name string) (*PlexdNodeState, error)
    CreateNodeState(ctx context.Context, state *PlexdNodeState) error
    UpdateNodeState(ctx context.Context, state *PlexdNodeState) error
    DeleteNodeState(ctx context.Context, namespace, name string) error
    WatchNodeState(ctx context.Context, namespace, name string) (<-chan PlexdNodeStateEvent, error)
    CreateSecret(ctx context.Context, secret *KubeSecret) error
    UpdateSecret(ctx context.Context, secret *KubeSecret) error
    DeleteSecret(ctx context.Context, namespace, name string) error
    WatchPlexdHooks(ctx context.Context, namespace string) (<-chan PlexdHookEvent, error)
    UpdatePlexdHookStatus(ctx context.Context, hook *PlexdHook) error
    CreateJob(ctx context.Context, job *PlexdJob) error
}

Sentinel errors

Error	Value	Description
ErrNotFound	kubernetes: resource not found	Resource does not exist
ErrAlreadyExists	kubernetes: resource already exists	Resource already exists
ErrUnauthorized	kubernetes: unauthorized	Client lacks valid credentials

CRDController

Periodically syncs local node state to a PlexdNodeState CRD resource.

type CRDController struct {
    client         KubeClient
    reportNotifier ReportNotifier
    cfg            Config
    namespace      string
    nodeID         string
    meshIP         string
    resourceName   string // "node-{node_id}"
    logger         *slog.Logger
}

Constructor

func NewCRDController(client KubeClient, cfg Config, nodeID, meshIP, namespace string, reportNotifier ReportNotifier, logger *slog.Logger) *CRDController

Config defaults are applied automatically.

ReportNotifier interface

type ReportNotifier interface {
    NotifyChange()
}

Called when the CRDController detects new or changed report entries in the status subresource.

Start

func (c *CRDController) Start(ctx context.Context) error

Creates or updates the PlexdNodeState resource and starts the status watcher goroutine. Blocks until ctx is cancelled.

Start behavior

1. Ensures the CRD resource exists (create or update)
2. Starts a watch on the resource for status changes
3. When status report entries change, calls ReportNotifier.NotifyChange()
4. Blocks until context is cancelled

K8sAuditLogReader

Implements auditfwd.K8sAuditReader by reading Kubernetes audit log files in JSON-lines format.

type K8sAuditLogReader struct {
    path   string
    offset int64
    logger *slog.Logger
}

Constructor

func NewK8sAuditLogReader(path string, logger *slog.Logger) *K8sAuditLogReader

ReadEvents

func (r *K8sAuditLogReader) ReadEvents(ctx context.Context) ([]auditfwd.K8sAuditEntry, error)

Behavior:

• Reads from r.offset to end of file; only new entries since last call are returned
• If the file does not exist or cannot be opened, returns an error
• If the file was truncated (offset > file size), resets offset to 0 and reads from the beginning
• Malformed JSON lines are skipped with a warning log
• Timestamp is parsed from stageTimestamp, falling back to requestReceivedTimestamp
• objectRef and responseStatus may be nil in the audit event; handled gracefully

Audit log format (JSON-lines):

Each line is a Kubernetes audit event:

{"kind":"Event","verb":"create","user":{"username":"admin"},"objectRef":{"resource":"pods","namespace":"default","name":"web-1"},"responseStatus":{"code":201},"stageTimestamp":"2025-01-15T10:30:01.000000Z"}

TokenReviewAuthenticator

Validates Kubernetes service account tokens via the TokenReview API.

type TokenReviewAuthenticator struct {
    client    TokenReviewClient
    logger    *slog.Logger
    audiences []string
}

Constructor

func NewTokenReviewAuthenticator(client TokenReviewClient, logger *slog.Logger, audiences []string) *TokenReviewAuthenticator

Authenticate

func (a *TokenReviewAuthenticator) Authenticate(ctx context.Context, token string) (*TokenReviewResult, error)

Condition	Result
Empty token	Error: kubernetes: authenticate: empty token
Client error	Error: kubernetes: authenticate: review failed
Not authenticated	Error: kubernetes: authenticate: token not authenticated
Audience mismatch	Error: kubernetes: authenticate: audience mismatch
Success	Returns *TokenReviewResult

HTTPTokenReviewClient

Production implementation that calls the Kubernetes API server:

func NewHTTPTokenReviewClient(apiServer, saTokenPath string) *HTTPTokenReviewClient

• Posts to {apiServer}/apis/authentication.k8s.io/v1/tokenreviews
• Authenticates with the service account token read from saTokenPath
• Loads cluster CA from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
• TLS minimum version: 1.2

Kubernetes manifests

All manifests are in deploy/kubernetes/.

File	Resources
namespace.yaml	Namespace (plexd-system)
crds/plexdnodestate-crd.yaml	CustomResourceDefinition (PlexdNodeState)
crds/plexdhook-crd.yaml	CustomResourceDefinition (PlexdHook)
serviceaccount.yaml	ServiceAccount
rbac.yaml	ClusterRole + ClusterRoleBinding + consumer roles
daemonset.yaml	DaemonSet with host networking and capabilities
secret.yaml	Example Secret template for bootstrap token

RBAC permissions

The plexd ClusterRole grants:

API Group	Resource	Verbs
plexd.plexsphere.com	plexdnodestates	get, list, watch, create, update, patch, delete
plexd.plexsphere.com	plexdnodestates/status	get, patch, update
(core)	secrets	create, get, update, delete
plexd.plexsphere.com	plexdhooks	get, list, watch
plexd.plexsphere.com	plexdhooks/status	get, update, patch
batch	jobs	create, get, list, watch
authentication.k8s.io	tokenreviews	create

Consumer RBAC roles:

Role	Resources	Verbs
plexd-state-reader	plexdnodestates	get, list, watch
plexd-state-reporter	plexdnodestates/status	get, patch
plexd-secrets-reader	secrets	get
plexd-hook-reader	plexdhooks	get, list, watch

DaemonSet configuration

Setting	Value	Reason
hostNetwork	true	WireGuard mesh requires host networking
dnsPolicy	ClusterFirstWithHostNet	Required with hostNetwork
priorityClassName	system-node-critical	Ensures scheduling on resource pressure
tolerations	operator: Exists	Run on all nodes including control plane
readOnlyRootFilesystem	true	Security hardening
Capabilities	NET_ADMIN, NET_RAW	WireGuard interface management

Environment variables

Variable	Source	Description
MY_NODE_NAME	fieldRef: spec.nodeName	Kubernetes node name (downward API)
PLEXD_BOOTSTRAP_TOKEN	secretKeyRef: plexd-bootstrap	Bootstrap token from Secret

Volume mounts

Mount path	Source	Access
/etc/plexd	ConfigMap plexd-config	read-only
/var/lib/plexd	hostPath	read-write
/var/run/plexd	hostPath	read-write
/var/log/kubernetes/audit	hostPath	read-only

Constants

Constant	Value
ServiceAccountBasePath	/var/run/secrets/kubernetes.io/serviceaccount
DefaultTokenPath	{ServiceAccountBasePath}/token
DefaultNamespacePath	{ServiceAccountBasePath}/namespace
DefaultCACertPath	{ServiceAccountBasePath}/ca.crt
DefaultAuditLogPath	/var/log/kubernetes/audit/audit.log
DefaultCRDSyncInterval	10s

See also

• Kubernetes Deployment Guide — Step-by-step deployment guide
• PlexdHook CRD Reference — Hook execution via Kubernetes Jobs
• Audit Forwarding Reference — Audit data collection and forwarding
• Local Node API Reference — Node state API
• Registration Reference — Node registration