Kubernetes DaemonSet Deployment Reference

Reference documentation for deploying plexd as a Kubernetes DaemonSet. Covers environment detection, configuration, CRD controller, audit log reader, authentication, and manifest structure.

KubernetesEnvironment

KubernetesEnvironment holds metadata about the Kubernetes environment in which plexd is running. A nil value from Detect() indicates the process is not running inside a pod.

type KubernetesEnvironment struct {
    InCluster           bool
    Namespace           string
    PodName             string
    NodeName            string
    ServiceAccountToken string
}

Field	Source
`InCluster`	`true` if `KUBERNETES_SERVICE_HOST` env var is set
`Namespace`	Read from `/var/run/secrets/kubernetes.io/serviceaccount/namespace`
`PodName`	`HOSTNAME` env var
`NodeName`	`MY_NODE_NAME` env var (set via downward API)
`ServiceAccountToken`	`/var/run/secrets/kubernetes.io/serviceaccount/token`

EnvironmentDetector interface

type EnvironmentDetector interface {
    Detect() *KubernetesEnvironment
}

DefaultDetector implements this interface using real environment variables and filesystem paths.

Config

Configuration for Kubernetes integration. Passed as a constructor argument; no file I/O in this package.

Field	Type	Default	Description
`Enabled`	`bool`	`false`	Must be explicitly enabled
`CRDEnabled`	`bool`	`true` (when `Enabled`)	Whether CRD management is active
`Namespace`	`string`	(empty — auto-detected)	Override namespace
`AuditLogPath`	`string`	`/var/log/kubernetes/audit/audit.log`	Path to Kubernetes audit log
`CRDSyncInterval`	`time.Duration`	`10s`	Interval for CRD state reconciliation
`TokenPath`	`string`	`/var/run/secrets/kubernetes.io/serviceaccount/token`	Path to service account token

Methods

ApplyDefaults(env *KubernetesEnvironment) — Sets default values for zero-valued fields. When env is non-nil and InCluster is true, sets Enabled=true, CRDEnabled=true, and auto-detects Namespace.
Validate() error — Skips validation when Enabled is false. Validates Namespace non-empty, AuditLogPath non-empty, CRDSyncInterval >= 1s, TokenPath non-empty.

PlexdNodeState CRD

The PlexdNodeState custom resource definition exposes node state to the Kubernetes API.

type PlexdNodeState struct {
    Name            string               `json:"name"`
    Namespace       string               `json:"namespace"`
    UID             string               `json:"uid,omitempty"`
    ResourceVersion string               `json:"resourceVersion,omitempty"`
    Labels          map[string]string    `json:"labels,omitempty"`
    Spec            PlexdNodeStateSpec   `json:"spec"`
    Status          PlexdNodeStateStatus `json:"status,omitempty"`
    LastUpdate      time.Time            `json:"lastUpdate"`
}

type PlexdNodeStateSpec struct {
    NodeID     string            `json:"nodeId"`
    MeshIP     string            `json:"meshIp,omitempty"`
    Metadata   map[string]string `json:"metadata,omitempty"`
    Data       []DataEntry       `json:"data,omitempty"`
    SecretRefs []SecretRef       `json:"secretRefs,omitempty"`
}

type PlexdNodeStateStatus struct {
    Report []DataEntry `json:"report,omitempty"`
}

CRD manifest: deploy/kubernetes/crds/plexdnodestate-crd.yaml

Property	Value
API group	`plexd.plexsphere.com`
Kind	`PlexdNodeState`
Plural	`plexdnodestates`
Short name	`pns`
Scope	`Namespaced`
Version	`v1alpha1`

Printer columns: Node ID, Mesh IP, Age.

KubeClient interface

Abstracts Kubernetes API interactions for testability. All state-modifying methods are idempotent.

type KubeClient interface {
    GetNodeState(ctx context.Context, namespace, name string) (*PlexdNodeState, error)
    CreateNodeState(ctx context.Context, state *PlexdNodeState) error
    UpdateNodeState(ctx context.Context, state *PlexdNodeState) error
    DeleteNodeState(ctx context.Context, namespace, name string) error
    WatchNodeState(ctx context.Context, namespace, name string) (<-chan PlexdNodeStateEvent, error)
    CreateSecret(ctx context.Context, secret *KubeSecret) error
    UpdateSecret(ctx context.Context, secret *KubeSecret) error
    DeleteSecret(ctx context.Context, namespace, name string) error
    WatchPlexdHooks(ctx context.Context, namespace string) (<-chan PlexdHookEvent, error)
    UpdatePlexdHookStatus(ctx context.Context, hook *PlexdHook) error
    CreateJob(ctx context.Context, job *PlexdJob) error
}

Sentinel errors

Error	Value	Description
`ErrNotFound`	`kubernetes: resource not found`	Resource does not exist
`ErrAlreadyExists`	`kubernetes: resource already exists`	Resource already exists
`ErrUnauthorized`	`kubernetes: unauthorized`	Client lacks valid credentials

CRDController

Periodically syncs local node state to a PlexdNodeState CRD resource.

type CRDController struct {
    client         KubeClient
    reportNotifier ReportNotifier
    cfg            Config
    namespace      string
    nodeID         string
    meshIP         string
    resourceName   string // "node-{node_id}"
    logger         *slog.Logger
}

Constructor

func NewCRDController(client KubeClient, cfg Config, nodeID, meshIP, namespace string, reportNotifier ReportNotifier, logger *slog.Logger) *CRDController

Config defaults are applied automatically.

ReportNotifier interface

type ReportNotifier interface {
    NotifyChange()
}

Called when the CRDController detects new or changed report entries in the status subresource.

Start

func (c *CRDController) Start(ctx context.Context) error

Creates or updates the PlexdNodeState resource and starts the status watcher goroutine. Blocks until ctx is cancelled.

Start behavior

Ensures the CRD resource exists (create or update)
Starts a watch on the resource for status changes
When status report entries change, calls ReportNotifier.NotifyChange()
Blocks until context is cancelled

K8sAuditLogReader

Implements auditfwd.K8sAuditReader by reading Kubernetes audit log files in JSON-lines format.

type K8sAuditLogReader struct {
    path   string
    offset int64
    logger *slog.Logger
}

Constructor

func NewK8sAuditLogReader(path string, logger *slog.Logger) *K8sAuditLogReader

ReadEvents

func (r *K8sAuditLogReader) ReadEvents(ctx context.Context) ([]auditfwd.K8sAuditEntry, error)

Behavior:

Reads from r.offset to end of file; only new entries since last call are returned
If the file does not exist or cannot be opened, returns an error
If the file was truncated (offset > file size), resets offset to 0 and reads from the beginning
Malformed JSON lines are skipped with a warning log
Timestamp is parsed from stageTimestamp, falling back to requestReceivedTimestamp
objectRef and responseStatus may be nil in the audit event; handled gracefully

Audit log format (JSON-lines):

Each line is a Kubernetes audit event:

json

{"kind":"Event","verb":"create","user":{"username":"admin"},"objectRef":{"resource":"pods","namespace":"default","name":"web-1"},"responseStatus":{"code":201},"stageTimestamp":"2025-01-15T10:30:01.000000Z"}

TokenReviewAuthenticator

Validates Kubernetes service account tokens via the TokenReview API.

type TokenReviewAuthenticator struct {
    client    TokenReviewClient
    logger    *slog.Logger
    audiences []string
}

Constructor

func NewTokenReviewAuthenticator(client TokenReviewClient, logger *slog.Logger, audiences []string) *TokenReviewAuthenticator

Authenticate

func (a *TokenReviewAuthenticator) Authenticate(ctx context.Context, token string) (*TokenReviewResult, error)

Condition	Result
Empty token	Error: `kubernetes: authenticate: empty token`
Client error	Error: `kubernetes: authenticate: review failed`
Not authenticated	Error: `kubernetes: authenticate: token not authenticated`
Audience mismatch	Error: `kubernetes: authenticate: audience mismatch`
Success	Returns `*TokenReviewResult`

HTTPTokenReviewClient

Production implementation that calls the Kubernetes API server:

func NewHTTPTokenReviewClient(apiServer, saTokenPath string) *HTTPTokenReviewClient

Posts to {apiServer}/apis/authentication.k8s.io/v1/tokenreviews
Authenticates with the service account token read from saTokenPath
Loads cluster CA from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
TLS minimum version: 1.2

HookLister

Lists PlexdHook resources through the API server for the capability manifest. It is one authenticated GET per call: no watch, no write.

func NewInClusterHookLister(env *KubernetesEnvironment) (*HookLister, error)
func (l *HookLister) ListPlexdHooks(ctx context.Context, namespace string) ([]PlexdHook, error)

Connects to https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT; construction fails when either variable is unset
Trusts the cluster CA at DefaultCACertPath; construction fails when the bundle cannot be read or holds no certificate
GETs /apis/plexd.plexsphere.com/v1alpha1/namespaces/{namespace}/plexdhooks
Reads the service account token from env.ServiceAccountToken on every call, because the kubelet rotates it
Has no timeout of its own; the caller's context bounds each call
TLS minimum version: 1.2

Response	Result
`200 OK`	The resources, or an empty slice
`401`, `403`	Error wrapping `ErrUnauthorized`
`404`	Error wrapping `ErrNotFound` (CRD not installed)
Other status	Error naming the status

Boot-time PlexdHook discovery

plexd up runs DefaultDetector.Detect() before it sends the capability manifest. In a pod it lists the PlexdHook resources in the ServiceAccount's namespace once, with a 10-second budget, and reports the digest-pinned ones as plexd_hooks (see PlexdHook CRD). The existing plexdhooks get, list, watch grant in the plexd ClusterRole authorizes the list, so no RBAC change is needed. Outside a cluster the step costs one environment variable check. Inside one, a missing grant, a missing CRD, a timeout, or an unreachable API server logs one warning, and the manifest goes out without plexd_hooks. The node boots either way.

Kubernetes manifests

All manifests are in deploy/kubernetes/.

File	Resources
`namespace.yaml`	`Namespace` (plexd-system)
`crds/plexdnodestate-crd.yaml`	`CustomResourceDefinition` (PlexdNodeState)
`crds/plexdhook-crd.yaml`	`CustomResourceDefinition` (PlexdHook)
`serviceaccount.yaml`	`ServiceAccount`
`rbac.yaml`	`ClusterRole` + `ClusterRoleBinding` + consumer roles
`daemonset.yaml`	`DaemonSet` with host networking and capabilities
`secret.yaml`	Example `Secret` template for bootstrap token

RBAC permissions

The plexd ClusterRole grants:

API Group	Resource	Verbs
`plexd.plexsphere.com`	`plexdnodestates`	get, list, watch, create, update, patch, delete
`plexd.plexsphere.com`	`plexdnodestates/status`	get, patch, update
(core)	`secrets`	create, get, update, delete
`plexd.plexsphere.com`	`plexdhooks`	get, list, watch
`plexd.plexsphere.com`	`plexdhooks/status`	get, update, patch
`batch`	`jobs`	create, get, list, watch
`authentication.k8s.io`	`tokenreviews`	create

Consumer RBAC roles:

Role	Resources	Verbs
`plexd-state-reader`	`plexdnodestates`	get, list, watch
`plexd-state-reporter`	`plexdnodestates/status`	get, patch
`plexd-secrets-reader`	`secrets`	get
`plexd-hook-reader`	`plexdhooks`	get, list, watch

DaemonSet configuration

Setting	Value	Reason
`hostNetwork`	`true`	WireGuard mesh requires host networking
`dnsPolicy`	`ClusterFirstWithHostNet`	Required with hostNetwork
`priorityClassName`	`system-node-critical`	Ensures scheduling on resource pressure
`tolerations`	`operator: Exists`	Run on all nodes including control plane
`readOnlyRootFilesystem`	`true`	Security hardening
Capabilities	`NET_ADMIN`, `NET_RAW`	WireGuard interface management and the nftables policy chain. Without `NET_ADMIN`, `plexd up` fails its firewall pre-flight and exits before registering

Environment variables

Variable	Source	Description
`MY_NODE_NAME`	`fieldRef: spec.nodeName`	Kubernetes node name (downward API)
`PLEXD_BOOTSTRAP_TOKEN`	`secretKeyRef: plexd-bootstrap`	Bootstrap token from Secret

Volume mounts

Mount path	Source	Access
`/etc/plexd`	ConfigMap `plexd-config`	read-only
`/var/lib/plexd`	hostPath	read-write
`/var/run/plexd`	hostPath	read-write
`/var/log/kubernetes/audit`	hostPath	read-only

Constants

Constant	Value
`ServiceAccountBasePath`	`/var/run/secrets/kubernetes.io/serviceaccount`
`DefaultTokenPath`	`{ServiceAccountBasePath}/token`
`DefaultNamespacePath`	`{ServiceAccountBasePath}/namespace`
`DefaultCACertPath`	`{ServiceAccountBasePath}/ca.crt`
`DefaultAuditLogPath`	`/var/log/kubernetes/audit/audit.log`
`DefaultCRDSyncInterval`	`10s`

Kubernetes DaemonSet Deployment Reference ​

KubernetesEnvironment ​

EnvironmentDetector interface ​

Config ​

Methods ​

PlexdNodeState CRD ​

KubeClient interface ​

Sentinel errors ​

CRDController ​

Constructor ​

ReportNotifier interface ​

Start ​

Start behavior ​

K8sAuditLogReader ​

Constructor ​

ReadEvents ​

TokenReviewAuthenticator ​

Constructor ​

Authenticate ​

HTTPTokenReviewClient ​

HookLister ​

Boot-time PlexdHook discovery ​

Kubernetes manifests ​

RBAC permissions ​

DaemonSet configuration ​

Environment variables ​

Volume mounts ​

Constants ​

See also ​

Kubernetes DaemonSet Deployment Reference

KubernetesEnvironment

EnvironmentDetector interface

Config

Methods

PlexdNodeState CRD

KubeClient interface

Sentinel errors

CRDController

Constructor

ReportNotifier interface

Start

Start behavior

K8sAuditLogReader

Constructor

ReadEvents

TokenReviewAuthenticator

Constructor

Authenticate

HTTPTokenReviewClient

HookLister

Boot-time PlexdHook discovery

Kubernetes manifests

RBAC permissions

DaemonSet configuration

Environment variables

Volume mounts

Constants

See also