Queue

Durable background processing state. Current state comes from the database; lifecycle totals are since this process started.

Refresh
{{if .Error}}
{{.Error}}
{{end}} {{if .Result}}
Done {{.Result}}
{{end}} {{if .Stats}}
{{if eq .QueueLevel "danger"}}Action required{{else if eq .QueueLevel "warn"}}Monitor{{else}}Operational{{end}} {{.QueueDetail}}
{{end}} {{if .Stats}}
{{.Stats.Ready}}
Ready now
{{if .Stats.OldestReadyAt}}
Oldest {{formatTimeRelative .Stats.OldestReadyAt}}
{{end}}
{{.Stats.Scheduled}}
Scheduled later
{{.Stats.Running}}
Running
{{.Metrics.ActiveWorkers}} / {{.Metrics.ConfiguredWorkers}} workers busy{{if gt (len .WorkerPools) 1}} ({{range $i, $p := .WorkerPools}}{{if $i}}, {{end}}{{$p.Name}} {{$p.Active}}/{{$p.Configured}}{{end}}){{end}}
{{.Stats.Failed}}
Retained failures
{{.Stats.StaleRunning}} stale

By job kind

{{$pools := .WorkerPools}}{{range .Stats.ByKind}}{{else}}{{end}}
Jobs by kind
KindReadyScheduledRunningFailed
{{.Kind}}{{if gt (len $pools) 1}} {{jobPoolLabel .Kind}}{{end}}{{.Ready}}{{.Scheduled}}{{.Running}}{{.Failed}}
The queue is empty.
{{end}}

Queue policy

Stale running timeout: {{formatDuration .StaleTimeout}} - Terminal retention: {{formatDuration .Retention}} - Maintenance interval: {{formatDuration .MaintenanceInterval}} - Retrying now: {{if .Stats}}{{.Stats.Retrying}}{{else}}-{{end}}

Currently processing

{{if .RunningJobs}}
{{$pools := .WorkerPools}}{{range .RunningJobs}}{{end}}
Currently processing
KindWorkDurationAttempt
{{.Kind}}{{if gt (len $pools) 1}} {{jobPoolLabel .Kind}}{{end}}{{.Summary}}{{.Timing}}{{.Attempts}} / {{.MaxAttempts}}
{{else}}

Workers are currently idle.

{{end}}

Queued contents by kind

Safe summaries of ready and scheduled work; raw payloads and internal identifiers are not shown.

{{if .ContentsError}}
{{.ContentsError}}
{{end}} {{if .QueuedGroups}}{{$pools := .WorkerPools}}{{range .QueuedGroups}}
{{.Label}}{{if gt (len $pools) 1}}{{jobPoolLabel .Kind}}{{end}}{{len .Jobs}}
{{range .Jobs}}{{end}}
Queued jobs: {{.Label}}
WorkStateTimingAttempt
{{.Summary}}{{.State}}{{.Timing}}{{.Attempts}} / {{.MaxAttempts}}
{{end}}{{else}}{{if not .ContentsError}}

No jobs are waiting.

{{end}}{{end}} {{if lt .ActiveJobsShown .ActiveJobsTotal}}
Showing the first {{.ActiveJobsShown}} of {{.ActiveJobsTotal}} active jobs.
{{end}}

Recent problem jobs

Individual federation delivery failures are expected and do not by themselves make the queue unhealthy. Deleting a queued retry cancels that retry. Raw payloads are not displayed.

{{if .Problems}}
{{range .Problems}}{{end}}
Recent problem jobs
KindStateAttemptsNext run / updatedErrorAction
{{.Kind}}{{if gt (len $.WorkerPools) 1}} {{jobPoolLabel .Kind}}{{end}}{{.State}}{{.Attempts}} / {{.MaxAttempts}}{{if eq .State "queued"}}{{formatTime .RunAt}}{{else}}{{formatTime .UpdatedAt}}{{end}}{{if .Error}}{{.Error}}{{else}}Retry scheduled{{end}}
{{else}}

No failed or retrying jobs are retained.

{{end}}