Loopback Sessions¶

Overview¶

Loopback (GH #64) re-runs the mutating part of an attack against only the newly-cracked plaintexts (the "delta"), repeating until a round produces no new plaintext. It exists because "run this preset again in a workflow" would re-try the whole wordlist for a handful of new candidates; loopback instead loops the mutation over just the delta, so each round is cheap and the loop terminates on its own.

The original GitHub issue asked to allow the same preset job multiple times in a workflow. That was resolved not by permitting duplicates, but by this delta-only re-run mechanism.

Concepts¶

Eligibility¶

models.IsMutatableAttack(mode, ruleIDs) decides whether an attack's mutation can be re-run against a delta wordlist:

Ineligible jobs are not re-run, but their cracks still feed the delta pool that eligible lineages loop on.

The delta¶

For each round, the delta is the set of distinct new plaintexts cracked by the session's own jobs that have not yet been used as loopback input for this session:

• Cracks are scoped to the session via hashes.cracked_by_task_id → job_tasks → loopback_session_jobs, so only this session's work contributes.
• A per-session guard set, loopback_session_plaintexts (the md5 of each plaintext already used as input), subtracts candidates that were fed in a previous round.

Because every distinct plaintext is consumed exactly once, the loop is guaranteed to terminate: a round is either strictly smaller than the pool of not-yet-tried cracks, or empty (dry) — at which point the session completes.

The delta is read straight from the hashes table (synchronous), so a potfile flush is not required before a round can start.

Session Lifecycle¶

A session is one loopback group: a workflow run, or a standalone preset/custom run.

waiting ──▶ active ──▶ completed
   │           │
   └──────┬────┴──────▶ failed / cancelled

The controller / monitor¶

services.LoopbackService is a durable controller — it survives a backend restart because each tick re-queries the waiting/active sessions from the database rather than holding them in memory (unlike the transient preparing job state, which is failed on restart). It is started once via a sync.Once in routes/user.go CreateJobsHandler.

An idle-gated poller (≈7 s) checks LoopbackRepository.HasActiveSessions — backed by a partial index on loopback_sessions(status) WHERE status IN ('waiting','active') — so it costs almost nothing when no sessions are in flight.

On each tick, for a session whose current round's jobs are all terminal:

1. Stop early if the hashlist is already fully cracked (HashlistHasUncracked) — no wasted round against an exhausted list.
2. Compute the delta.
3. If the delta is empty → complete the session (dry).
4. Otherwise spawn one re-run per mutatable lineage and advance the round.

Re-runs reuse the pre-filter job path (CreatePreparingFilterJob → materialize the ephemeral delta wordlist → FinalizeFilterJob). They are ordinary job_executions, so the scheduler, agents, and progress tracking are entirely unchanged — the loopback layer only decides what to enqueue and when.

Delta wordlist materialization¶

wordlist.Manager.MaterializeEphemeralWordlist([]string, …) writes the delta to a custom/__eph__*.txt wordlist flagged is_ephemeral with an owner_job_id, inheriting the wordlist filtering cleanup lifecycle. (This differs from the GH #40 materializer, which filters a parent wordlist rather than writing an explicit line set.)

Configuration¶

Loopback config lives on the workflow or at job-start time — never on the preset job definition:

The create-job wiring is in handlers/jobs/user_jobs.go CreateJobFromHashlist: preset and custom runs read the request loopback flag; workflow runs read the step/master flags. The frontend surfaces are JobWorkflowForm.tsx (workflow builder — master + per-step, eligibility aware), CreateJobDialog.tsx (per-run toggle on the Preset and Custom tabs; a "Loopback" badge on the Workflow tab), and components/jobs/LoopbackSessionsPanel.tsx (the monitoring panel on the Jobs page and Dashboard, backed by GET /api/loopback-sessions).

GET /api/loopback-sessions¶

The panel's endpoint (ListLoopbackSessions) is a live view, not a history:

• In-flight only. It always sets LoopbackSessionFilter.InFlightOnly, so the query returns status IN ('waiting','active') and nothing else. Terminal sessions are never deleted (nothing in the backend deletes a loopback_sessions row — a session whose jobs were removed by Delete Finished Jobs simply keeps an empty job list), so filtering on read is what keeps finished sessions off the panel (GH #79). The literal status list matches the partial index idx_loopback_sessions_active.
• Team-scoped. Non-admins are restricted through hashlists → clients → client_teams exactly like ListJobs, and fail closed (no results) when teams are enabled and the user belongs to none. Because hashlists.client_id is nullable and NULL IN (...) is never true, client-less legacy hashlists drop out of team-filtered results — same as the Jobs list, and intentional.

• ?scope=mine|visible. Default mine adds a created_by restriction; visible drops it so the caller sees every session they are allowed to see. visible can never widen past the team filter. The Jobs page requests visible; the Dashboard uses the default.

  The created_by restriction applies to admins too. The admin check in ListLoopbackSessions governs only the team filter (admins skip it, as in ListJobs); it does not exempt anyone from the scope filter, which is applied afterwards and unconditionally for any scope other than visible. So an admin's Dashboard panel lists only the sessions that admin personally started. That is intended — the Dashboard answers "what did I start" and the Jobs page answers "everything I can see" — but it is a visible behaviour change for admins who previously saw every in-flight session on the Dashboard. An admin who wants the old view can use the Jobs page panel, or request ?scope=visible directly. - Capped at 100 rows, newest first. The service hydrates each session's jobs with a per-session GetSessionJobs query — an N+1 that is acceptable only because the filter bounds the result set to in-flight sessions.

loopback_max_rounds¶

A system_settings row (loopback_max_rounds, default 10, integer) caps how many delta rounds a session runs before it stops, even if new cracks keep appearing. It is resolved when a session is created (resolveMaxRounds), falling back to defaultLoopbackMaxRounds = 10 if the setting is missing. See Job Settings.

Schema¶

Migration 20260717120000_add_loopback:

• job_workflows.loopback_all_eligible BOOLEAN — workflow master toggle.
• job_workflow_steps.loopback_enabled BOOLEAN — per-step toggle.
• loopback_sessions — one row per session (hashlist_id, source_type workflow/preset/custom, source_workflow_id, status, current_round, max_rounds, created_by). Partial index on the in-flight statuses for the monitor's idle gate.
• loopback_session_jobs — links each job_execution to a session per round, with a role (original | rerun), is_mutatable, and origin_job_id (the round-0 job a lineage descends from). round 0 = original jobs; round >= 1 = controller re-runs.
• loopback_session_plaintexts — the per-session used-set (plaintext_md5) that guarantees each plaintext is fed exactly once.
• system_settings row loopback_max_rounds (default 10).

Scope and caveats (v1)¶

• Custom loopback is not combined with the GH #40 wordlist filter — the two are mutually exclusive in the UI.
• Combination (-a 1) and association (-a 9) are intentionally excluded.
• Only the UI create-job path (user_jobs.go) creates sessions; the programmatic api/v1 job-create path does not start loopback sessions.

Related¶

• Loopback (user guide)
• Increment Mode
• Job Completion System
• Wordlist Filtering
• Scheduler v2 Overview