SkillOpt Native for Agent Zero

A native community plugin that trains skill documents using Microsoft's SkillOpt optimizer and runs SkillOpt-Sleep over scoped Agent Zero transcripts. Version 0.1.0 is a preview implementation, pinned to upstream commit 79124b37e9a6371e13b753f8bcd7adb1e493ade1.

Open Plugins → SkillOpt Native → Open, or use Dashboard in its settings. Upload independently scored tasks, train, inspect held-out scores and changes, and adopt a proposal. Adoption affects subsequent prompts in the selected project/profile. Scheduling and automatic adoption start off.

Install

Download the skillopt_native-0.1.0.zip asset from v0.1.0.

In Agent Zero, open Plugins → Install → ZIP and select the release archive. The archive has plugin.yaml at its root. Reload the page when prompted. No extra dependencies, host Python, Compose file, model download, or core patch is needed. The A0 framework provides Python 3.12+, PyYAML, Flask, and its existing model infrastructure.

For a manual Docker installation, first identify the intended container with docker ps, then copy the extracted directory:

docker cp ./skillopt_native CONTAINER:/a0/usr/plugins/skillopt_native

Use that container's published WebUI URL. The plugin must be named skillopt_native and installed under /a0/usr/plugins/skillopt_native. Do not put it under bundled /a0/plugins/. This independent community integration is being submitted to the Plugin Index. Source and ZIP releases are available separately.

First run

1. Select a project (blank means global) and agent profile in the dashboard.
2. Use Load sample, then Train, to run a deterministic demo without provider calls. Its scores demonstrate plumbing, not real model improvement.
3. Configure optimizer and target roles through the native plugin Settings dialog. Blank provider/model pairs inherit the selected A0 profile's chat model. Both fields must be filled for an override. Credentials remain with A0.
4. Upload real JSON/JSONL tasks, uncheck Scripted demo, and train. The target sees the instruction/context and candidate skill; independent references remain with the evaluator/analyst.
5. Review a proposal's diff and validation receipt. Adopt, discard, or roll back. A changed live baseline blocks adoption. Rollback also refuses later owner edits.

For Codex subscription providers in current A0, the provider identifier is codex_oauth, with a supported model configured on that instance. An older codex_auth setting is not automatically rewritten. Use the working provider identifier in this plugin's role overrides.

TaskSets

See examples/tasks.json. Each task needs a unique safe ID (letters, numbers, _, -), instruction, and an independent reference. Optional context, task_type, source_sessions, split, and judge fields are supported. Judges: exact, contains, regex, llm_rubric, and tool_called. Regex execution has a separate process timeout. Tool judges use recorded execution events.

Provide either all splits or no splits. Available splits are train, val, and test. Related sessions and normalized duplicate prompts cannot cross splits. Training and selection must each contain at least one independent group. Keep a separate test set when you need an evaluation untouched by selection. Set eval_test only when a test split exists.

Settings accept flat keys or the upstream structured training/gradient/optimizer/evaluation form. The controller's import_config action accepts YAML and a dictionary of uploaded _base_ documents, without reading server-supplied paths. Unsupported external backends and credentials are rejected. See compatibility for boundaries.

Engines and target modes

Training retains Rollout → Reflect → Aggregate → Select → Update → Evaluate, accumulated minibatches, support-count ranking, protected regions, three update families, learning-rate controls, optional skill-aware reflection, longitudinal slow updates, meta-skills, strict improvement gating, score caches, and checkpointed resume.

model mode makes direct calls through A0 wrappers. agent mode creates fresh background contexts with a pinned candidate, a plugin-owned evaluation profile, and no MCP servers. Only response and the inert skillopt_native_fixture echo tool are permitted. It is not a general shell/browser benchmark sandbox. The target never gets the private oracle. Normal chat contexts are not reused as evaluation subjects.

Sleep harvests bounded persisted chat snapshots from the selected scope, excludes internal/background runs, redacts common secret forms, mines checkable tasks, replays, reflects, applies edits, gates, and stages changes. Optional dream rollouts, synthetic variants, past-task recall, and multi-skill fan-out use the pinned upstream algorithms. A durable split ledger keeps previously seen sessions on their original side of the holdout boundary. Linked historical conflicts fail closed for review.

Use Add seed skill for separate plugin-owned skills, then enable multi-skill fan-out in Settings. Review and select named proposals independently. Seed skills are explicit owner-provided guidance. Existing repository files and system prompts are never edited. Managed learned memory lives in plugin-owned MEMORY.md; the upstream staging artifact retains its proposed_CLAUDE.md filename for compatibility.

Budgets and recovery

Each provider request is charged input UTF-8 bytes plus the full output token cap, including failures. A0 wrapper retries are disabled for these requests; optimizer retries are individually budgeted. This is a conservative reservation, not billed token measurement. max_tokens: 0 disables the token limit. Timeouts, wall-clock limits, turn limits, and Stop remain available. Model output caps depend on provider enforcement.

Run state and TaskSets are under data/scopes/<scope-hash>/. Completed checkpoints include optimizer state, skill versions, history, meta artifacts, and step digests. Resume pins inputs, preserves consumed budget, and replays unfinished work from the last checkpoint; interrupted in-flight calls may incur cost. Process restarts surface orphaned runs as interrupted. Sleep cycles restart as a new cycle rather than pretending to resume an incomplete provider call.

Adoption uses upstream integrity pins and transaction recovery. Plugin rollback journals restore prior bytes, including an empty initial skill, and can finish after an interrupted write. A scope lease prevents concurrent training/adoption. Scheduling uses A0's existing job loop and a durable due-time lease; it creates no OS service or cron job.

Lifecycle hooks run retention during installation and before updates. Older terminal runs beyond keep_runs move into data/archives/; they are not deleted. Busy scopes, active runs, and runs referenced by proposals are retained. No Execute button or sidebar shortcut is needed.

Verify and uninstall

From the A0 framework root:

/opt/venv-a0/bin/python -m pytest usr/plugins/skillopt_native/tests -q

Use the framework interpreter, not the agent shell environment. Validation evidence distinguishes deterministic, host, browser, and real provider checks.

Stop runs, back up data/, then disable or uninstall via Plugins. Disabling stops active work at its next cancellation check and prevents new API actions. Pre-update/uninstall requests cancellation. Uninstall removes plugin-owned data with the plugin; keep the backup for recovery. Nothing installs dependencies or permanent services.

Microsoft SkillOpt is MIT licensed. See LICENSE, THIRD_PARTY_NOTICES.md, and UPSTREAM.json.

Brand asset

The teal-and-silver folded S represents iterative skill improvement. Identical 256 × 256 thumbnail.jpg assets in the root and webui/ are each below the Plugin Index 20 KB limit; the WebUI discovers its copy automatically.

Settings compatibility

SkillOpt exposes 56 supported settings. Full rewrites bypass edit-count budgets; their ranking controls are disabled in the UI. The former meta_edit_budget option was unsupported upstream and has been removed. Existing configs with its old default 3 still load; custom values produce a clear validation error. Cross-epoch meta-skill learning remains available through Improve the optimizer’s guidance. Provider and model overrides must be supplied together.

Preview identity

The public plugin ID is skillopt_native, distinct from other SkillOpt integrations. Early local builds used skillopt. They are separate installations: stored runs and settings are not automatically migrated. Keep a backup of the old plugin's data, and disable it before enabling this one if you previously used that local preview.