Turn a tutorial into a skill an agent can use.

The problem

A video can show a useful procedure, but an agent cannot follow it from the video alone. Someone has to replay it, take notes and turn those notes into instructions. I built Watch & Learn to do that work once.

How it works

1. 01
   Check the video. Stop if it doesn’t teach a repeatable task.
2. 02
   Watch locally. Python saves a few frames and transcribes the speech.
3. 03
   Write the skill. The agent writes a SKILL.md file with ordered steps.
4. 04
   Try it. The agent follows the new skill on a real task and revises it if needed.

PyAV extracts frames and faster-whisper handles transcription. Both run locally. The tool needs internet to fetch an online video and to install its video tools the first time; it needs no separate API key or paid service. The coding agent you use still has its own model and account.

A finished example

I used it on a conference talk about finding the decisions people leave out of a software plan. It produced a skill called find-your-unknowns, with instructions for asking those missing questions. Here is the start of the file it wrote, and its table of patterns:

1---2name: find-your-unknowns3description: >4  Before turning an agent loose on a coding task, surface the decisions you never specified —5  the gaps between your prompt (the "map") and the real codebase (the "territory"). Runs six6  prompt patterns (blind-spot pass, brainstorm/prototype, interview-me, reference-as-map,7  implementation-notes logging, quiz-me) to enumerate unknowns and shape the architecture up8  front. Use when the user says "help me scope this before I build it", "what am I missing",9  "find my blind spots", "interview me about this feature", "I don't know this codebase / this10  domain, help me prompt better", "de-risk this before the big run", or "surface the unknowns"11  — and BEFORE kicking off a large or architecture-shaping agent run on an under-specified task.12---⋯39| Kind of unknown | Pattern | When | Skip if |40|---|---|---|---|41| "I don't even know what I don't know here" | **1. Blind-spot pass** | scoping | rarely — this is the anchor |42| Aesthetic / "know it when I see it" | **2. Brainstorm & prototype** | scoping | the surface isn't primarily visual |43| Under-specified requirements | **3. Interview me** | scoping | requirements are already crisp |44| An example is easier than a spec | **4. Reference — give it another map** | scoping | you have no reference to point at |45| Deviations that only appear mid-build | **5. Implementation notes** | build-time | never — it's a cheap always-on trail |46| Do *I* actually understand the result? | **6. Quiz me** | post-build | you already own the result cold |

The repository also has a walkthrough of the test. I later added a hook that reminds the agent to use that skill before it presents a plan. The hook is an example of what someone can do with a finished skill; Watch & Learn doesn’t install it for you.

Decisions

What isn't tested or measured

• On every push, CI checks the skill’s structure: its frontmatter, the files it refers to, and that its scripts compile.
• There are no unit tests for the video pipeline. The plugin-manifest check is skipped on CI because the runner has no Claude CLI; I run it locally before a release.
• I haven’t measured how often other people use the tool or whether the skills it writes for them succeed.

Lessons

A keynote or one-off demo may be interesting but make a poor skill. The video needs to teach a procedure someone can repeat. Even then, a well-formatted skill file can give bad instructions. The agent has to try it on a task.

The published example shows one skill that worked in a test and is in use on my machine.