{"slug": "skill-recorder-desktop-app-that-records-your-on-screen-work-session", "title": "Skill Recorder: Desktop app that records your on-screen work session", "summary": "Microsoft released Skill Recorder, a desktop app that captures an on-screen work session — clicks, window switches, pages visited, optional terminal commands and spoken narration — and uses the GitHub Copilot CLI to reconstruct the session as an intent plus an ordered list of steps, then generates a reusable SKILL.md procedure or a scheduled Automation. The app also imports local MP4 or WebM files instead of screen recording, with initial limits of 512 MiB, 15 minutes and 3840 x 2160 pixels, and a 2 GiB decoder working-set guard; a Windows ARM64 stress test with a 512 MiB input peaked at approximately 4.6 GB (4.25 GiB) of memory across Electron processes. Generated skills and automations prefer the agent's native tools, such as the gh CLI or web_fetch, over replaying UI clicks, and generalize from a single recorded example.", "body_md": "**Record yourself doing a task once, then turn it into a skill your AI agent can repeat.**\n\nTip\n\n**Want to install Skill Recorder? [Go to the latest release](https://github.com/microsoft/skill-recorder/releases/latest).**\nScroll to **Commit-pinned installation** and copy the complete command for your\noperating system. It is ready to paste: you do not need to edit it or download\nthe source code yourself. [Step-by-step instructions below](#install-it).\n\nSkill Recorder captures a real work session on your screen: the clicks, the app and\nwindow switches, the pages you visit, commands/output from its optional recorded terminal,\nand (if you want) your spoken narration. It then uses\nthe **GitHub Copilot CLI** to reconstruct *what you actually did* as a clear **intent plus\nan ordered list of steps**. From there, one step turns that single run into something an\nagent can reuse:\n\n- a **Skill** : a`SKILL.md` procedure an agent runs on demand, or\n- an **Automation** : the same procedure on a schedule or trigger.\n\nBoth prefer the agent's **native tools** (like the `gh` CLI or `web_fetch`) over replaying\nUI clicks, and generalize from your one example, so recording yourself submitting *one*\nform can teach the agent to submit *all* of them.\n\n1. 🔴 **Record.** Hit record (or`⌘⇧R` /`Ctrl+Shift+R` from anywhere) and just do your\ntask. Skill Recorder captures your screen and activity locally, in the background.\n2. 🎛️ **Control.** While recording, a small always-on-top bar shows capture,\nmicrophone, and recorded-terminal state. Open the session-scoped terminal, mute,\nunmute, or switch mics, then finish or discard with confirmation.\n3. 🧠 **Analyze.** Click Analyze and your selected AI provider reconstructs one overall intent and\nan ordered list of steps. Review and edit until it reads right.\n4. ✨ **Create.** From an approved analysis, generate a reusable**Skill** and/or a\nscheduled**Automation** .\n\nFor skills, **Review SKILL.md** optionally shows the complete, read-only file text\n(including frontmatter) before installation or export. Preparing a preview writes\nno skill files. Close and reopen it without regenerating; Add/Export from either\nsurface uses that exact preview. Editing the plan discards the preview. You can\nstill choose Add/Export directly without reviewing or adding a confirmation step.\nReview provides visibility into the generated instructions, not a security guarantee.\n\nInstead of recording your screen, use the smaller **folder icon beside Record** to\nchoose a local **MP4 or WebM** file, such as a downloaded Teams demonstration.\nChromium checks the actual video and audio before the details screen opens.\nUnsupported codecs, encrypted or unsupported track layouts, corrupt media, and\nfiles outside the import limits are rejected with an explanation. No converter,\nstandalone FFmpeg installation, or additional codec download is required.\n\nThe initial limits are **512 MiB**, **15 minutes**, and **3840 x 2160 pixels**\n(or an equivalent pixel count). Audio must be mono or stereo at no more than\n48 kHz. Import supports one video track and at most one audio track. MP4 track\ntypes recognized are AVC, HEVC, VP9, or AV1 video with AAC/Opus audio; WebM\nrecognizes VP8, VP9, or AV1 with Opus/Vorbis. Recognized track metadata is not a\ncodec-support guarantee: the shipped Chromium runtime must actually decode it.\n\nImport memory use is higher than the compressed file size. A Windows ARM64\nstress check with a 512 MiB input, 15-minute stereo audio, and the existing\nWhisper model resident peaked at approximately **4.6 GB (4.25 GiB)** across\nElectron processes. This is a measurement, not a guarantee for every file or\nplatform, and excludes other apps and the external Copilot CLI process. The\nimport decoder has a **2 GiB** working-set guard; recordings that exceed it are\nrejected even when their file size is within the limit.\n\nAfter validation, you can enter **Additional details**, or leave the field empty.\nExplain the task, supplement what is visible, or indicate where the demo happens.\n**Timestamps are guidance, not trimming:** other parts of the recording may still\nbe processed. Choose the recording's spoken language when it contains audio.\nAnalyze transcribes audio locally using the existing Whisper model (with the\nfirst-use download disclosed before proceeding); transcription failures stop the\nimport's analysis so you can retry. Videos without audio and recordings with no\ndetected speech are supported.\n\nThe resulting activity list uses the normal review flow: edit details, reorder, add or remove activities, then create and review your skill as usual. Imported analysis uses frames, speech, and your details, not captured OS events; it cannot recover hidden clipboard contents or commands that were never visible or spoken.\n\nThe original file is untouched. A local copy and derived artifacts are stored in Sessions, and deleting the session removes only that copy. Selecting a file does not send it to the cloud. Choosing Analyze sends selected frames, transcribed speech, and your details through the existing privacy-protection pipeline to your selected AI provider. Debug exports include the imported media and details, just as they include private data from live recordings. Only import recordings you are authorized to use, and keep sensitive/confidential content out of them.\n\nLive recording, its permissions and settings, and its event-first analysis are unchanged. Imports do not require microphone or screen-capture permission.\n\nFor development, `npm test` includes the import contract/privacy tests.\n`npm run test:import:runtime` uses the reviewed Electron runtime to generate\nsynthetic MP4/WebM files with Chromium and exercise decoding, seeking, silent and\nabsent audio, unsupported codecs, cancellation, and source-file preservation.\nRun that check on each release target; it does not replace checking representative\nTeams files. Add `-- --large` to also import an exact 512 MiB synthetic MP4 and\ncheck rejection of a file one byte over the limit; this check needs additional\nmemory and temporary disk space. The initial local measurements and runtime check were performed on\nWindows ARM64; other platforms and real meeting recordings require their own\nverification.\n\nSkill Recorder is published as a **source release**: one command downloads a pinned Node.js\nruntime, builds the exact release commit on your machine, and adds a **Skill Recorder (Source)**\napp you can relaunch anytime. Nothing is installed globally. You'll need either a GitHub account with\n**Copilot access**, or a supported model provider's API key; the Copilot CLI ships\nwith the app and powers both options.\n\nmacOS is the primary target. Windows 11 (x64 and ARM64) is supported too (see\n[`WINDOWS-VALIDATION.md`](https://github.com/microsoft/skill-recorder/blob/main/WINDOWS-VALIDATION.md)).\n\nOpen the **[latest release](https://github.com/microsoft/skill-recorder/releases/latest)** and\nfollow these steps:\n\n1. On the release page, scroll to **Commit-pinned installation** . Find**Windows 11\nx64 or ARM64** or**macOS or Ubuntu** , then copy the**entire command** in that\nsection using the copy button at the top-right of the code block.\n2. Open **PowerShell** from the Windows Start menu, or**Terminal** on macOS or\nUbuntu. Paste the command and press**Enter** .\n3. Keep that window open while installation completes. The installer downloads\nwhat it needs, builds the app on your computer, and creates shortcuts.\n**On Windows** , follow the completion notice: finish recording, choose**Quit** in every running copy, wait for them to close, then open**Skill Recorder\n(Source)** from the desktop or Start menu. Windows does not launch the app\nautomatically. On macOS or Ubuntu, the installer launches Skill Recorder.\nThe first installation can take several minutes.\n\nImportant\n\n**Copy the command from the release page, not a template from the technical docs.**\nThe release command already contains the correct version identifier; there is\nnothing to replace. Do not paste commands containing\n`<40-character-release-commit>` literally.\nYou do **not** need Git, a separate Node.js installation, or the **Manual developer\nsetup** instructions. The **Source code (zip)** and **Source code (tar.gz)** links\nunder **Assets** are not app installers.\n\nTo review the script before running it, follow the release page's\n**inspect-first instructions** instead. Installation remains subject to your\norganization's policies.\n\n**Open it again later:** use **Skill Recorder (Source)** on your Windows desktop or\nStart Menu; on macOS, find it in `~/Applications`, Spotlight, or Launchpad; on\nUbuntu, use the matching application entry. You do not need to reinstall each time.\nLaunching again reveals the existing recorder or recording controls instead of\nstarting another copy. Older revisions without this protection must be quit\nmanually before opening an updated revision.\nAdvanced options, including running in the background on macOS or Ubuntu, are in\n[`INSTALL.md`](https://github.com/microsoft/skill-recorder/blob/main/INSTALL.md#commit-pinned-one-line-installation).\n\n1. **Grant Screen Recording.** On first launch, macOS asks for Screen Recording permission;\ngrant it and you're ready to record.\n2. **Record, Analyze, Create.** Do your task, then Analyze. When no AI connection\nis configured, an existing GitHub Copilot login is recognized and reused\nautomatically, including after an upgrade from 0.8.0. Otherwise, choose**Bring your own model** or**GitHub authentication** the first time AI is needed.\nBoth options remain available to everyone, including Microsoft FTEs, through**AI connection** in Sessions. Recording needs neither.\nIf you choose GitHub, Skill Recorder reuses a valid login or offers sign-in. Finish\nauthorization in your browser; the app verifies sign-in and automatically retries\nthe interrupted analysis once. There is no login terminal window to close.\nCanceling sign-in or leaving that analysis prevents the automatic retry.\n\nOn Windows, a local Microsoft Entra tenant hint can offer a choice between\n**Microsoft Enterprise SSO** and a **Personal GitHub account**. This is only a\nrouting hint, not proof of employment. Microsoft sign-in has two separate steps:\nfinish SSO in the browser, then authorize the GitHub Copilot app. When SSO lands on\n`github.com/enterprises/microsoft`, close that tab, return to Skill Recorder, and\nchoose **Authorize Copilot**. SSO alone does not authorize the app. Use the same\nbrowser profile and Microsoft-linked GitHub account for both steps.\nWithout a hint, and on macOS/Linux,\nsign-in goes directly to the standard GitHub flow. No work-email prompt is required.\n\nThe authorization step opens the default browser using the CLI's web OAuth flow, not a device code. Keeping the same browser profile lets GitHub reuse the session from SSO, so you can approve the GitHub Copilot app without entering a code. If GitHub asks you to sign in again, check that you are using the same browser profile. After authorization, close that tab and return to Skill Recorder; the app verifies access before continuing.\n\nSkill Recorder pins Copilot CLI 1.0.78, which supports web OAuth without a terminal,\nand runs `--no-auto-update login --web-flow` so sign-in does not depend on a\ndifferent CLI version in the user's auto-update cache. Older bundled versions\nsuch as 1.0.71 do not support `--web-flow`.\n\nSign-in can be canceled and times out after five minutes. If browser login fails, the app shows an error and a manual command for the bundled CLI (use PowerShell on Windows). After manual sign-in, retry the analysis yourself. Signing in from a skill or automation builder does not automatically repeat a build or installation.\n\nTo inspect the script before running it, set install options, update, or uninstall, see\n[`INSTALL.md`](https://github.com/microsoft/skill-recorder/blob/main/INSTALL.md).\n\nOpen **AI connection** in Sessions to configure or change your connection.\nBYOM uses the bundled Copilot SDK/CLI without GitHub login or a Copilot\nsubscription. Model requests go to your selected endpoint and are billed by\nthat provider; BYOM does not mean local/offline processing.\n\n| Provider | Required fields | \n|---|---|\n| Microsoft Foundry / Azure OpenAI (v1) | API key, full `/openai/v1/` base URL, deployment name | \n| Azure OpenAI (versioned) | API key, resource host without an API path, deployment name, Azure API version | \n| OpenAI | API key and model ID; the standard base URL is prefilled | \n| Anthropic | API key and model ID; the standard base URL is prefilled | \n| Custom OpenAI-compatible | API key, HTTPS base URL, model ID, supported API format | \n\nUse a model supporting **image input and tool calling**. **Advanced** selects\nResponses versus Chat Completions for compatible providers. **Test and save**\nsends a small, billable synthetic-image/tool test, never a recording. Custom\nendpoint support depends on actual protocol and model compatibility.\nKeyless local services and managed-identity authentication are not exposed by\nthis first API-key flow.\n\nKeys are protected using the operating system's key storage on Windows 11 (x64/ARM64), macOS, and Ubuntu with a secure desktop keyring. If secure storage is unavailable, the key-entry screen says:\n\nSecure space is unavailable. The Key will be saved in-memory and will be re-asked next time.\n\nThere is no extra step: BYOM still works for the current app session. Non-secret\nsettings are remembered where possible. **Forget key** removes this app's stored\nand in-memory credential, not the key at your provider. Finish or cancel active\nAI work before changing connections. Saved recordings and outputs are preserved,\nbut live conversations are reset. Keys are not included in debug exports.\nBYOM connection settings and SDK conversation state are stored separately from\nrecordings in app data. Forgetting a key does not erase conversation history.\n\nFor development, `npm run test:ai` covers configuration, protected/memory-only\nstorage, and connection lifecycle. `npm run test:ai:runtime` exercises the bundled\nSDK/CLI against a local synthetic provider, including tool-image transport and\ncredential persistence checks. Neither command needs a real API key or sends\nrequests to a paid model. Native key-storage and live-provider release checks\nare described in [`WINDOWS-VALIDATION.md`](https://github.com/microsoft/skill-recorder/blob/main/WINDOWS-VALIDATION.md#ai-connection-and-credential-smoke-checks).\n\n**Keep secrets out of your recordings.** Don't record, type, paste, or narrate\npasswords, tokens, API keys, or other confidential info. Choosing *Analyze* sends\nrecording data to your selected AI provider. Skill Recorder reminds you before every recording.\nDetails in [What gets captured](#what-gets-captured).\n\n⚠️ \n\n*Everything below is for people who want the details, or want to hack on the code.*\n\nRecording, storage, frame extraction, and optional narration transcription all happen\n**on your computer**; nothing leaves while you record. Only when you choose **Analyze**\ndoes Skill Recorder send the event timeline (window/document titles, URLs, clipboard\npreviews, and terminal commands), extracted screen images, narration text, and bounded\nterminal-output excerpts requested during analysis to your selected AI provider for processing.\nWhen protection is enabled, terminal commands and the complete local terminal transcript\nparticipate in the same on-device secret/PII scan before any requested excerpt is returned.\n\nThe in-app \"Records your screen and activity\" panel spells out exactly what's collected:\n\n- **Window tracking:** active-app / window switches.\n- **Browser URLs:** the page you're on (macOS).\n- **Screen video:** recorded by Chromium; low-rate snapshots are kept only when the\nscreen changes or a heartbeat is due.\n- **Clipboard:** short previews of copied text that tie steps together.\n- **Narration***(optional)* : spoken commentary, transcribed**on-device** in any of\nWhisper's 99 supported languages (a one-time ~252 MB model download on first use).\n- **Recorded terminal***(only when opened from the floating recording bar)* : command,\ncwd, shell, exit status, duration, and the complete terminal output. The transcript\nhas no app-imposed size cap and counts toward the saved session's storage. It captures\nonly the app-owned terminal for that recording—there are no global hooks, profile\nedits, background command monitors, or observation of existing terminals.\n\n**Please don't capture secrets.** Passwords, access tokens, API keys, credentials, and\nother confidential information should never be recorded, typed, pasted, shown, copied,\nor narrated during a session.\n\n⚠️ \n\n**For developers changing the code, not for installing the app.** To use Skill\nRecorder, follow [Install it](#install-it) instead.\n\nRequires **Node.js 24**. After checking out a release revision:\n\n```\nnpm ci\nnpm run compliance:licenses\nnpm run dev\n```\n\n`npm run dev` starts Vite and launches the Electron app with hot-reload; `⌘⇧R` (macOS) /\n`Ctrl+Shift+R` (Windows) toggles recording from anywhere. Full manual setup, the build and\n`dist` scripts, and the licensing boundary between local source builds and redistributable\npackages are in [`INSTALL.md`](https://github.com/microsoft/skill-recorder/blob/main/INSTALL.md). Maintainers changing versions, dependencies,\nassets, or releases must follow [`RELEASING.md`](https://github.com/microsoft/skill-recorder/blob/main/RELEASING.md).\n\nThe Copilot **describer** and **builders** have a fixture-based eval suite; see\n[`evals/README.md`](https://github.com/microsoft/skill-recorder/blob/main/evals/README.md).\n\n```\nnpm run eval            # score the describer against synthetic recordings\nnpm run eval:terminal   # score indexed terminal-output use and redaction\nnpm run eval:builder    # score the skill/automation generalization\n```\n\n- **[INSTALL.md](https://github.com/microsoft/skill-recorder/blob/main/INSTALL.md):** install options, inspect-first install, updating,\nuninstalling, and manual developer setup.\n- **[RELEASING.md](https://github.com/microsoft/skill-recorder/blob/main/RELEASING.md):** maintainer release runbook.\n- **[evals/README.md](https://github.com/microsoft/skill-recorder/blob/main/evals/README.md):** the describer / builder eval harness.\n- **[THIRD-PARTY-NOTICES.md](https://github.com/microsoft/skill-recorder/blob/main/THIRD-PARTY-NOTICES.md):** licenses for bundled dependencies.\n\nPlease don't report security vulnerabilities through public GitHub issues. See\n[`SECURITY.md`](https://github.com/microsoft/skill-recorder/blob/main/SECURITY.md) for Microsoft's coordinated-disclosure process and reporting\nchannels.\n\nFile bugs and feature requests through\n**[GitHub Issues](https://github.com/microsoft/skill-recorder/issues)** (search existing issues\nfirst to avoid duplicates). Support is limited to the resources described in\n[`SUPPORT.md`](https://github.com/microsoft/skill-recorder/blob/main/SUPPORT.md).\n\nThis project welcomes contributions and suggestions.  Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit [Contributor License Agreements](https://cla.opensource.microsoft.com).\n\nWhen you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft\ntrademarks or logos is subject to and must follow\n[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.", "url": "https://wpnews.pro/news/skill-recorder-desktop-app-that-records-your-on-screen-work-session", "canonical_source": "https://github.com/microsoft/skill-recorder", "published_at": "2026-11-14 08:40:42+00:00", "updated_at": "2026-11-14 08:49:46.300011+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "artificial-intelligence"], "entities": ["Microsoft", "Skill Recorder", "GitHub Copilot CLI", "Electron", "Windows ARM64"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/skill-recorder-desktop-app-that-records-your-on-screen-work-session", "markdown": "https://wpnews.pro/news/skill-recorder-desktop-app-that-records-your-on-screen-work-session.md", "text": "https://wpnews.pro/news/skill-recorder-desktop-app-that-records-your-on-screen-work-session.txt", "jsonld": "https://wpnews.pro/news/skill-recorder-desktop-app-that-records-your-on-screen-work-session.jsonld"}}