If you're using pytest doctests, Jupyter notebooks, custom shell scripts, or manual spot-checks to verify code examples in your docs, this guide shows how to replace them with DocsCI — typically in under an hour, with zero friction for authors.
Migration effort: ~30 min for most repos.
| Before (ad-hoc) | After (DocsCI) |
|---|---|
| Run tests manually before releases | Runs on every push automatically |
| Only Python supported via pytest | Python + JavaScript + TypeScript |
| No accessibility or copy checks | a11y + copy lint built in |
| No API drift detection | OpenAPI spec drift detected |
| Failures discovered late | PRs annotated with precise fixes |
| Test infra maintained by you | Zero infra: hosted, scalable |
| Custom scripts per team | Single docsci.yml per repo |
| No AI fix suggestions | AI patch diffs for every error |
Go to Projects and click + New Project. In the wizard:
docs)Click ▶ Run CI on the project page. DocsCI will immediately find the same issues your doctests would — plus accessibility and copy problems.
Tip: review the findings before committing the CI template — this gives you a baseline.
Create docsci.yml in your repo root to match your existing test scope:
# docsci.yml
docs:
path: docs # where your Markdown lives
include:
- "docs/**/*.md"
exclude:
- "docs/internal/**" # exclude internal drafts
snippets:
languages: [python, javascript] # match your existing doctest languages
skip_patterns:
- "# doctest: skip" # reuse your existing skip markers!
- "# docsci: skip"
checks:
accessibility: true
copy_lint: trueThe skip_patterns field is backward compatible — your existing # doctest: skip markers will work.
Pick the template for your CI system:
Set DOCSCI_TOKEN as a CI secret and DOCSCI_PROJECT_ID as a variable.
Once DocsCI is running clean, remove the ad-hoc doctest infra:
Remove
pytest --doctest-glob
Replace with
DocsCI snippet execution
Remove
nbval / nbmake (Jupyter)
Replace with
Extract code cells to .md and run via DocsCI
Remove
Custom bash test script
Replace with
docsci.yml + /api/runs/queue
Remove
Manual review before releases
Replace with
PR comments with AI patch diffs
If your docs already use skip markers, add them to docsci.yml:
snippets:
skip_patterns:
- "# doctest: skip" # pytest doctest marker
- "# doctest: +SKIP" # alternate pytest marker
- "# type: ignore" # mypy ignore (usually means untestable)
- "# noqa" # flake8 ignore
- "# docsci: skip" # DocsCI native marker
- "// @ts-nocheck" # TypeScript skip
- "/* eslint-disable */" # JS linting disableQ: Can I keep running pytest doctests alongside DocsCI?
A: Yes. DocsCI is additive — run both. Over time you can remove doctests as DocsCI covers the same ground plus more (a11y, drift, copy).
Q: What if my code examples need environment setup (pip install, env vars)?
A: Use skip_patterns to skip snippets that require specific setup. DocsCI sandboxes are ephemeral — they can't install arbitrary packages yet (roadmap: custom runtime images).
Q: How do I handle examples that are intentionally wrong (showing error output)?
A: Add `# docsci: skip` to those snippets, or configure a custom skip_pattern like `# expected error`.
Q: Can I migrate GitBook / Mintlify / ReadMe docs?
A: Yes — DocsCI works with any Markdown. Point docs.path at the folder containing your .md files.
Q: Do I need to change how authors write docs?
A: No. Fenced code blocks work as-is. Authors only need to add `# docsci: skip` to examples that can't run.
Create your first project and run a check in under 5 minutes. No credit card required.