Development with AI uses generative models and agents to support or perform software work: understanding a codebase, proposing code, writing tests, reviewing changes, explaining failures, updating documentation and operating tools. The opportunity is real, but output speed is not delivery performance. Teams need bounded authority, protected context, strong verification and a feedback loop that measures complete software outcomes.
Use this guide with the development with AI implementation checklist and development with AI FAQ. For systems whose product behavior includes models, the AI application development guide adds model evaluation and runtime controls.
Define the scope of development with AI
Start with tasks, not a company-wide tool mandate. Candidate tasks include code explanation, migration planning, test generation, repetitive refactoring, dependency updates, documentation and bounded issue implementation. Record the repository, data sensitivity, tool access, model provider, human reviewer, allowed actions and expected evidence. A chat assistant that drafts a function and an autonomous agent that opens a pull request have different risk and cost profiles.
Classify authority in levels: advise only; create uncommitted local changes; create a branch or pull request; execute CI; use deployment tools; or change production. Begin with low-authority work and expand only after measured evidence. NIST's AI Risk Management Framework remains a voluntary risk framework organized around Govern, Map, Measure and Manage, though NIST states that version 1.0 is under revision. Use current published material and monitor revisions.
| Use case | Potential value | Primary risk | Minimum control |
|---|---|---|---|
| Code explanation | Faster orientation and discovery | Plausible but wrong architecture claims | Citations to repository paths and engineer verification |
| Test generation | More boundary examples and scaffolding | Assertions that reproduce current defects | Requirement review, mutation or fault check and coverage inspection |
| Refactoring | Mechanical change across a codebase | Behavior drift and oversized diffs | Narrow scope, tests, diff limits and staged review |
| Issue-to-PR agent | Asynchronous delivery of bounded tasks | Excess tool authority or incomplete intent | Sandbox, allowlisted tools, reviewer and branch protection |
| Incident assistance | Faster log and runbook synthesis | Sensitive context leakage or unsafe action | Read-only access, redaction, source links and commander approval |
Control context, data and intellectual property
Inventory what the tool can receive: source, prompts, logs, tickets, customer data, secrets, build artifacts and dependency metadata. Determine whether providers retain prompts, use them for training, process them in permitted regions and expose administrative audit. Apply repository and file exclusions. Do not rely on developers to notice every sensitive string after it has been sent.
Provide curated context rather than unlimited access. Stable architecture notes, coding conventions, test commands, threat assumptions and ownership files make outputs more useful and reviewable. Keep instructions versioned with the repository. Treat external content, issue text and dependency documentation as potentially untrusted input to agents; tool calls and secrets must not be triggered merely because retrieved text asks for them.
Build an AI-assisted development control loop
A robust loop begins with an issue that states desired behavior, constraints and acceptance tests. The assistant or agent receives least-privilege context and tools, produces a small change, executes allowed checks and records its work. Automated policy, security and test gates run independently. A qualified human reviews intent and implementation. Progressive release and production signals reveal what pre-release evidence missed.
Do not ask the same model output to certify itself. Independent tests, compilers, linters, scanners and reviewers provide stronger evidence. GitHub's responsible-use documentation explicitly frames Copilot features in terms of purposes, capabilities and limitations. Apply the same discipline to any vendor: users remain accountable for assessing suggestions and the context in which they are used.
Secure AI-assisted software delivery
Agent credentials should be short-lived, attributable and scoped to a repository, branch and task. Default to read-only; permit writes through narrow APIs. Block production secrets, package publishing and deployment unless a separately approved use case requires them. Log prompts, selected context, tool calls, changed files and test results according to privacy and retention policy. Provide a rapid kill switch and token revocation path.
NIST SP 800-218A augments the SSDF for generative AI and foundation-model development; it is intended for model producers, AI system producers and acquirers. CISA and UK NCSC's secure AI system development guidelines cover secure design, development, deployment and operation. For AI coding tools, preserve ordinary software supply-chain controls while adding model, context and tool-authority controls.
Verify generated code and tests
Review generated code for requirement fit, simplicity, error behavior, authorization, data handling, concurrency, performance, dependencies, licensing and maintainability. Require tests that fail before the fix when feasible. Inspect assertions rather than trusting pass counts. Generated tests often overfit implementation or mock away the risk; add contract, integration and adversarial cases selected by a human risk model.
Limit diff size and generated dependency changes. Pin and scan dependencies, verify package identity and preserve build provenance. Run static analysis and secret scanning, but keep manual threat reasoning for new interfaces and trust boundaries. The NIST Generative AI Profile identifies risks including confabulation, data privacy, information security, intellectual property and value-chain integration. Map relevant risks to concrete development controls.
Model the cost of development with AI
| Cost area | Driver | Evidence to collect |
|---|---|---|
| Licensing and inference | Seats, model tier, tokens, agent sessions and premium tools | Cost by team and completed use case |
| Platform integration | Identity, repository, policy, telemetry and sandbox work | Initial and recurring engineering effort |
| Review and rework | Diff volume, correctness and reviewer familiarity | Review time, revisions and rejected output |
| Quality controls | CI use, security scanning, test environments and evaluation | Compute, queue time and maintenance |
| Risk and compliance | Assessment, contracts, audit, incident and data controls | Control operation and exceptions |
| Change management | Training, documentation, support and workflow redesign | Adoption, proficiency and support demand |
Compare total flow, not typing time. Baseline task completion, review time, deployment frequency, lead time, change failure, rework, maintainability and developer experience. An assistant can shorten coding while increasing review queues or change size. DORA's 2025 research describes AI as an amplifier: organizational capabilities such as fast feedback, internal platforms and clear workflows shape whether local gains become system performance.
Use a staged delivery plan
In discovery, inventory current tools, sensitive repositories, policies and delivery metrics. Select two or three low-risk tasks with motivated teams. In the pilot, configure identity, context boundaries, logging and independent gates; train reviewers; and compare complete work against a baseline. Review incidents, near misses and rejected suggestions. Expand repository and tool authority only when evidence supports it.
At scale, maintain an approved-tool catalog, use-case register, provider review, model-change monitoring and exception process. Define support for developers who encounter unsafe or low-quality output. Reassess when the provider, model, retention policy, tool access or repository sensitivity changes. Retain a manual workflow and exportable artifacts so development does not stop during provider outage or contract exit.
Example: piloting a coding agent for dependency updates
Dependency updates are a bounded first use case when repositories already have reproducible builds and tests. Give the agent read access plus permission to create a branch, but no package-publishing or deployment authority. The issue names the dependency range, prohibited major versions, supported runtimes and required checks. The agent must update the manifest and lockfile, summarize release-note risks, run tests and scanners, and open a small pull request with source links.
Build a benchmark from prior updates: routine patch, incompatible transitive change, package with a known advisory, renamed API and malicious lookalike package. Score correct completion, unsafe dependency choice, unsupported change, test quality, review time and total cost. A human verifies package identity, licensing, behavior and release impact. Reject output that broadens scope merely to make tests pass.
During the pilot, compare agent-assisted and ordinary updates of similar difficulty. Measure elapsed time, reviewer effort, CI consumption, revisions, escaped defects and developer confidence. Inspect whether more pull requests create a review queue. Exercise model outage, revoked token and hostile instructions in release notes. The receiving team should be able to disable the agent without blocking manual security updates.
Expand next to similarly bounded maintenance only if the evidence is positive. A successful dependency pilot does not justify production deployment authority or customer-data access. Each new task class receives its own risk assessment, benchmark and permissions. This prevents a useful assistant from quietly becoming an ungoverned general operator.
Pilot acceptance checklist
- Approve use cases, repositories, data classes, models, regions, retention and tool authority.
- Configure attributable identities, short-lived credentials, repository policy and emergency revocation.
- Create benchmark tasks with known acceptance criteria and representative security edge cases.
- Measure completion, review, rework, defects, delivery stability, cost and developer experience.
- Test prompt injection through retrieved content, secret blocking, unsafe tool calls and provider outage.
- Document human accountability, incident response, model-change review and supplier exit.
Key takeaways
- Adopt development with AI by bounded task and authority level.
- Protect source, prompts, tickets and tool credentials as governed context.
- Use independent automated evidence and qualified human review.
- Measure end-to-end delivery and quality, not suggestion volume or typing speed.
- Expand access only after pilot evidence, and keep provider change and exit operable.
Frequently asked questions
Does AI-generated code need human review?
For material production changes, yes. Review depth can be risk-based, but branch protection and accountability should not disappear because a model produced the diff. Low-risk mechanical changes can use streamlined review when strong tests and rollback evidence exist.
How soon should productivity gains appear?
Individual task speed may improve quickly, while team flow takes longer because policy, context, review and platform work must mature. Run a sufficiently long pilot, compare similar work and inspect quality and queue effects. Do not extrapolate a coding benchmark directly to business delivery.
Conclusion
Development with AI works best as a controlled change to the engineering system. Give models clear tasks and limited authority, protect their context, verify outputs independently and observe what reaches production. The organizations that benefit will not be those that generate the most code; they will be those that turn assisted work into small, understandable and reliable changes.