Generative AI services can range from a short discovery engagement to the design, build and operation of a production application. Buyers often compare proposals that use the same label for very different deliverables. A useful scope therefore starts with the business task and ends with measurable production responsibilities: data access, model behavior, integrations, security, evaluation, release, monitoring, support and knowledge transfer.
The central buying question is not whether a provider can call a model API. It is whether the proposed system can perform a bounded task with acceptable quality and risk inside the buyer's operating environment. The model is one dependency in a larger application. Retrieval, prompts, tools, policy, identity, user experience and fallback behavior usually determine whether the result survives contact with real work.
Define what the service must deliver
Begin with a decision or task, its users, input population, expected output and consequence of error. Then specify the service boundary. A discovery may produce a use-case portfolio, risk classification and architecture options. A prototype may prove retrieval or workflow fit with representative data. A production build must also cover authentication, authorization, testing, deployment, observability, incident response and support. Managed operation adds release management, monitoring, evaluation refresh and service reviews.
| Engagement | Expected deliverables | What it does not prove |
|---|---|---|
| Discovery and assessment | Prioritized use case, baseline, data map, risk register and delivery options | That a model meets production quality |
| Proof of concept | Representative test, feasibility evidence, constraints and revised estimate | Security, scale, support or adoption readiness |
| Pilot | Limited users, production-like controls, telemetry and feedback workflow | Performance across the full population |
| Production implementation | Hardened application, integrations, evaluation suite, runbooks and handover | Ongoing quality without monitoring |
| Managed service | Operations, incident handling, controlled changes, reporting and optimization | Transfer of accountability away from the buyer |
Choose the simplest technical pattern that fits
Prompting a managed model is suitable when the task depends mainly on instructions and supplied context. Retrieval-augmented generation adds approved knowledge at request time and is useful when answers must reflect changing private material. Fine-tuning can shape behavior or task performance when sufficient appropriate examples exist, but it does not automatically make facts current. Tool-using systems can read or change external systems; that capability raises the need for authorization, constrained actions and transaction controls.
- Use deterministic search, templates or rules when they satisfy the requirement more reliably.
- Use retrieval when provenance and changing knowledge matter, and evaluate retrieval separately from answer generation.
- Use fine-tuning only with a stated objective, lawful data, a baseline and a repeatable evaluation.
- Keep business policy outside the prompt when it can be represented as testable rules.
- Give tools least privilege, allowlisted parameters, time and spend limits, and approval for consequential actions.
- Design a non-AI fallback for model, provider, retrieval and integration failure.
Understand the cost drivers
Project cost depends more on uncertainty and operating requirements than on the chat interface. Important delivery drivers include the number and condition of data sources, access approvals, document parsing, retrieval quality, workflow complexity, integrations, risk classification, evaluation depth, expected load and availability. A regulated external assistant with write access requires more assurance than an internal drafting aid with no sensitive data.
Operating cost includes model input and output, embeddings or indexing, storage, search, orchestration, safety checks, observability, network traffic, human review, support and recurring evaluation. Token price is only one line. Measure cost per completed business task and per accepted output, not merely cost per request. Cache only where freshness and privacy allow, route simple tasks to appropriately capable models, limit context, and stop loops or retries that no longer add value.
| Cost area | Questions for the proposal | Control |
|---|---|---|
| Data and retrieval | How many sources, formats, permissions and refresh paths? | Scope a representative corpus and quality gate |
| Model usage | What volume, context length, output length and model mix? | Budget limits, routing and usage telemetry |
| Integration | Which systems are read or written, and how are failures reconciled? | Contract tests, idempotency and operator queue |
| Assurance | Which security, privacy, legal and domain reviews are required? | Risk-tiered evidence and approval gates |
| Operations | Who monitors, supports, evaluates and releases changes? | Runbook, ownership, service objectives and exit plan |
Make the architecture part of the contract
A production design should show trust boundaries and data flow from user to output. It should identify identity provider, application, model endpoint, retrieval store, source systems, tool gateway, policy checks, audit events and human review. Record which organization owns each component and which data crosses a supplier boundary. Clarify whether prompts, files, feedback and logs are retained, where they are processed and how they are deleted or exported.
Treat the complete chain as a versioned artifact. Google Cloud's production guidance emphasizes end-to-end evaluation, revision history, monitoring and introspection for chains because component interactions can create failures that isolated tests miss. A useful release record includes model and endpoint version, prompt, retrieval configuration, tool schemas, policy, evaluation results and deployment identifier. This makes rollback and incident reconstruction possible.
Require an evaluation plan before implementation
The buyer should own or be able to export the evaluation assets. Build a dataset from permitted, representative examples and define what acceptable means for each task. Evaluate factual support, instruction following, completeness, refusal behavior, harmful content and task-specific correctness. For retrieval, measure whether the required source was found before judging the final answer. Use expert human review where context or consequence cannot be reduced to an automatic score.
- Include normal, ambiguous, incomplete, multilingual and adversarial inputs relevant to actual use.
- Segment results by task, source, user group and consequence rather than averaging away a dangerous failure.
- Compare with the current process and a simple non-generative baseline.
- Test permissions, citations, tool arguments, retries, timeouts and fallback end to end.
- Define regression gates for model, prompt, retrieval, policy and integration changes.
- Sample production outcomes and feed reviewed corrections into the next evaluation set.
Control security, safety and information risk
Prompt injection is a system-design problem, not a phrase that can be permanently filtered. Treat retrieved documents, web pages and tool output as untrusted input. Separate instructions from data, restrict tools independently of the model, validate output before execution and require confirmation for high-impact actions. OWASP also highlights sensitive information disclosure, improper output handling, excessive agency, supply-chain weakness, misinformation and unbounded consumption. These risks overlap and should be tested as scenarios.
The NCSC guidance places secure AI work across design, development, deployment, operation and maintenance. Ask the provider for threat models, dependency inventory, secret handling, access controls, environment separation, vulnerability management, logging, incident procedures and update policy. Model safeguards do not replace ordinary application security. Conversely, conventional controls alone do not test whether an AI system follows malicious instructions embedded in trusted-looking content.
| Risk scenario | Required design response | Acceptance evidence |
|---|---|---|
| Untrusted content changes behavior | Content isolation, constrained tools and adversarial tests | Attack suite and documented residual risk |
| Sensitive data appears in output | Minimization, authorization and output checks | Permission and disclosure test results |
| Unsupported answer is treated as fact | Grounding, source display and human review by consequence | Citation support and harmful-error rate |
| Agent performs an unintended action | Allowlist, least privilege, limits, preview and approval | Action simulation and rollback test |
| Provider or model changes | Change notice, regression suite and fallback | Replacement and rollback exercise |
Put delivery and exit terms in writing
Define acceptance against artifacts and measured behavior, not a broad promise to deliver an AI assistant. State ownership and permitted use of source code, prompts, evaluation sets, indexes, generated data and feedback. Record subprocessors, data locations, retention, security notification, service objectives and responsibility for provider changes. The buyer should receive architecture records, deployment instructions, runbooks, known limitations and a tested export or transition path.
Use an evidence-gated delivery plan
Phase one establishes outcome, baseline, risk, users and data permissions. Phase two tests the highest uncertainty with representative material. Phase three designs the production architecture and evaluation suite. Phase four builds the narrow workflow with controls and telemetry. Phase five pilots with trained users, captures corrections and exercises fallback. Phase six releases gradually, monitors quality and cost, and transfers operating knowledge. Each phase should end with an explicit decision to proceed, change scope or stop.
Key takeaways
- Define deliverables and acceptance evidence for every engagement phase.
- Choose the simplest model, retrieval and tool pattern that meets the task.
- Estimate data, integration, assurance and operating costs beyond model usage.
- Require representative evaluation, constrained actions and lifecycle security controls.
- Preserve ownership, observability, rollback and a tested supplier exit path.
How long does a generative AI project take?
There is no responsible universal duration. Timing depends on data access, risk reviews, integrations, evaluation material, workflow complexity and production assurance. Request phase estimates with entry assumptions, exit evidence and dependencies. A short feasibility test and a production rollout should be estimated separately.
Can generative AI services be fixed-price?
Bounded discovery or implementation can be fixed-price when inputs, environments, integrations and acceptance tests are known. Where data or model feasibility is uncertain, price an initial evidence phase first and re-estimate from its findings. Otherwise the proposal will hide uncertainty in exclusions, contingency or reduced assurance.
Should the provider choose one model at the start?
Choose against task evaluations, security and data requirements, latency, availability, context needs and total operating cost. Preserve an abstraction only where it supports a realistic fallback or comparison; forced portability can suppress useful provider features. Re-evaluate material model changes before release.
How should service success be measured?
Combine business outcome, task quality, risk, adoption, reliability and cost. Examples include completed cases, accepted drafts, supported-answer rate, harmful errors, manual corrections, latency, incidents and total cost per completed task. Agree targets and the measurement owner before implementation.
Conclusion
Good generative AI services turn uncertainty into evidence. The scope names the task and production boundary; the architecture constrains data and actions; evaluation defines acceptable behavior; commercial terms preserve accountability and exit options; and rollout advances through measured gates. That is the difference between purchasing a demonstration and commissioning an operable system.