Generative AI services combine a model with data access, prompts, retrieval, software, review, security, and ongoing operations. Buying model access is therefore not the same as buying a dependable service. Leaders need to know which task is bounded, what information may cross the boundary, how outputs are checked, who remains accountable, and how the service changes or stops.
This generative AI services FAQ is a practical decision guide. Edilec's scope, cost, and delivery plan, implementation checklist, and data and AI FAQ provide useful companion detail.
Key takeaways
- Buy or build a bounded workflow outcome, not a general promise to add AI.
- Classify prompts, retrieved context, outputs, feedback, and logs before sending real data.
- Evaluate on representative tasks with explicit rubrics, adversarial cases, latency, and cost.
- Keep consequential authority outside the model and enforce permissions in ordinary software controls.
- Version the full service, monitor production behavior, and retain provider exit and manual fallback plans.
Scope and buying questions
What is a generative AI service?
It is an operated capability that generates text, code, images, audio, or structured proposals within a workflow. The service includes user experience, model and version, system instructions, context sources, tools, output validation, identity, logs, evaluation, support, and change management. A standalone chat subscription may be a product, but enterprise use still creates an operating responsibility.
What is a good first use case?
Choose high-volume work where a person already reviews the output, errors are detectable, data is permitted, and failure is reversible: draft a response from approved material, summarise a non-consequential case, or classify work into a review queue. Avoid autonomous commitments, eligibility decisions, privileged actions, and tasks where correctness cannot be judged.
| Use case | Initial posture | Why |
|---|---|---|
| Internal draft from approved sources | Proceed with review | Bounded evidence and reversible output |
| Public factual answer | Pilot with citations and fallback | Hallucination and freshness risk |
| Code suggestion | Use isolated testing and review | Generated code can create vulnerabilities |
| Contract or clinical decision | Assist qualified expert only | High consequence and context dependence |
| Autonomous account action | Defer until authority is separately controlled | Excess agency and recovery risk |
Data and vendor questions
What data may the service use?
Only data approved for the purpose and provider configuration. Classify user input, retrieved documents, tool results, output, feedback, and logs. Check residency, retention, training use, subcontractors, deletion, encryption, access, incident terms, and export. Minimise context and redact where that preserves task value. Never place credentials or hidden authority in a prompt.
What should vendor due diligence cover?
Review service architecture, security reports, data terms, model and policy change notice, availability, rate and context limits, abuse controls, evaluation support, logging, incident process, intellectual-property terms, accessibility, portability, and termination. The NCSC secure AI guidance recommends assessing AI supply chains, documenting assets, and planning secure operation and failover.
| Vendor area | Evidence to request | Exit requirement |
|---|---|---|
| Data handling | Contracted retention, use, location, deletion | Verified deletion and export |
| Model change | Version controls and notice policy | Pinned or tested replacement path |
| Security | Identity, isolation, testing, incident terms | Credential revocation and log retention |
| Reliability | Limits, status, support, recovery history | Manual route or alternate provider |
| Portability | APIs, formats, prompts, evaluation assets | Owned workflow and test corpus |
Evaluation and oversight questions
Can we trust generative AI output?
Trust the controlled process, not an output merely because it is fluent. Build a representative test set, define task-specific rubrics, require evidence where appropriate, and measure correctness, completeness, refusal, harmful content, privacy, security, latency, and cost. The NIST Generative AI Profile identifies risks and actions across the lifecycle; teams should select controls based on context and consequence.
When is human oversight meaningful?
When the reviewer has competence, time, source evidence, clear authority, and a simple way to reject or correct. A hurried approval click after the model has already acted is not oversight. Show uncertainty and provenance, route only the fields needing judgment where possible, and sample auto-accepted low-risk outputs to detect drift.
Security and action questions
How should prompt injection be handled?
Treat user and retrieved content as untrusted data, not instructions. Separate trusted control messages, constrain context, enforce authorization outside the model, allowlist tools, validate structured outputs, and require confirmation for consequential actions. The OWASP LLM application risks include prompt injection, insecure output handling, sensitive disclosure, and excessive agency; testing should cover direct and indirect attacks.
May the model call tools or update records?
Only through a narrow application layer that authenticates the user, checks current permission and policy, validates arguments, limits amount and rate, records the action, and makes retries idempotent. The model proposes a tool and arguments; deterministic software decides whether execution is permitted. High-consequence actions require a separate approval or should remain unavailable.
Operations, cost, and compliance questions
What should a production service monitor?
Track task success from evaluated samples, refusals, citation support, policy denials, security events, human corrections, tool failures, latency, token use, cost per completed task, provider changes, and user-reported harm. Segment by workflow and version. NIST's AI RMF supports continuous governance, mapping, measurement, and management rather than one pre-launch review.
How should cost be estimated?
Model input and output charges are only one line. Include retrieval and storage, orchestration, evaluation, security, observability, reviewer time, exception handling, vendor management, support, incident response, and rework. Measure cost per successfully completed workflow, with quality held constant. Caching and smaller models help only after privacy, freshness, and task suitability are tested.
How should teams approach changing regulation?
Inventory role, use, geography, affected people, data, model provider, and consequence, then obtain qualified advice. The European Commission's current AI Act information describes phased application, transparency duties, and risk-based obligations. Do not infer compliance from a vendor badge; obligations depend on the system and organisational role.
What should the release gate include?
- Approved use-case and data boundaries with named business, technical, security, and risk owners.
- Locked evaluation results for normal, edge, adversarial, refusal, and recovery cases.
- Current permission checks, constrained tools, output validation, and meaningful human authority.
- Versioned model, prompts, retrieval, policies, and dependencies with canary and rollback.
- User disclosure, feedback, incident, manual fallback, provider outage, and exit procedures.
What should a service contract contain?
Document the supported users, task, languages, input and output limits, prohibited uses, data classifications, model and region, retrieval sources, tools, human authority, evaluation thresholds, service objectives, support, incident contacts, and change-notice policy. State what happens when the model refuses, context is unavailable, a provider rate limit is reached, or evidence is weak. User expectations should describe limitations in the workflow, not hide them in procurement files.
Attach an ownership map. The business owner accepts outcome and process risk; product and engineering own experience and operation; data owners approve sources; security and privacy owners set controls; domain experts define evaluation; procurement and legal manage supplier terms. A named service owner coordinates versions and incidents. Shared accountability is useful only when each decision still has one clear authority.
How large should an evaluation set be?
Size follows diversity and consequence, not a universal count. Cover normal work, rare but material cases, languages, long and short inputs, missing evidence, conflicting sources, refusals, malicious instructions, privacy probes, and tool errors. Define rubrics before judging and measure agreement between qualified reviewers. Keep a holdout set for release decisions and add production failures to a separate regression set after root-cause review.
Automated graders can expand coverage but need calibration against expert judgment and should not grade themselves without independent checks. Use deterministic assertions for schemas, citations, permissions, and tool calls. Report distributions and failure categories rather than one average score. A version that improves style while worsening unsupported claims should not pass because the composite number rose.
How should model changes be released?
Record the complete configuration: provider and model version, system instructions, retrieval index, tool schemas, safety settings, code, and policy. Run offline evaluation, security tests, shadow traffic, and a limited canary. Compare quality, refusals, latency, cost, and tool behavior by task cohort. Preserve the previous configuration and data compatibility long enough for rollback. Provider aliases that change underneath the service require the same scrutiny as an application dependency update.
Communicate material behavior changes to users and operators. Update reviewer guidance, support diagnostics, and incident playbooks. During a canary, keep actions narrow and independently authorised. If results degrade, stop exposure first, then investigate; do not tune prompts directly in production without a versioned evaluation trail. Reconcile any records or external actions created before rollback.
When should a service stop or be retired?
Pause when current permissions cannot be enforced, data terms are breached, security controls fail, evaluation falls below the agreed threshold, harmful outcomes exceed tolerance, or the provider changes behavior without enough evidence. Degrade to a manual or non-generative route where possible. Retirement is appropriate when value no longer covers operating cost, a simpler deterministic method performs better, risk cannot be reduced, or the use case has changed beyond the approved boundary.
Retirement needs its own runbook: notify users, disable entry points and tools, revoke credentials, export required records, verify provider deletion, preserve evaluation and incident evidence, remove stale retrieval indexes, update dependent workflows, and close vendor commitments. A dormant assistant with active data access remains a liability. The service lifecycle ends only when authority and data have been withdrawn.
Conclusion
Generative AI services are dependable when a bounded task is surrounded by ordinary engineering discipline and context-specific governance. Control data, evaluate the real workflow, preserve human or deterministic authority, secure every tool boundary, and operate versions visibly. The right question is not whether a model is impressive; it is whether the complete service produces useful outcomes while remaining correctable and accountable.