{"id":"KM-SW-0090","slug":"test-strategy-decisions-that-matter-before-the-first-build","title":"Test Strategy Before the First Build: Evidence for Critical Journeys","excerpt":"A sound test strategy starts with the behavior and risk a product must protect, then assigns fast, realistic, and security-focused evidence to the right layer before the first build.","kind":"Comparison","category":"software-engineering","tags":["test strategy","Software Engineering","custom software","comparison","IT managers"],"seoKeywords":["test strategy","test strategy guide","risk-based software testing","test strategy architecture","critical journey testing"],"authorId":"krishnam-murarka","publishedAt":"2026-06-24","updatedAt":"2026-09-09","readingTime":"14 min read","image":"/social-images/blog/edilec-photo-km-sw-0090-91362cd6c318.jpg","featured":false,"trending":false,"sourceCredits":[{"title":"Playwright Best Practices","url":"https://playwright.dev/docs/best-practices","author":"Microsoft"},{"title":"Google Testing Blog: Test Sizes","url":"https://testing.googleblog.com/2010/12/test-sizes.html","author":"Google"},{"title":"OWASP Web Security Testing Guide","url":"https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/","author":"OWASP"},{"title":"Testcontainers Documentation","url":"https://java.testcontainers.org/test_framework_integration/manual_lifecycle_control/","author":"Testcontainers"}],"researchSources":[{"title":"Playwright Best Practices","url":"https://playwright.dev/docs/best-practices","author":"Microsoft","reason":"Inspected for reliable browser test boundaries, isolation, and diagnostics."},{"title":"Google Testing Blog: Test Sizes","url":"https://testing.googleblog.com/2010/12/test-sizes.html","author":"Google","reason":"Inspected for balancing test speed, isolation, and realism."},{"title":"OWASP Web Security Testing Guide","url":"https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/","author":"OWASP","reason":"Inspected for security testing coverage across the application lifecycle."},{"title":"Testcontainers Documentation","url":"https://java.testcontainers.org/test_framework_integration/manual_lifecycle_control/","author":"Testcontainers","reason":"Inspected for reproducible dependency-backed integration environments."}],"mediaAssets":[],"status":"published","body":[{"type":"paragraph","text":"Test strategy is a product decision before it is a test tool decision. A checkout, approval, data import, or account recovery journey can fail even when individual functions pass because authority, timing, validation, and recovery are split across systems. Start by naming the promise the user must be able to trust and the harm if it is wrong, delayed, duplicated, or exposed. Then choose evidence that makes that behavior observable at the cheapest reliable layer. Google’s discussion of [test sizes](https://testing.googleblog.com/2010/12/test-sizes.html) is a useful starting point, but the right mix depends on your boundaries, data, and consequence."},{"type":"heading","id":"test-strategy-risk","text":"Start with risk and a representative journey","depth":2},{"type":"paragraph","text":"Write one journey in plain language: a person initiates a request, supplies information, receives a decision, and knows what to do when work is not complete. Identify the authority record, permissions, external dependencies, user-visible states, and correction path. A payment flow may need to reserve inventory, call a provider, create a receipt, and notify a customer; a test that only checks a button click misses the business risk."},{"type":"paragraph","text":"Rank risks by consequence, likelihood, uncertainty, and detectability. High-consequence behavior deserves realistic boundary tests even if it is technically simple. A rarely used administrator path may need stronger authorization checks than a frequently used read-only list. Link this framing to the [test strategy operations playbook](/blog/km-sw-0170/test-strategy-operations-playbook/) so the release suite includes the recovery evidence operators will need."},{"type":"table","columns":["Risk question","Example","Evidence to plan"],"rows":[["Can the wrong actor act?","A user can submit another tenant’s record","Authorization tests at API and UI boundary with positive and negative identities"],["Can a repeat create harm?","A timeout leads to a second payment or shipment","Idempotency and retry tests with persisted operation identity"],["Can a late result mislead?","An old import overwrites a newer correction","Ordering, version, reconciliation, and visible stale-state tests"],["Can a failure be recovered?","A provider fails after the local record is created","Fault injection, pending state, operator runbook, and correction verification"],["Can sensitive data leak?","An error or log includes a credential or private field","Security test, redaction check, and access review"]]},{"type":"heading","id":"test-strategy-model","text":"Model the facts and failure states before the interface","depth":2},{"type":"paragraph","text":"Define entities, lifecycle states, identifiers, time semantics, and version or rule references before writing only screen-level assertions. A fixture should represent a meaningful business state: a partially fulfilled order, a locked account, a record awaiting review, or an import with duplicate rows. Record what is authoritative and what is derived. This lets tests challenge the domain model rather than merely confirm that a component displays a value."},{"type":"heading","id":"test-strategy-fixtures","text":"Make fixtures explain the scenario","depth":3},{"type":"paragraph","text":"Name fixtures by intent and set them up through a stable boundary. Avoid a large shared database that makes test order meaningful. Include a clock, identity, permissions, and dependency response that the scenario controls. If a fixture uses anonymized production-like data, document provenance and retention. The more consequential the path, the more important it is that someone can understand why the expected result is correct."},{"type":"heading","id":"test-strategy-boundaries","text":"Set boundaries and failure rules","depth":2},{"type":"paragraph","text":"Assign tests to the boundary that can produce trustworthy evidence. A pure calculation belongs in a fast unit test; a database query needs an integration check; a customer journey across identity, API, and browser deserves a small end-to-end path. Define timeouts and retry rules per dependency, not as a single global value. A retry can hide a real outage or duplicate a side effect, so the test should make idempotency and outcome uncertainty visible."},{"type":"image","src":"/social-images/blog/edilec-photo-km-sw-0090-91362cd6c318.jpg","alt":"A paper test plan connects a critical user journey with rule, boundary, and recovery evidence.","caption":"A useful test strategy assigns each product risk to evidence that can actually prove the behavior.","width":1200,"height":750},{"type":"table","columns":["Layer","Best evidence","Common misuse"],"rows":[["Unit","Deterministic business rule, parser, reducer, or selector","Testing framework internals or duplicating every integration assumption"],["Component","Accessible interaction, visual state, input and error behavior","Mocking away the state and asserting implementation details"],["Integration","Database, queue, cache, provider adapter, or contract collaboration","Using a shared environment that makes results order-dependent"],["End to end","A small number of critical user journeys and recovery paths","Covering every permutation with slow, brittle browser scripts"],["Security","Authorization, input handling, secrets, dependency and abuse cases","Treating a scanner report as the complete security decision"]]},{"type":"heading","id":"test-strategy-realism","text":"Choose realistic dependencies deliberately","depth":2},{"type":"paragraph","text":"Mocks are useful when they isolate a local rule, but they can conceal serialization, transaction, timeout, and permission behavior. Use a real or faithful dependency when the boundary is the thing under test. [Testcontainers documentation](https://java.testcontainers.org/test_framework_integration/manual_lifecycle_control/) illustrates the value of reproducible containerized dependencies; the same principle applies to other stacks when the environment is controlled, disposable, and fast enough for the feedback loop. Keep a small contract suite for providers whose full environment is unavailable."},{"type":"paragraph","text":"Create failure modes intentionally. Stop a dependency, delay its response, return malformed data, exhaust a connection pool, or revoke an identity. Confirm that the system gives a safe state and that a person can recover. Test data cleanup and parallel execution too; an environment that passes only when run serially may be masking a shared authority problem."},{"type":"heading","id":"test-strategy-security","text":"Make security testing part of the strategy","depth":2},{"type":"paragraph","text":"Map the attack surface to the journey: authentication, authorization, input, file handling, sensitive output, rate limits, session state, and dependency callbacks. Use the [OWASP Web Security Testing Guide](https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/) to structure manual and automated checks. Add negative tests for tenant boundaries and privilege changes, then review logs and errors for leakage. Security evidence should be linked to an owner and a release decision rather than stored as a separate report no one consults."},{"type":"heading","id":"test-strategy-delivery","text":"Keep fast feedback close to change","depth":2},{"type":"paragraph","text":"Arrange the pipeline so developers get local rule and component feedback quickly, integration evidence soon after, and broader journey or security suites at a deliberate gate. A failing test should show the scenario, inputs, authority, environment, and useful diagnostics. Playwright’s [best practices](https://playwright.dev/docs/best-practices) emphasize isolation, user-facing assertions, and traceable failures; apply the same principle to other test layers. Do not increase parallelism until the suite is deterministic under parallel execution."},{"type":"paragraph","text":"Define what can block a merge, a release, or a post-release expansion. A flaky test can be quarantined only with a named owner, reason, and expiry. A security finding may need risk acceptance rather than a green pipeline. A failed end-to-end check should identify whether the user promise, environment, or test itself changed. Make these decision rules visible so “green” has a shared meaning."},{"type":"heading","id":"test-strategy-operations","text":"Operate from evidence after release","depth":2},{"type":"paragraph","text":"Production signals tell you where the test strategy is blind. Track escaped defects by journey, rollback or hotfix frequency, flaky test rate, duration, coverage of high-risk behaviors, and time to diagnose. Correlate failures with releases and dependency changes. A test suite that grows while the same authorization defect keeps escaping is optimizing the wrong evidence. Review incidents with the test owner and add the smallest check that would have caught or bounded the behavior."},{"type":"paragraph","text":"Revisit the strategy when the product changes its data authority, identity model, provider, or operating consequence. A suite that was sufficient for a read-only prototype may be insufficient when users can submit irreversible actions. Store decisions and test ownership where new contributors can find them. The [frontend performance decision guide](/blog/km-sw-0091/frontend-performance-decisions-that-matter-before-the-first-build/) is a useful example of treating a quality concern as a boundary with explicit signals rather than as a generic checklist."},{"type":"paragraph","text":"Test strategy should connect to the way work is reviewed and operated. A [test strategy guide for custom software](/blog/km-sw-0050/test-strategy-for-custom-software-a-practical-guide/) can provide a product-level comparison, while the [code review systems guide](/blog/km-sw-0098/code-review-systems-decisions-that-matter-before-the-first-build/) helps place review evidence near the code that changes the behavior. Together they make it easier to decide whether a missing check belongs in a fast local test, a boundary test, or a release rehearsal rather than adding another broad suite with weak diagnostic value."},{"type":"paragraph","text":"A strategy should also make test ownership visible during planning. Assign one person to each high-risk behavior, name the fixture or environment that supports it, and record the signal that prompts a change. When a provider, identity model, or data schema changes, review the corresponding evidence instead of assuming the old suite still represents the journey. The [test strategy operations playbook](/blog/km-sw-0170/test-strategy-operations-playbook/) is a useful reference for turning failures into owned improvement work rather than accumulating more unreviewed cases."},{"type":"paragraph","text":"Do not confuse coverage with confidence. A team may execute thousands of assertions while never exercising the exact transition that matters, such as a permission change during an in-progress save or a provider response that arrives after a correction. Keep a risk register beside the suite, revisit it at each material release, and remove tests that no longer protect a decision. The useful question is always what a passing result lets the team claim about the product."},{"type":"heading","id":"test-strategy-takeaways","text":"Test strategy takeaways","depth":2},{"type":"list","title":"What to settle before the first build","items":["Name the user promise, authority, failure consequence, and recovery path.","Use fast tests for local rules and realistic tests for meaningful boundaries.","Control time, identity, data, dependencies, and parallel execution explicitly.","Keep authorization, sensitive data, abuse, and recovery in the same risk model.","Use escaped defects and flaky failures to improve the evidence mix."]},{"type":"heading","id":"test-strategy-faq","text":"Test strategy questions","depth":2},{"type":"heading","id":"test-strategy-faq-e2e","text":"How many end-to-end tests are enough?","depth":3},{"type":"paragraph","text":"Enough to protect the highest-value journeys, boundary transitions, and recovery actions that lower layers cannot prove. Keep the set small, readable, and diagnostic. A dozen valuable journeys can be stronger than hundreds of scripts that repeat implementation details."},{"type":"heading","id":"test-strategy-faq-flake","text":"What should happen when a test is flaky?","depth":3},{"type":"paragraph","text":"Classify the cause, preserve diagnostics, and fix the boundary or fixture. If quarantine is necessary, add an owner and expiry and report the risk separately. Repeated reruns are not a strategy because they convert uncertainty into false confidence."},{"type":"heading","id":"test-strategy-faq-unit","text":"Can unit tests replace integration tests?","depth":3},{"type":"paragraph","text":"No. Unit tests prove local logic quickly; integration tests prove that serialization, persistence, authentication, and dependencies collaborate as expected. Use both where the risk crosses a boundary, and avoid paying for overlap that produces no new evidence."},{"type":"heading","id":"test-strategy-conclusion","text":"Conclusion: choose evidence before tools","depth":2},{"type":"paragraph","text":"A test strategy is the plan for knowing whether a product keeps its promise under normal use, change, and failure. Start with risk and a real journey, assign each question to the layer that can answer it, control the environment, and use production evidence to close gaps. The first build is safer when the team already knows what “works” must mean."},{"type":"image","src":"/attachments/article-media/editorial/edilec-batch108-test-strategy-evidence-path.svg","alt":"Test strategy evidence path","caption":"The six-stage path aligns representative journeys, test boundaries, realistic dependencies, release gates, and production learning."}],"faqs":[{"question":"Should end-to-end tests cover every feature?","answer":"No. Use end-to-end tests for a small set of high-value journeys and boundary risks, then cover rules and component behavior at faster layers. The set should be broad enough to protect outcomes and small enough to diagnose."},{"question":"How can a team reduce flaky tests?","answer":"Classify failures, remove hidden shared state, control time and data, wait on observable conditions, capture diagnostics, and quarantine only with an owner and expiry. Rerunning a flaky test without diagnosis hides risk."},{"question":"Do integration tests replace unit tests?","answer":"No. Integration tests prove collaboration with real boundaries, while unit tests provide fast feedback on local rules. Choose each layer for the evidence it can produce and keep overlap intentional."}],"relatedIds":["KM-SW-0078","KM-SW-0081","KM-SW-0093","KM-SW-0091"],"relatedArticleIds":["KM-SW-0078","KM-SW-0081","KM-SW-0093","KM-SW-0091"]}