Codemix for QA Engineers

Know what correct looks like before you start testing.

QA can only test against a spec it can see. Too often the real spec lives in tickets, Slack threads and whoever happens to be available.

Codemix keeps the product's concepts, flows, rules, and decisions in a queryable model. Before you test, you can ask what correct means. That's very different from inferring it from the implementation.

Review suggestions

Resolve ambiguous behaviour before accepting the proposal.

1 open 1 approved 1 rejected

Accept proposal Back to draft

It's not clear what owns the decision about tenant access

open

The proposal doesn't specify which part of the product is authoritative when a plan changes. If that's ambiguous, different parts of the system could make conflicting decisions about what a tenant is allowed to do — and access could be stale or inconsistent depending on which path is hit.

Suggested resolution

The spec should state clearly: when Stripe reports a subscription change, what is the single source of truth for tenant access? That owner should be the only thing that updates access level, and everything else should read from it.

Approve Reject

The same plan change could be applied more than once.

approved

If Stripe sends the same notification twice — which it does under normal retry conditions — the proposal doesn't guarantee that only the first delivery has any effect. A tenant's access could be updated twice, which risks putting them in an incorrect state.

Suggested resolution

The spec should commit to a clear model: either each notification is applied exactly once, or every update is safe to apply repeatedly. The proposal should state which it is and how it's enforced.

Clear

New billing signals from Stripe would be missed silently.

rejected

If Stripe introduces a new type of billing event that Stratum should react to — a new plan tier, a trial conversion, a payment recovery — it will arrive and nothing will happen. There's no way to know a signal was received but ignored.

Suggested resolution

The product should have a defined list of billing signals it handles, and anything outside that list should surface somewhere visible. Ignoring a billing signal should be a deliberate choice, not the default.

Proposal review

Catch the problems before the work starts.

Codemix flags gaps in the proposal — ambiguities, missing cases, or things that conflict with how the product already works. Resolve them here, and the work that follows is far less likely to need rethinking.

The intent gap

You can only test against a spec you can actually see.

QA engineers are good at testing. The harder part is knowing what correct behaviour is, especially for undocumented edge cases and agent decisions nobody reviewed.

When intent is implicit, the implementation becomes the spec. Testing against that tells you what the product does. It does not tell you whether it is right.

Correct behavior is often implicit

When the spec lives in threads and heads, QA has to infer correct behaviour. Sometimes the inference is wrong.

Edge cases with no documented decision

Agents fill gaps with plausible behaviour. QA gets no signal about which edges were approved and which were invented.

Testing the implementation, not the intent

Testing what the product does is not the same as testing whether it does what was decided.

Ambiguity becomes sign-off risk

When correct is unclear, sign-off becomes judgement. The later bug report is just the expensive version of an unanswered question.

Start from what was decided

Know what the product is supposed to do before you start testing

QA tests against the spec. The problem is that the real spec often is not queryable.

Codemix keeps product intent structured: concepts, flows, rules and decisions. Before writing cases, you can ask what correct means.

Demo

Multi-tenant analytics SaaS platform

Work in motion

Proposals 2
Reviewing 1
Approved 1
Tasks 5
Running 1
Ready 3
Blocked 1
Completed 3

Model inventory

  • Requirements Intent 8
  • Concepts Domain 14
  • Personas Audience 3
  • Flows Journeys 9
  • Screens Surfaces 6
  • Stories Behavior 18
  • Systems Mechanics 5

Build Checklist

TECHNICAL READINESS

  • All critical API endpoints covered by integration tests
  • Rate limiting in place on public endpoints
  • Database backups verified and restore tested

PRODUCT & ONBOARDING

  • Onboarding flow end-to-end tested with external user
  • Help docs cover the top 5 user questions
  • Feedback channel (Slack / email) ready and monitored

Activity

01 Proposal sent to review 3/10/2026
02 Task plan generated from approved scope 3/10/2026
03 Project phase changed from explore to build 3/10/2026

Project overview

See everything in flight, at a glance.

The overview surfaces every active proposal, running task, and recent conversation in one place — no status updates required. Codemix keeps it current as work progresses, so anyone on the team can see what's happening and where things stand.

Test edge cases against real decisions

Know which edge cases were decided and which were assumed

The hard edge cases depend on decisions. Those decisions are often the least visible part.

Codemix keeps decisions, constraints and edge handling with the product context. You look them up instead of guessing.

React to plan changes in real time

Ready to execute from the accepted proposal.

Overview

Stratum currently has no way to respond when a customer's subscription changes. Stripe sends a notification every time a plan is upgraded, downgraded, or cancelled — but the product ignores them. This means a tenant's access level in the product can be out of step with what they're actually paying for.

This work makes billing changes real. When Stripe reports a subscription change, the product reacts immediately: the tenant's access updates, and everyone on that account sees the correct limits before their next action.

Behavioral change (what must now be true)

  • When a customer upgrades, their new plan limits apply before their next action in the product.
  • When a customer downgrades, their access steps down at the end of their current billing period.
  • When a customer cancels, they keep full access until their paid period expires, then move to the free tier.
  • If the same billing notification arrives more than once, the tenant's access is updated exactly once.
  • If a notification can't be verified as genuine, it is rejected and Stripe will automatically retry.

What to implement

  1. Receive plan change notifications — The product listens for billing notifications from Stripe. Each notification describes a change in subscription state — an upgrade, a downgrade, or a cancellation.
  2. Verify each notification is genuine — Before acting on any notification, the product confirms it actually came from Stripe. Notifications that can't be verified are rejected, causing Stripe to retry delivery.
  3. Apply each notification exactly once — Under normal conditions, Stripe may send the same notification more than once. The product records each notification it has acted on, so duplicates have no effect.
  4. Update tenant access — Once a notification is verified and accepted, the tenant's access level is updated to reflect their current plan. Upgrades apply immediately; cancellations apply at period end.
  5. Confirm the behaviour end to end — Tests verify the product-level outcomes: that upgrades take effect, that cancellations move tenants to the free tier, that duplicates are safely ignored, and that forged notifications are rejected.

Out of scope

  • Sending email notifications when subscription state changes.
  • Changing which features are available on each plan.
  • Reacting to payment failures or invoice events.
  • Registering the notification endpoint with Stripe.

Task plan

A complete task plan, ready to execute.

Once a proposal is approved, codemix breaks it into parallelisable tasks and executes them autonomously. The task plan keeps every behavioural commitment visible while the build is underway.

Ask what the right behavior is

Get the answer when a test case depends on knowing the rule

Edge cases surface during testing. That is fine. Hunting through old threads is not.

Codemix lets you query the product model during testing. You get the rule for the case in front of you, grounded in approved intent.

Proposal

When a customer upgrades their plan, how long does it take for their new limits to apply?

Right now, they don't apply automatically. Stripe notifies Stratum when a subscription changes, but the product doesn't react — so a tenant can pay for a higher tier and still be on their old limits until someone manually updates the database.

The gap is that there's no handler for plan change notifications. Until that exists, billing reality and product access are out of sync.

Want me to put together a proposal for closing this?

Yes. Make sure it covers upgrades, downgrades, and cancellations.

Proposal ready. Here's what changes for tenants:

  • Upgrade: new limits apply before their next action in the product
  • Downgrade: access steps down at the end of their current billing period
  • Cancellation: they keep full access until the period they've paid for expires, then move to the free tier

I've broken the work into 5 tasks that can run in parallel once the core notification handler is in place.

Verify what shipped against what was approved

Check whether the product conforms to its own intent

The final QA question is not just whether the tests pass. It is whether the product matches the decision.

Codemix compares current behaviour with approved intent. It catches the divergence that compiles cleanly and fails conceptually.

How it works

How Codemix carries context through the full lifecycle

Most product drift happens in the handoffs: from discussion to spec, spec to task, task to code, and code back to the roadmap. Codemix keeps those handoffs explicit.

  1. Start with a conversation or from code.
  2. Model the product.
  3. Shape the change.
  4. Write the tasks.
  5. Answer questions mid-build.
  6. Review what changed.
  7. Ship, then update the spec.
  8. Bring the context forward.

Pricing

Codemix pays for itself on day 1.

Build better apps with codemix faster than ever before. Save time, stop wasting tokens and keep humans in control.

Pay As You Go

for solo builders

$50 pro user / month, for teams
20k credits per pro user / month

Solo

  • For one-person workspaces that want to try codemix without committing to a subscription.
  • Pay as you go
  • Single-user workspace
  • Try for just $20
  • Upgrade to Pro for Slack, hosted code analysis, invites, and code review

Pro

Self-serve pricing for teams ready to start now.
$50 per user / month
$500 per user yearly. 2 months free.

  • 20,000 pooled credits per user
  • Monthly or annual billing
  • Start immediately

Enterprise

For teams that need custom terms, SSO, compliance, or support.

  • Contact sales for custom pricing
  • SSO and Access Control
  • Custom terms
  • Compliance and support

Ready to test with more confidence?

Stop testing the accident. Test the decision.

Codemix gives QA engineers the product model to write test cases from intent, resolve edge case ambiguity and sign off against the product's actual specification.

No credit card required.