Writing tests by hand cannot keep up with how fast software changes.
Reads a GitHub repo, scores the risk and generates runnable Playwright tests, learning each sprint.
Cost per run cut from ₹50 to ₹8, an 84% reduction, with quality held.
The problem
Test coverage rarely keeps pace with application change. Manual test authoring is inherently slower than development velocity, creating a structural gap through which real defects reach production — not from lack of diligence, but because manual processes cannot scale to match modern release cadence.
Who it is for
- Engineering organizations operating without dedicated QA function, requiring test coverage without justifying full-time headcount allocation
- QA leadership at scaling organizations requiring test coverage growth that outpaces headcount growth
- Engineering management requiring visibility into under-tested codebase areas, beyond aggregate pass/fail metrics
Market context
This represents an active, competitive category with established, well-capitalized incumbents — Mabl, testRigor, Functionize, Testim, and Katalon — each offering AI-assisted test authoring and self-healing maintenance. The category divides into two dominant models: low-code platforms requiring test authorship within proprietary tooling, or fully managed services operated by external teams.
The gap it closes
Established platforms require migration of testing operations into proprietary environments. QAIP instead operates directly against existing GitHub repository and Jira infrastructure — producing standard Playwright code retained in customer-owned repositories, independent of vendor platform lock-in, with capability that compounds through retrieval-augmented memory of prior test patterns.
How it works
- QAIP is configured against the target GitHub repository and associated Jira project
- The system evaluates story requirements, identifies coverage gaps, and generates corresponding Playwright test code
- Generated tests execute automatically against the live application environment
- Failures are translated into plain-language explanations, with defect tickets generated automatically
- The system retains generated patterns, improving subsequent test generation quality on the same codebase over time
What makes it different
- Proprietary code ownership, not vendor lock-in — generates standard Playwright code within your own repository, eliminating dependency on a third-party authoring platform
- Codebase-specific institutional learning — retrieval-augmented memory of prior test patterns, engineered for your architecture rather than a generic model applied uniformly across customers
- Verified economic impact — a published, audited 84% cost reduction achieved through deliberate model routing, presented as engineering fact rather than marketing claim
Architecture
System Overview
Component Breakdown
| LangGraph State Machine | Owns pipeline sequencing, observability, and state persistence across the 5 outer stages |
| CrewAI 5-Agent Crew | Nested specifically inside test generation — five specialist roles reviewing each other's work |
| pgvector Memory Store | Stores every previously-generated test case; retrieves the 3 most similar past patterns |
| Model Router | Routes test-generation to Groq (fast), defect-explanation to Claude (faithfulness-critical) |
Data Flow
- Jira story text enters — only acceptance criteria extracted, reducing tokens from ~4,000 to ~800
- pgvector queried for the 3 most similar historically-generated tests
- CrewAI crew generates new test code, cross-reviewed by 5 specialist roles
- Playwright executes generated tests against the real, live application
- Failures explained via Claude specifically — faithfulness over speed at this stage
- A Jira defect ticket is raised automatically, linked to the failing test
Technology Stack
| Outer orchestration | LangGraph | Explicit state graph for control and observability |
| Nested collaboration | CrewAI | Role-based pattern fits specialist review better than raw nodes |
| Test generation | Groq | Fast; output variance acceptable at this stage |
| Defect explanation | Claude | Faithfulness-critical, validated via deepeval |
| Vector memory | PostgreSQL + pgvector | Institutional memory, queryable by similarity |
Real Design Trade-off, Documented
Running two frameworks instead of one means more moving parts. Hand-rolling the review logic as more LangGraph nodes, or running everything as one CrewAI crew, were both considered and rejected — LangGraph's state graph was judged necessary for observability CrewAI alone doesn't provide.
Performance, Measured Not Estimated
- Cost per run: ₹50 → ₹8, a real 84% reduction (model routing + prompt compression + RAG context selection)
- Node 3 latency: 42 seconds → 8 seconds, after prompt compression
- Faithfulness held at 94.2% throughout the cost optimization, confirming quality wasn't traded away
See QAIP working
A guided walkthrough, one-on-one. We usually reply within one business day.