Orca cuts cloud-security query costs with a Bedrock agent

Cloud queries without a specialist language

Orca Security and AWS have described how Discovery AI turns natural-language security questions into structured filters for Orca’s Sonar data graph. The 13 October account says Sonar covers more than 900 cloud infrastructure and application models, along with nested relationships and 18 comparison operators. Constructing the underlying JSON directly therefore requires both security knowledge and detailed familiarity with the schema.

Discovery AI gives analysts a conversational entry point while retaining the existing query system underneath. That distinction is important. The agent is not being asked to invent an answer from general model knowledge; it must produce a valid filter that can be executed against an organisation’s own current cloud data. The resulting evidence remains tied to resources and relationships visible in Sonar.

Why the first prototype was too expensive

Orca initially placed the complete schema and a large set of examples into a prompt for the largest available model. The approach produced encouraging output but consumed roughly two million input tokens for a comparable workload and took about 60 seconds end to end. Complex requests spanning several resource models still needed refinement. A promising demonstration was not yet a sustainable production design.

The lesson applies beyond security products. Sending every piece of context to one model can simplify an early prototype, but it transfers retrieval, selection and reasoning into an expensive prompt. Production systems benefit when deterministic code handles exact patterns, retrieval narrows the evidence and different models are assigned to stages that match their cost and capability.

A nine-stage pipeline narrows the problem

The production architecture uses a nine-stage pipeline. It begins with request and organisation context, then a deterministic matcher detects high-confidence inputs such as IP addresses, CVEs, ARNs, hashes and resource identifiers. A guardrail screens unsafe, irrelevant or unsupported prompts. Semantic retrieval finds 20 similar validated examples, and Cohere Rerank 3.5 selects the five strongest examples for later generation.

Subsequent stages construct, validate and refine the Sonar filter. Amazon Bedrock provides access to different foundation models through one interface, letting Orca allocate models by stage rather than forcing one model to do everything. AWS says the implementation also uses Claude Haiku 4.5, Amazon Nova 2 Lite and Titan embeddings. The model-agnostic arrangement allows components to change without rebuilding the entire pipeline.

Measured gains in production

AWS reports that input-token use fell by 93 per cent, from about two million to 140,000 for a comparable workload. Overall cost declined by 90 per cent, and latency fell by 75 per cent, from about 60 seconds to 15. Discovery AI has handled thousands of queries a day in production since March 2026, with quality monitored through expert review and an evaluation harness.

These figures are more useful than a generic claim of efficiency because they describe the original baseline, the deployed workload and the dimensions that changed. Organisations should still avoid assuming the same percentages will transfer to another schema. Retrieval quality, request mix, model pricing and the cost of validation all influence the result.

A pattern for governed enterprise agents

The architecture illustrates a practical way to place an agent in front of sensitive operational data. It applies deterministic fast paths where possible, rejects unsuitable requests before generation, limits retrieved examples and validates the model’s structured output. AWS governance controls such as IAM, private networking and CloudTrail support the surrounding service boundary, while Orca remains responsible for application-level access and query semantics.

For security teams, natural language can widen access to powerful investigation tools, but it should not erase the underlying controls. Analysts need to inspect generated filters, understand scope and retain an audit trail connecting the question to the executed query. Discovery AI’s strongest contribution is not conversation alone; it is the disciplined pipeline that makes the conversation operationally useful.

Quality remains an operational discipline

Orca’s human-in-the-loop labelling and evaluation harness are essential because a syntactically valid filter can still misunderstand an analyst’s intent. Useful test sets should include simple identifiers, ambiguous resource names, multi-cloud relationships and requests the system ought to reject. Teams should monitor false negatives as carefully as false positives: a narrow query that misses an exposed asset may look clean while creating false reassurance.

Production monitoring should also separate retrieval failure, model failure and data freshness. If an answer deteriorates after a schema change, engineers need to know whether examples were stale, reranking selected the wrong pattern or the generator produced an invalid relationship. The staged pipeline makes that diagnosis possible because each decision leaves a boundary that can be measured and tested independently.

Discovery AI also demonstrates why savings should be reported per successful workload rather than per model call. A cheaper stage is not useful if it increases rework or causes analysts to run several follow-up queries. Orca’s expert validation can connect infrastructure metrics with the quality of the final filter, giving the company a basis for deciding when to use a deterministic shortcut, a smaller model or a more capable generative stage.