The unit of risk is becoming a workflow

A chatbot that returns a bad answer creates one kind of problem. An agent that reads an email, looks up customer data, calls a service and changes a record creates another. Microsoft makes that distinction central in its 2026 Responsible AI Transparency Report, published on 1 September.

The company says governing agentic systems means looking beyond an individual model or application to the interaction among models, agents, tools, data and people. Its report points to agent identities, tool permissions and action monitoring as controls that matter when a system can do more than generate text.

This is not a claim that those controls make agents safe by default. It is a useful change of focus. A model may be well tested in isolation and still behave badly once it is connected to real data, broad permissions or an unclear handoff to a human operator.

What Microsoft says it has changed

Microsoft says it re-engineered its Responsible AI Standard so that it is structured around models, platform services and applications, as well as the role Microsoft plays in each. It describes a set of core requirements and additional requirements intended for specific scenarios as capabilities and risks change.

The report also highlights tools that Microsoft says help developers evaluate agents, simulate adversarial use, apply policy checks at runtime and monitor behaviour. It names an AI Red Teaming Agent, agent evaluators, RAMPART, ASSERT and an Agent Control Specification. The names are less important than the pattern: testing is meant to continue through development and deployment, rather than be a gate applied once before launch.

Microsoft also reports training thousands of engineers and product managers on topics including agentic threat modelling and prompt-injection defences. That is a company account of its own programme. External readers cannot infer from it how every Microsoft product or customer deployment performs in practice.

What the tools can actually inspect

Microsoft's documentation for its AI Red Teaming Agent explains that it can simulate adversarial probing, score attack-response pairs and generate reports for supported model and agent workflows. For agents, its categories include prohibited actions, sensitive-data leakage and task adherence. That is a useful expansion from judging only a final answer to looking at how tools were used.

The documents make the operational point clear. A policy can say that an agent must not make an irreversible change without a person. A meaningful evaluation then needs to test whether the agent selects the right tool, sends the right input, respects the stated policy and stops for approval at the right point.

Microsoft's Foundry material similarly distinguishes system evaluation from process evaluation. It includes measures such as task adherence, tool-call accuracy and task completion. These measures can help turn a vague claim of trustworthiness into a set of testable behaviours. They do not replace a team's own understanding of its system and its real-world stakes.

The limits are part of the story

The same Microsoft documentation is unusually clear about what the red-teaming tool does not cover. Some agent-risk tests are single-turn and English-only. They use synthetic data and mock tools, and they exclude some forms of memory or training-data leakage. The documentation also says the results can be non-deterministic and non-predictive, with false positives possible.

Support is limited as well. At the time of writing, the documentation lists support for selected Foundry-hosted agents and Azure tool calls, while excluding several other kinds of agents and tools. That is normal for an evolving platform. It is also exactly why a dashboard result should not be mistaken for a broad assurance claim.

The practical lesson is modest. Use automated evaluation to find more problems, more often. Then inspect the important findings with people who understand the workflow, the data and the consequence of a mistake. A passing score is evidence about a specified test, not a certificate that a live agent will behave well everywhere.

What is confirmed, what Microsoft says, and what is open

Confirmed: Microsoft published its third Responsible AI Transparency Report on 1 September 2026. Its accompanying article says the company has revised its standard and is focusing more on agent identities, tool permissions, evaluation and action monitoring. Its published documentation describes agent-focused evaluation and red-teaming features, along with explicit limitations.

Microsoft's claims: its governance programme has become more adaptive, its tools help customers identify and manage risks, and its practices support trustworthy AI deployment at scale. Those are Microsoft's descriptions of its programme, tools and intended outcomes.

Open questions: how consistently these controls are used across products and customers, what independent testing will show about the tools' coverage, whether teams can make meaningful use of the resulting signals, how false positives and missed risks are handled, and what public evidence will demonstrate that runtime controls prevent harm rather than only record it.

Sources

  1. Microsoft — Responsible AI in 2026: How we are adapting for what's aheadPrimary Microsoft publication, 1 September 2026. Source for the transparency report, revised Responsible AI Standard and the company's stated approach to governing agentic systems.
  2. Microsoft Learn — AI Red Teaming AgentPrimary Microsoft documentation. Source for supported agent-risk tests, red-teaming workflow and stated technical, scope and interpretation limitations.
  3. Microsoft Learn — Agent evaluatorsPrimary Microsoft documentation. Source for system and process evaluation concepts, including task adherence, tool use and preview limitations.