Skip to the cards
Filed
Source
Robert Smith
Loans
645

Microsoft MAI Models and Maia 300 Target Lower AI Inference Costs

Microsoft is pairing its MAI models with Maia 300 accelerators to lower inference costs, while adding options for Foundry apps, agents and local use.

Microsoft MAI Models and Maia 300 Target Lower AI Inference Costs

Microsoft is building its own AI models and inference hardware around a clear business case: many companies need useful model results without letting token and infrastructure costs run unchecked. Its Microsoft AI (MAI) models join the options in Azure AI Foundry, while Maia 300 is designed to run inference efficiently across Microsoft’s data centre infrastructure.

That gives IT teams another route to evaluate alongside models from OpenAI, Anthropic, Meta, Mistral, DeepSeek and Hugging Face. Foundry offers access to more than 11,000 models, along with agent features and tools for grounding models in Microsoft’s graph-based IQ data sources. Until the MAI rollout, Microsoft’s own offerings in Foundry included its Cognitive Services models for vision and text processing, and the lightweight Phi series for edge use. MAI adds Microsoft’s own large scale foundation models to that mix.

MAI focuses on practical workloads and cost

Microsoft Research has worked on deep learning for much of the past decade, producing models that have placed well in benchmarks. The company’s research has often focused on specific tasks rather than the general purpose approach taken by OpenAI, Anthropic and Google. MAI marks a broader move into foundation models, with product offerings aimed at scenarios such as cybersecurity, coding and transcription.

Microsoft’s stated focus is delivering competitive results at a lower token cost. That matters as providers invest billions of dollars in AI data centres and companies weigh the cost of running larger models for everyday work. Sending large data sets into a model without considering the expense can be hard to sustain. Microsoft’s position is that more economical inference, model selection and hardware efficiency can make AI deployments more manageable. MAI describes its approach as “humanist,” with safety and environmental impact among its stated considerations.

MAI also offers a tuning service, allowing customers to use their own data to create customized versions of its models. The models may not match the reach of the most widely used general purpose systems, but their task focus gives teams a way to assess whether a smaller or more specialized option meets a particular application’s needs.

Foundry integration and agent features

Most MAI models are available through Azure Foundry. Two have narrower roles: MAI-Cyber-1.1-Flash is used in Microsoft’s MDASH security system, while MAI-Code-1.1-Flash is used in GitHub Copilot.

For more demanding applications, Microsoft offers MAI-Thinking-1 through Foundry. Microsoft says the model runs with a smaller footprint than competing models while still producing useful results. Published benchmarks show performance across a range of comparisons, including some where larger models from Anthropic and OpenAI score better. For buyers, the relevant question is whether MAI-Thinking-1’s results are sufficient for the workload at a lower operating cost.

Teams with existing Azure OpenAI API code can test MAI-Thinking-1 by deploying it in a Foundry instance and changing the API call to use the deployed model’s endpoint. Requests can also set a token limit, which gives developers a way to manage usage budgets. MAI-Thinking-1 supports function calls for agent tools and can preserve encrypted reasoning chains between requests, helping an agent continue a multi-step task without restarting its work each time.

The model can stream response data in chunks, so an application can show progress while a response is still being generated. That can help interactive experiences; a long-running workflow may instead pass outputs from one step to the next without streaming. Microsoft’s decision-scoring model can also be used in an agent router to select workflow steps and determine which models handle them.

Flash models bring inference closer to the workload

Microsoft’s Flash models are designed to run locally or in the cloud. Local inference on NVIDIA RTX-class hardware can reduce reliance on data centre compute and power, though the hardware requirements affect whether that approach makes financial sense for a given team.

MAI-Code-1.1-Flash needs more than 120GB of RAM, which puts it beyond the relatively low cost Arm-based Copilot+ PCs described in the source. A system such as the NVIDIA RTX Spark-powered Surface Ultra, configured with at least 128GB of RAM, should have enough memory. Whether buying an expensive workstation costs less than cloud inference depends on how often and how heavily teams use agents.

MAI-Cyber-1.1-Flash is used to offload work in the agents that power MDASH. Microsoft says it handles 90% of tasks previously assigned to larger, more expensive models. If it cannot complete an operation, the agent passes it to a more capable model. Microsoft also says version 1.1 costs 18% less to run than version 1.0.

Maia 300 connects model efficiency to infrastructure

Microsoft’s cost strategy also depends on the hardware running its models. The latest Azure Maia inference accelerators are optimized for Microsoft models and OpenAI models. Maia is designed as a platform spanning racks, servers and networking, so software can account for how data moves through the system instead of treating the accelerator as an isolated component.

Microsoft calls its approach “Software Defined Locally Accessed Dataflow Architecture” (SDLA). The design uses non-uniform memory access and chiplet-based silicon that separates memory and I/O functions. According to Microsoft, Maia 300 increases compute operations by 36% and improves high bandwidth memory performance by 2.5 times compared with the prior generation. The company says those changes deliver roughly twice the overall performance per watt.

A rack can contain 96 accelerators, and jobs can be distributed across multiple accelerators and racks. Microsoft says the architecture supports multiple jobs per rack and multi-tenant inference, scaling across the system as demand requires. For administrators evaluating AI workloads, the practical takeaway is to compare model quality and hardware costs against the needs of each task. MAI and Maia 300 make that cost calculation a central part of Microsoft’s AI offering.

Source: Robert Smith · www.infoworld.com

Filed alongside