Skip to content
  1. Home
  2. Articles
  3. Reserved GPUs You Do Not Fully Use: Where Inference Should Run
113
AI & Infrastructure

Reserved GPUs You Do Not Fully Use: Where Inference Should Run

Reserved GPUs You Do Not Fully Use: Where Inference Should Run

AWS reserved GPU prices moved several times in 2026, and some survey respondents report paying for capacity they do not use. How to decide where inference runs, judged by how predictable the load is.

What the sources say, and what they do not prove

Suppose your team reserves GPU machines for a quarter to run inference for an internal chatbot, and at month end the usage report shows the machines idle for more than half the hours you paid for. That figure appears in no quotation. It appears in the budget. The question it raises is where your inference should have run in the first place.

We use two sources. The first is the AWS pricing page for EC2 Capacity Blocks for ML, which we opened ourselves on 12 November 2026. It says reservation prices are updated regularly based on trends in supply and demand, and that the current prices are scheduled to be updated next in January 2027. A purchase is billed at the rate in effect on the day of purchase, even if the block starts after an update. Pricing has two parts: a reservation fee paid up front and an operating-system fee that applies while instances run.

The second is an article by Louis Columbus in VentureBeat on 4 November 2026, on the AI race raising companies' bills. It reports that AWS raised Capacity Blocks for ML prices in 2026 by about 15 percent on 4 January, roughly 10 percent for some GPU configurations in April, about 20 percent on 1 July, and approximately 15 percent for most U.S. GPU configurations on 7 October. Those percentages are VentureBeat's reporting. We could not confirm them on an AWS page, and the AWS pricing page itself shows no history of increases.

The same article reports a VentureBeat Intelligence survey fielded in October 2026: 172 responses, 156 qualified, from people at organisations with 100 or more employees plus consultants, about 87 percent of them through a research panel. Respondents chose to take the survey, so the figures describe only them. Of those, 38 percent named a cost overrun as a reason an AI project was delayed, scaled back or cancelled in the past 12 months, and 12 percent named a shortage of computing capacity; the article does not state the base for those two. Among the 114 who reserve capacity in advance, 31 percent said more than a quarter of their AI compute spend goes to capacity they reserved but did not use, and 22 percent did not know.

What this proves: AWS reservation prices move over time, and some respondents report unused reserved capacity. What it does not prove: a market share, the prices a Thai organisation would get, or whether reserved capacity pays off in general. We treat it as a reason to question your own numbers.

Cost per used hour

The hourly price on a quotation tells you what you pay per reserved hour. It does not tell you what you pay per hour in which work actually ran. The formula we use: effective cost per used hour equals what you pay for the reservation divided by the share of it you actually use.

The example below is invented to show the arithmetic, and reflects no provider's price. Reserve a machine for 200 hours at 10 units per hour, 2,000 units in total, and run work for 120 of those hours. That is 60 percent use. The cost per used hour is 2,000 divided by 120, about 16.7 units, which is exactly 10 divided by 0.6. A reservation used 60 percent of the time costs two thirds more per useful hour than its sticker price.

That divisor is what the quotation never shows.

Now suppose on-demand costs 15 units per hour of actual running. The break-even for the reservation is 10 divided by 15, about 67 percent use. Below that, paying full price for only the hours you use is cheaper. All of this is invented; in a real contract, replace the numbers with those in each vendor's quotation, and check each quote yourself, because we do not assert whether any third-party price includes VAT.

Agree what "used" means before you measure. Counting only hours with requests gives a lower figure than counting hours an instance was on, which is what an administrator's dashboard usually reports because it is easy to pull. We suggest the first definition, written into the report so months can be compared. One more point from the AWS page: the operating-system fee applies while instances run, so idle time falls on the reservation fee, which is paid up front regardless. Other providers may charge differently.

Four places inference can live

A hosted model API charges per call and you maintain no machines. On-demand cloud rents GPUs by the hour and you run the software. Reserved capacity trades commitment for certainty of supply or a lower rate. Your own hardware, including a corporate data centre, puts everything in your hands.

Each moves risk somewhere else. An API moves utilisation risk to the provider, but you keep unit-price risk and the question of what data leaves your boundary. On-demand gives flexibility, and you accept the risk that machines may not be available when you want them. Reserved moves availability risk to you in the form of commitment. Your own hardware gives a price that does not follow the market after purchase, but it carries the same condition as a reservation: enough work must run on it, or the sunk cost divided by few used hours produces the same high number.

Five variables that decide it

We would not decide on the headline price. These five give a picture you can defend upward.

Steady or bursty load

A load that is steady through the day makes it easy to fill a reservation. A load that spikes briefly and disappears leaves it idle almost always. For the second kind, an API or on-demand usually fits the usage pattern better.

Hours per day the load really runs

A system used only in office hours runs roughly eight to nine hours a day. Reserve a machine for 24 hours and the rest is idle time you pay for. Compute this ratio before signing, because it sets the divisor in the formula above.

Which data class may leave your boundary

Data sent to an external API must first clear your data classification and policy. If some classes may not leave, the remaining options are your own hardware, or a cloud under terms and regions your organisation has approved. For the legal requirement itself, ask your counsel; we state no statute or penalty here.

Commitment and price-change exposure

A reservation ties you to a quantity and a period, while the provider's price can move. Our example is AWS Capacity Blocks, whose page says prices follow supply and demand and are scheduled for their next update in January 2027. That is accurate as of 12 November 2026 and will age, so reopen the page before you rely on it. Because billing follows the purchase date rather than the start date, a purchase before an update locks the old rate. The next purchase pays the new one.

Who on your team can run and patch it

Machines you run yourself need someone to update drivers, maintain the runtime, track vulnerabilities and cover out-of-hours incidents. If only one person knows how, the risk sits in that person more than in the hardware. If nobody can take the job, a managed API or cloud is the safer choice.

Cost that flows through a bill with nobody holding a budget is covered in our article on cost drift, and spreading across clouds to reduce dependence on one provider is in multi-cloud versus vendor lock-in. We do not repeat either here.

When a vendor buys compute on your behalf

Many projects have a contractor or system integrator open the account and reserve GPUs for the agency's system. The first question for the TOR and contract is who owns the reservation. If it sits in the contractor's account, you may have no right to the capacity you paid for once the contract ends.

Next, who bears reserved capacity that goes unused. The contract should say whose cost the idle hours are, and whether the next reservation is resized from measured use. Price pass-through must be explicit too: if the provider changes prices mid-year, how does the difference reach you, is there a cap, how many days of notice, and may you decline a new reservation.

Finally, reporting. Require monthly reserved hours and used hours, in a form you can check against the provider's console yourself. Without it you learn about idle capacity when the invoice arrives.

The limit we should state plainly

Most organisations should start on a hosted model API. The placement decision only matters once there is a steady load large enough that unused reservation becomes a visible line in a cost report that someone asks about. If you are in a pilot, have a few dozen users, or do not yet know what the real load looks like, reserving or buying machines solves a problem you do not have yet and adds work for your team.

The survey carries its own limit. It reflects respondents who chose to take it. That 31 percent of 114 reserving respondents reported over a quarter of spend on unused capacity shows the problem occurs at some organisations, not that it occurs at yours. Your own data has to answer that.

Objections we often hear

A common one: if we do not reserve, there may be no machine on the day we need it. The concern is fair. In the same survey 12 percent of respondents named a shortage of computing capacity, and reservation exists to buy that certainty. For work that cannot slip, certainty has value. We ask only that you price it as a number, for example how much more per month you will pay to have machines waiting, and compare that with the idle hours you measure, so that the size of the reservation comes from a calculation.

Where to start

Name the reservation's owner as a person or a role, not a shared team name. That person should see the bill and the usage in one place. Then measure two numbers every week, reserved hours and hours with work actually running, and divide them. Keep the series for at least a quarter before deciding to renew or enlarge.

If you have no reservation yet, run the real load on an API or on-demand for a while to learn how steady it is and how many hours a day it needs, then compare against the break-even from the formula above using prices from real quotations. For costs teams tend to set from untested assumptions, one example is in our article on assumed costs in IT budgets.


Summary

If you keep one thing from this article, ask of every reservation what share of it was used in the last 30 days and who gave that answer, because the hourly rate shows only half of what you actually pay.

If you would like to talk through the details of a project, you can reach us at 088-983-9386 or [email protected]. Our office is in Bang Kapi, Bangkok. For organizations still drafting a scope of work, we are glad to arrange a conversation to understand the problem first, without needing complete documents. And if we find that the problem is not yet at the point of needing a new system, we will say so plainly, because starting a project at the wrong time costs more than waiting another quarter.

FAQ: Frequently Asked Questions about This Article

A collection of questions and answers to help you better understand the content of this article.

Tags:

reserved GPU capacityGPU inference costAWS Capacity Blocks for MLon-demand vs reserved GPUhosted model API vs self-hostingeffective cost per used hourAI compute contract terms
Share:

Other Articles

Stay tuned for upcoming articles!