Your AI inference server,run in Canada
A large enterprise has dozens of tools calling a language model, across as many teams, and the bill grows with every team that joins. At that scale, a reservation is counted in servers: half a server, a full server, or several. Agent Libre hosts and operates them in Canada, for a fixed amount per month, and puts its inference expertise to work for your teams.
Discuss your capacityThree conditions before a large enterprise reserves a server
The CIO, the head of security and the platform architect each set one condition.
Capacity counted in servers
Dozens of tools spread across teams, coding agents that process all of a project’s code, a customer service operating around the clock. At that volume, a large enterprise’s demand exceeds the share available on a shared server. The 24-hour to one-week audit establishes each tool’s actual consumption and serves as the basis for sizing.
A security review that passes
The data processing location, encryption in transit and at rest, access traceability and the infrastructure providers’ attestations go through a formal review before anything goes into service. Each requirement calls for a documented answer, which your security team verifies against its own criteria.
Inference expertise that stays current
Database, security, development and solution architecture expertise is already covered, in-house or through vendors. Inference is a distinct specialty, and its methods evolve from one month to the next. A large enterprise requires a partner that practices it full time, measures its results in production and passes those methods on to its teams.
Four guarantees your security team can verify
What every call to the model goes through, from the network to the server, and what we document for your security review.
Canadian hosting
The infrastructure is located in Canada and operated by personnel based in Canada. Traffic between your tools and the model does not cross the border, and no foreign subcontractor takes part in processing. The processing location is written into the agreement and can be demonstrated to an auditor or a customer from the logs.
Isolation and access control
A full server is reserved for your organization alone; on half a server, your capacity and your logs remain separate from those of other members. Each team has its own access keys, revocable without delay, and every call is tied to the team that issued it. Encryption in transit and at rest is in place by default and appears in the documentation handed to your team.
Centralized governance
A mandatory control point between your tools and the model enforces the rules set by your security team: masking of designated information, blocking of sensitive data leaks, logging of every call with its origin. No request is used to train a model. The full arrangement is described on the Governance and compliance page.
Certified providers
Our infrastructure providers are attested SOC 2 Type II, with ISO 27001 / 27017 and GDPR or HIPAA compliance depending on the provider. Their attestation reports are handed to your security team before signing. Audit logs export to your own tools.
A reservation is counted in servers
Half a server
18 of a server’s 37 PTU, reserved for you, the rest for one or two other members, each with guaranteed capacity and separate logs. The usual starting point for a large enterprise that begins with one division.
A full server
All 37 PTU of a server, for you alone. Your masking rules, your logs, your schedule for model updates. Nobody else makes calls on it.
Several servers
For an enterprise that serves several divisions or several countries, or wants a reserve in case of failure. Each server is added from the measurement, on the same term, and the GPU cost reduction negotiated on volume comes back to you.
The inference expert, the role AI adds to your organization
You already fill several specialized roles. This one is new, and it evolves rapidly.
The roles you know
Database expert, security expert, developer, solution architect. Highly specialized roles, in-house or at a vendor, with methods established for years and people you can easily find on the market.
The inference expert
Getting a model to answer fast, at a low cost and with no loss of quality, on a given server, with dozens of tools calling it at the same time. The methods change every month and few people practice them. At Agent Libre, inference is our fuel: it is our everyday work.
What the partnership gives
Becoming an Agent Libre partner is more than sharing a server. A large enterprise gets access to the latest inference methods and strategies, tested in production on our servers. Your experts work with ours, and what we learn on one server benefits every partner.
What our partners benefit from
A glimpse of the methods, tools and results we follow and test on our servers. The full list, with our measurements, goes to partners.
OKF Agent Memory: a project memory for agents, stored in the Git repository
An agent loses what it learned when its conversation closes. This open project (MIT license) stores decisions and findings in Markdown files versioned with the code, of which the agent loads only the useful part. The author estimates the context reduction at 80%.
LLMLingua: compressing the request before it reaches the model
A small model removes from a long request the words that add nothing to the answer, before sending to the large model. Microsoft reports a request up to twenty times shorter with minimal quality loss. We measure that loss on our tools before recommending a setting.
RTK: trimming what coding agents send back to the model
A coding agent reads the full result of every command. RTK filters out noise and truncates repetitions before that result enters the conversation. The authors report a 60 to 90% reduction on those outputs. We quantify the actual effect on the number of PTU required.
codebase-memory-mcp: a map of the code, so the agent stops rereading files
The project indexes a repository into a map of functions, classes and their calls, which the agent consults instead of rereading files. The authors measure 120 times less text sent to the model than a file-by-file search. We verify the gain on our tools before adopting it.
Reusing the shared beginning of requests
A hundred requests that start with the same twenty pages of instructions do not force the server to reread those pages a hundred times. The server keeps the work already done on that shared beginning and processes only what follows. How much capacity this frees depends on how your tools write their instructions, and that is what we tune with your team.
SGLang and vLLM: two engines serving the same model, compared on the same GPUs
Same model, same hardware; only the software that organizes requests changes. The SGLang team measures up to three times vLLM’s throughput on a 70-billion-parameter model, on A100 and H100. The study comes from one of the two camps and dates from July 2024. We redo the comparison on our hardware with each release.
What each role decides before reserving a server
Four decisions that belong to your organization. The model does not make them for you.
Pick the first team to connect and ask, during the 24-hour to one-week audit, for the consumption of an ordinary day and a peak day for each tool. The sum says whether you need half a server, one server or several.
Set the masking rules by data category, designate who keeps each team’s access keys and choose where logs are exported. Ask for the infrastructure providers’ attestation reports before signing.
Check that each tool lets you choose the address and the name of the model it calls, and establish an orchestration. Name the person who will work with Agent Libre’s inference expert on the server measurements.
Settle the term, from 12 to 24 months, and the rule for revising capacity. Write into the agreement the data processing location, the log retention period and the pass-through of GPU cost reductions.
What we get asked on this topic
How is the number of servers required determined?
By measurement. During the 24-hour to one-week audit, a gateway is placed in front of your current provider, division by division if you wish, and records each tool’s consumption over an ordinary day and a peak day. The total is converted into PTU (provisioned throughput units), then into half a server, a full server or several servers, and the calculation is delivered with the report.
Is a full server reserved exclusively for our organization?
Yes. The server’s entire capacity is allocated to you, your masking rules and your logs are the only ones in force on it, and no other organization has access to it. On half a server, capacity is shared with one or two other members, each with guaranteed capacity and separate logs.
Which documents are provided to our security team before signing?
A complete review of our security program, together with the attestation reports of our infrastructure providers. These documents are delivered under a confidentiality agreement, before signing, for review against your criteria.
What are the responsibilities of Agent Libre’s inference expert?
Tuning the server for your tools, weekly measurement of actual throughput, migration to new models when measured quality justifies it, GLM 5.2 to date, and passing validated methods on to your teams. Throughput freed up over the term is returned to you, on the same hardware.
What availability commitments are made, and how is an outage handled?
The PTU you reserve are guaranteed to you at all times on the assigned server. An organization that requires uninterrupted continuity reserves standby capacity on a second server, sized with it. Any incident results in an immediate notice to your teams, followed by a written report on the cause and the corrective measures. The outage provision included in the plan is credited to the group when it has not been used.
How does capacity evolve if our consumption doubles during the term?
Capacity is increased from what remains available on your server or the next one. Failing that, your organization is given priority for the commissioning of an additional server, under the conditions of the current term. In the meantime, traffic above your PTU goes through on a best-effort basis, without a throughput guarantee, but with no service interruption and no additional charge.
See also
Governance and compliance
The monthly report, the inventory of AI tools and the rule review your compliance team produces from the gateway, with no request to IT.
Pricing
The monthly plan, made of the server cost with no markup and the Agent Libre fee, both calculated on the number of PTU you reserve.
Buy GPUs
Own, pro rata, an eight-GPU server hosted in Canada instead of leasing it, when the commitment exceeds two years.
Your company has two supply chains
The first one you already know. Materials come in, they are transported, stored, transformed, delivered. Every step has a supplier, a lead time, a unit cost. When the margin tightens, you walk down that chain step by step, and you generally know where to look.
The second one moves information instead of materials. A price request comes in and becomes a quote. An order comes in and becomes a production order, an invoice, a confirmed delivery. A site report becomes a change order, or it does not. This chain runs through every one of your departments and it employs a large share of your people.
The 24-hour to one-week audit
We measure your actual consumption at your current provider: who calls what, what it costs, and what is leaving your walls. Free, no commitment, and the report is yours even if you go no further.