Running AI on Your Own Server for Sensitive Data

Run an AI model on your own hardware when the data genuinely cannot leave: a law, a client contract, or an internal rule forbids it. On cost alone, a paid AI service usually wins at small-business volumes. Classify the data first, then compare the monthly bill, then check the model's license.

This article walks through those three checks in order, with numbers that were current in October 2026. It assumes you already know what you want the AI to do, such as summarising contracts, answering questions over internal documents, or sorting customer messages.

Which data actually cannot leave your building?

Most "we can't use cloud AI" conversations mix three different reasons. Separate them, because each one has a different answer.

The law. Indonesia's personal data law (UU PDP, Law 27/2022) allows transfers of personal data abroad, but under conditions. Article 56 asks the sender to make sure the destination country protects data at least as well as Indonesia does; failing that, to have binding safeguards in place; and failing that, to get the person's consent. Its implementing regulation, PP 33/2026, was promulgated on 16 July 2026 and takes effect on 16 January 2027 (Katadata, 3 September 2026). According to a law-firm summary published 9 October 2026, it adds concrete steps before a transfer: map where the data goes, send only what is needed, assess the risk to the people concerned, and tell them the purpose and the safeguards. Sending customer records to an AI service hosted overseas is a transfer. That makes it a paperwork question, not a ban.

The contract. An NDA or a client agreement can be stricter than the law. A manufacturer may forbid its drawings from leaving your network, whatever UU PDP says. Read the clause; it often decides the matter faster than any legal analysis.

Internal policy. Some companies simply do not want salary data, board minutes, or source code on anyone else's servers. That is a legitimate choice, but it is a preference, and it should be priced like one.

There is also a middle path before buying hardware:

The second row costs a little more. Anthropic, for example, charges a 1.1x multiplier for keeping the processing inside the United States, and a 10% premium for regional access points on cloud platforms. Neither option places the data in Indonesia, so check which regions your provider actually offers before you count on it.

The third row is underused. A short script that replaces names, phone numbers, and ID numbers with placeholders before the text leaves, and swaps them back when the answer returns, removes most personal data from the transfer. It does not work when the answer depends on the identity itself.

What does the monthly bill look like?

AI services charge per token, a fragment of a word. Prices are quoted per million tokens, separately for the text you send (input) and the text you get back (output).

As of October 2026, Anthropic's price list puts Claude Haiku 5.5 at $0.10 per million input tokens and $0.50 per million output tokens, and Claude Sonnet 5.5 at $2 and $10. Take a realistic workload: 2,000 documents a day, each about 3,000 tokens in and 500 tokens out.

Now the other side. Renting one H100, a data-centre graphics card built for AI, costs $2.89 an hour on Runpod's public price list (October 2026). Running it around the clock comes to roughly $2,100 a month, before anyone sets it up or keeps it healthy. A rented card abroad does not solve the data-location problem either; it only gives a price anchor.

Buying your own hardware changes the shape of the cost, not the conclusion at this volume. You pay up front, spread that over about three years, and add electricity, cooling, spare parts, and the hours of whoever maintains it. Get real quotes from local suppliers; graphics card prices in Indonesia move with the exchange rate and import stock.

The honest summary: for most small and mid-sized businesses, running your own model is a privacy decision you pay extra for, not a way to save money. Volume changes that only when the server stays busy most of the day. For routing work across cheap and expensive models, see controlling LLM costs.

Which models can run on one machine?

An open-weight model is one whose model file you can download and run yourself, rather than renting access through someone else's service. Capable ones now fit on hardware a business can buy.

OpenAI's gpt-oss models, released in August 2025, are a useful reference point. The larger one, gpt-oss-120b, is designed to fit on a single GPU with 80 GB of memory, such as an H100. The smaller gpt-oss-20b runs within 16 GB of memory, which puts it within reach of a high-end desktop graphics card.

Smaller models are cheaper to run and noticeably weaker. They lose detail in long documents, follow complex instructions less reliably, and are worse in Indonesian than the largest hosted models. Do not trust a general benchmark for this. Take 50 to 100 real documents from your own work, run them through the candidate model, and compare the answers with what your staff would write. That small test set, described in evals for LLM products, tells you more than any public ranking.

Is the model's license clean for business use?

Downloading a model does not make it yours to use however you like. Each model ships with its own license, and it can differ between versions of the same family.

The 700-million-user threshold will not bind an Indonesian SME, but the acceptable use policy still applies. Keep a copy of the license file for the exact version you deploy, next to the model itself. The wider point, that the code, the model file, and the training data each carry their own terms, is covered in checking AI licenses layer by layer.

What does running it yourself actually take?

A model on your own server is a production system. It needs the same care as your database, and some extra.

• Access control. Who can send it questions, and can they see each other's documents? An internal AI that answers anyone from any folder leaks data inside the company instead of outside it.
• Logs. Record who asked what and when. You will need this the first time someone asks where an answer came from.
• Retention. Prompts and answers contain the same sensitive data you were protecting. Decide how long you keep them and delete on schedule.
• Updates. Model files, the software that runs them, and the GPU drivers all get updates. Someone has to test and apply them.
• A second person. If one engineer is the only one who can restart the server, the system stops when that person is on leave.

Keeping data in-house does not by itself satisfy UU PDP. Purpose, access, retention, and incident handling still apply, and the administrative exposure described in UU PDP fines in 2026 does not shrink because the server sits in your building.

A short decision checklist

1. Write down which data the AI will see and which rule (law, contract, policy) restricts it.
2. If location matters but sending is allowed, price a fixed-region service or name removal first.
3. If nothing may leave, estimate monthly tokens and compare the API bill with the full cost of your own server over three years.
4. Test two or three candidate models on 50 to 100 of your own documents.
5. Save the license file for the exact model version you choose.
6. Name the two people who maintain the server, and set access, logging, and retention before the first real document goes in.

If you are weighing this choice for a specific system, I can help map the data, the options, and the real monthly cost through system architecture consulting.