Private inference can reduce external data exposure. The surrounding system still has to be secured, monitored, maintained, and recoverable.

On September 19, 2026, President Donald Trump said he plans to create an “AI Force” and appoint a new AI adviser. Reuters reported that the announcement did not include implementation details and that the White House did not immediately comment. The initiative’s mandate, authority, staffing, and operating model therefore remained unspecified in the available announcement. The same distinction applies inside a company: naming an AI body does not define its controls.

“We run custom LLMs locally.”

“Good. Local to what? An employee workstation, an office server, a private cloud, an offline environment, or an air-gapped network? Who authenticates users? What data can the system reach? Who can deploy a model or dependency update? Where do prompts, outputs, logs, and backups go? How would you isolate it, restore it, or shut it down after a compromise?”

“We keep the code in GitHub and use [platform].”

“That names two components. It does not explain the controls around them.”

This conversation exposes a security gap that becomes easy to miss once a company moves inference onto equipment it controls. Local models can reduce the amount of business data sent to an outside model provider. They can also improve speed, support specialized configurations, and give the operator more control over when and how a model changes. Those are legitimate benefits.

I run local models for many of the same reasons. Working with them also makes one fact difficult to ignore: the model is only one component in a much larger system. Its location says nothing about who can reach it, what it can execute, where its data travels, or whether the company can recover it. The model may be local while the risk remains distributed.

“Local” needs an address

A model running on a laptop inherits the condition of that laptop, including the user’s permissions, files, browser extensions, malware exposure, and backup habits. A shared office server introduces authentication, network segmentation, encryption, patching, capacity, monitoring, and administrative access. A model inside a private cloud still depends on cloud identity, service accounts, images, secrets, telemetry, and network configuration.

Offline and air-gapped systems have different boundaries. An offline machine may reconnect or receive updates through controlled transfers. An air gap is a deliberately enforced isolation boundary, yet removable media, insiders, physical access, supply-chain compromise, and recovery procedures remain relevant. “Local” should begin the architecture discussion because it does not finish one.

The word “custom” deserves the same scrutiny. A company may train a model, fine-tune a third-party model, attach an adapter, or convert downloaded weights. It may also add retrieval or change prompts and an interface. Each path carries a different chain of source models, code, formats, data, licenses, dependencies, and update decisions. The label provides almost no security evidence.

The perimeter begins before the model loads

A downloaded model belongs inside the software-supply-chain review. It is easy to treat model files like passive data, although some formats can execute code while loading. Hugging Face’s own pickle security guidance explains that unpickling can execute arbitrary code and that its scanning is not foolproof. A familiar repository page cannot replace verification.

The practical response is straightforward. Record the approved source and exact version, preserve hashes or signatures, and review the license and model card. Prefer safer serialization formats, scan the artifact, and test it inside a restricted environment. Promotion into production should require an identifiable decision and a retained record. The same discipline applies to tokenizers, adapters, containers, packages, base images, retrieval components, and conversion tools.

This is where ordinary development controls become part of AI security. Source repositories, dependencies, build actions, container registries, deployment pipelines, secrets, endpoints, and service accounts decide what reaches the model and what the model can reach. A private repository restricts source visibility. Workflow permissions, credential movement, code review, and production changes still need their own controls.

GitHub documents these risks directly. Its secure-use guidance for Actions recommends minimum token permissions, immutable commit references, and review of workflow changes. It also calls for careful handling of secrets and untrusted code. GitHub warns that a compromised action may reach repository secrets or use a workflow token to write back to the repository. The platform provides useful controls, although the company must configure and govern them.

For many small and mid-sized organizations, leaving GitHub would solve the wrong problem. A sound default uses an organization-owned private account, phishing-resistant multifactor authentication, limited administrators, and protected branches or rulesets. It also requires reviews, narrowly scoped workflow tokens, and controlled deployment environments. Long-lived deployment credentials should give way to short-lived federated identity where the target platform supports it. Production secrets belong in a governed secrets system with rotation, revocation, and audited access.

Self-hosted GitLab, Forgejo, or Gitea may fit a sovereignty, isolation, residency, or customization requirement. That choice transfers patching, monitoring, availability, capacity, hardening, backup, and recovery to the company operating the service. Another managed forge changes the provider and feature set. It leaves the need for identity, change control, pipeline security, independent recovery, and tested restoration exactly where it was.

Collaboration, continuity, and recovery solve different problems

The primary forge supports collaboration and controlled change. A secondary Git host can provide continuity if the primary service becomes unavailable. An encrypted, retained, independently administered backup provides recovery after deletion, corruption, ransomware, or credential compromise. Combining these functions under one identity and one failure boundary creates the appearance of redundancy without dependable recovery.

Git is distributed, so additional clones improve source durability. Developer laptops and live mirrors can still share compromised credentials, accept propagated deletions, omit large-file objects, and lose required platform configuration. GitHub’s repository backup documentation separates mirror cloning from Git LFS retrieval. It also explains that migration archives omit some assets and lack a supported restoration path into GitHub.

Critical AI systems require a broader recovery set. It may include repositories, all references, submodules, large-file objects, releases, packages, container images, build outputs, software bills of materials, and attestations. It may also include repository settings, rulesets, webhooks, runner configuration, model weights, adapters, tokenizers, prompts, evaluations, licenses, and provenance records. Infrastructure definitions, database backups, retrieval sources, vector-store rebuild instructions, and key-reconstitution procedures complete the operating picture.

The backup should sit behind a separate administrative identity and preferably a different provider or isolated account. Version retention or immutability limits the damage that stolen primary credentials can cause. At least one appropriately isolated copy should exist for critical assets. None of this matters for long if the company has never restored the system from those materials.

The risk changes when the model can act

An inference-only model produces an answer for a person to review. A connected model may read repositories, query databases, open files, send messages, call business APIs, run shell commands, or change records. Its effective security boundary then includes every tool, permission, destination, and approval path attached to it.

The OWASP Top 10 for LLM Applications 2026 reflects that change. Excessive Agency moved to third, while System Prompt Leakage expanded into Hidden Context Exposure. The broader category covers system instructions, retrieved information, tool definitions, permission metadata, workflow logic, and other non-user-visible context. Sensitive policy and credentials should remain outside model-visible context whenever the architecture permits it.

A system prompt cannot enforce authorization by itself. Critical policy belongs in deterministic, auditable application components that check identity, scope, arguments, and current state before an action executes. High-impact or irreversible actions should require clear human approval, and the reviewer should see the actual proposed action. A summary generated by the same model is weak evidence.

OWASP’s emerging Agent Control Standard points toward external interception of agent actions with allow, deny, modify, ask, or defer decisions. The direction is useful, especially for tool-connected local systems. Its current v0.1 reference implementation also documents unauthenticated communication, default fail-open behavior, and limited live-hook coverage. It is an early specification to evaluate, rather than a production control to trust without additional engineering.

Logging deserves equal care. Detection and incident response require enough information to reconstruct access, changes, tool calls, policy decisions, and failures. Prompts, outputs, retrieved documents, tokens, and error traces can turn the log store into another collection of sensitive data. Logging therefore needs minimization, redaction, access control, tamper resistance, retention limits, alert ownership, and a tested investigative process.

The system cannot own the decision

AI is not a person. It does not possess human moral agency, accept civic responsibility, or answer for the consequences of a business decision. People and organizations choose the model, data, tools, permissions, deployment boundary, and actions taken from its output. Responsibility remains with those who design, deploy, authorize, and use the system. The NCSC and CISA guidance similarly directs providers to accept responsibility for downstream security outcomes.

That does not mean a person deliberately wrote every output or intended every result. Modern models can produce probabilistic and unexpected behavior. A serious failure analysis still has to reach the controllable causes, including design, training, data, access, permissions, incentives, oversight, and use. Blaming “AI” ends that investigation too early.

Access to vast amounts of data does not make truth automatic. A governed process makes discernment more reliable by testing provenance, source independence, dates, incentives, contradictions, and primary evidence. It also records what remains unresolved. Easy answers, familiar conclusions, and unexamined trust in an interpreter remain human choices, even when the tool has changed.

Security has to be demonstrated

Current guidance treats AI security as a lifecycle responsibility. The NCSC-hosted Guidelines for Secure AI System Development is also U.S.-backed guidance, published with CISA and supported by NSA and FBI. It covers secure design, development, deployment, operation, monitoring, updates, and incident management.

The closest U.S.-led companion is NSA’s Deploying AI Systems Securely, developed with CISA, FBI, and international partners. It focuses on organizations deploying externally developed AI in managed environments. NIST SP 800-218A covers AI-specific secure software-development practices, while NIST AI 600-1 addresses supplier assessment, component integration, monitoring, incident response, recovery, and deactivation. NIST AI RMF 1.0 remains current while its revision is in progress.

A credible company should be able to produce a current system and data-flow diagram, asset inventory, and identity matrix. It should also maintain provenance records, a secrets lifecycle, deployment controls, a logging policy, and tested recovery evidence. The company should know who owns each control, when it was last verified, which exception was accepted, and what event requires revalidation. A product name or architecture label cannot supply that evidence.

I would place those records in a Local AI Security Evidence Register with an owner, evidence location, verification date, exception, remediation action, and revalidation trigger. That turns a security claim into a governed record.

Local inference remains a valuable option, and I expect its use to grow as models improve and businesses demand more control. That growth raises the standard for the people operating it. Security begins when the answers are current, owned, tested, and available before the incident.

Primary references