During a recent WEB WEBINARSean Tracey, Head of Developer Relations at Expanso, presented a compelling vision for running large language models (LLMs) securely, efficiently, and locally. The conversation centered on a pressing problem: how organizations can leverage the power of LLMs without exposing their sensitive data to proprietary, cloud-hosted models. Tracey introduced a novel framework that combines open-source LLMs with a unique data processing architecture to solve this challenge—securely running compute where the data lives.

What Are LLMs, and Why Is Their Architecture Both Powerful and Problematic?

LLMs, such as GPT-based models, are built on the transformer architecture introduced in the influential “Attention is All You Need” paper. While the transformer design dates back to 2017, it exploded into public consciousness in 2022 with ChatGPT.

However, ChatGPT and similar commercial models are closed-source, meaning developers can’t inspect, modify, or self-host them. Organizations dealing with proprietary or regulated data—such as healthcare providers, financial institutions, and governments—face an obstacle: they need powerful AI tools but cannot risk data privacy by sending sensitive information to opaque external services. This tension has pushed many to explore open-source alternatives.

The Rise and Power of Open-Source LLMs

The open-source community has responded by replicating and, in many cases, surpassing the capabilities of commercial LLMs. Early on, these models lagged in performance. Today, tools like DeepSeek R1 not only rival closed-source models but also offer distinct advantages: transparency, adaptability, and control.

Open-source LLMs allow researchers and enterprises to determine how the models are trained, which datasets are used, and where the models are hosted—whether on local CPUs or custom GPU clusters. DeepSeek R1, for instance, is a reasoning-focused model that performs competitively while running on modest hardware, making it ideal for local deployments.

The Core Problem: Moving the Data

Despite LLMs’ growing accessibility, a fundamental issue remains: traditional data infrastructure requires that data be moved to compute environments. Data is often generated at the edge—IoT sensors, mobile devices, and user applications—and transferred to centralized storage for processing.

This model is inefficient and costly. According to forecasts, global data generation is expected to reach 181 zettabytes by 2025. Transferring even a fraction of that volume leads to significant bandwidth costs and egress fees. More importantly, moving sensitive data increases the risk of breaches and non-compliance with data protection regulations.

Level Up Your AI Expertise! Subscribe Now:

Expanso’s Battle Yao: Compute Over Data

Expanso addresses this challenge with Battle Yao, a distributed workload orchestrator built to run jobs directly where the data resides. Instead of transporting gigabytes or terabytes of data across networks, Battle Yao uses a compute-over-data (COD) model—sending lightweight jobs (code and dependencies) to the data.

This design is not only bandwidth-efficient but also security-conscious. Jobs can be tightly scoped to control what they access and whether they can export any data. Organizations maintain complete control over how their data is used and ensure that nothing leaves the system without explicit authorization.

Secure In-Place LLM Execution with Battle Yao

LLMs are naturally suited to Battle Yao’s architecture. A model and its associated logic can be containerized and deployed as a job to systems holding the data. The LLM processes the data locally, returning only the result, not the raw data.

For example, a user might bundle a query and an LLM into a container, send it to a secure server hosting private documents, and receive a summarized report, without ever exposing the source material to external systems.

Demonstrating the Model: Olama, DeepSeek, and Local Data

During the webinar, Tracey demonstrated this setup using Olama, an open-source tool for running models locally, and DeepSeek R1. Olama abstracts model complexity and provides a clean API for interaction.

In the demo, Wikipedia articles were stored on local machines equipped with the Battle Yao client. The Expanso Cloud orchestrator manages job distribution. A simple CLI command—bacal job run—deployed the containerized LLM app to the data nodes. Users then interacted with the LLM through a web UI that securely connected to the remote Battle Yao instance using a reverse proxy over websockets.

Notably, data never left the secure system. The LLM could even be hot-swapped mid-session via the Olama API, switching seamlessly between models like DeepSeek, LLaMA, or Small Thinker. Models ran efficiently on local CPUs or GPU hardware.

Solving Broader ML Deployment Challenges

Machine learning deployments often struggle with cross-platform dependencies, infrastructure constraints, security reviews, and inconsistent update paths. Battle Yao addresses these issues with a unified orchestration model. Its built-in security features—end-to-end encryption, access control, and audit logs—help meet compliance requirements.

Model updates are simplified, too. Developers only need to update the job configuration, saving both time and effort, two of the most constrained resources in enterprise AI workflows.

Conclusion: The Future Is Local, Open, and Secure

As LLMs continue to revolutionize data interaction, their adoption hinges on one critical factor: trust. Expanso’s Battle Yao, combined with open-source models like DeepSeek R1, provides a compelling pathway forward. This architecture eliminates the need to move data, mitigates privacy concerns, and accelerates secure AI adoption.

By flipping the traditional model—sending compute to data—Expanso unlocks new levels of efficiency and control. While Battle Yao excels with LLMs, its flexibility makes it suitable for a wide range of data processing workloads.

With Expanso Cloud simplifying orchestration, organizations now have a powerful toolset to reclaim control over their data and deploy AI where it belongs—securely, locally, and openly.