A Local Model Does Not Make Agent Context Local

GitHub's 11 August Copilot for JetBrains update put three attractive capabilities in one release: Copilot memory can retain useful information across agent chat sessions, Ollama can act as a bring-your-own-key model provider, and administrators get server-managed controls covering plugin availability, MCP access, permission-bypass behaviour, and OpenTelemetry settings. The release also makes Codex sessions visible in agent debug logs.

Those features solve different problems. Memory reduces repeated explanation. Ollama can move model inference into an environment the operator controls. Managed settings constrain clients and tools. Debug logs make runs easier to inspect. Combining them in one interface does not combine their data boundaries.

Cloudflare's recent agent releases expose the same separation from another direction. AI Search can crawl owned websites or ingest files, build hybrid keyword and semantic search, query several indexes together, and expose search through an application, an MCP server, or a public endpoint. The new MCP specification removes protocol sessions from the core request path while making request identity, capabilities, method, and authorization context more explicit. Cloudflare is also unifying Workers AI and AI Gateway so one control plane can route inference, meter tokens, attribute cost, and log requests and responses.

The repeated angle to avoid

The ten most recent posts here covered form-security boundaries, server-side agent conversion tracking, agent ROI receipts, prototype exits, Copilot billing handovers, code-quality policy, comment-triggered agents, gateway budgets, model entitlements, and worktree isolation. Older overlapping posts covered inference-region routing, review-agent context, stateless agent intake, long-running checkpoints, runtime credentials, and agent harness sessions.

The weak version would repeat the old X needs Y formula: agent memory needs governance. The sharper thesis is that an agent does not have one context store. It creates a chain of context copies, and each copy can have a different owner, access rule, location, retention period, deletion path, and audit surface. A builder cannot infer the chain's privacy from whether the final model runs through Ollama or a hosted API.

Fresh evidence and background context

The source map separates this week's product evidence from earlier implementation context:

Source	Freshness	What it contributes
Copilot memory and Ollama for JetBrains	11 August 2026	Persistent cross-session memory, a local-model provider option, server-managed controls, and agent debug visibility in one client
Cloudflare AI Search	6 August 2026	Crawling, ingestion, hybrid retrieval, multi-index queries, MCP exposure, public endpoints, access control, and an identifiable crawler
Cloudflare's next-generation MCP implementation	6 August 2026	Stateless core requests, explicit client context, gateway-visible method metadata, tighter authorization, and a migration window
Workers AI and AI Gateway unification	7 August 2026	Default gateway creation, payload logging, token and cost attribution, unified billing, and planned model-first routing
AI inference regions as routing logic	Background, 28 July 2026	The earlier distinction between data class, inference region, client, tool scope, approval, and telemetry

The fresh sources describe their own products, not a universal architecture. Their overlap still reveals a general mechanism: useful context is copied before inference, during execution, and after the answer. Those copies remain even when the selected model is local.

An agent has a context path, not a context window

A context window is the set of tokens presented to a model for one inference. The context path is everything that made those tokens available and everything that records what happened next.

A practical path can include:

source record
  -> crawler or connector snapshot
  -> parsed chunks and embeddings
  -> retrieval result
  -> persistent agent memory
  -> active prompt and model context
  -> MCP or application tool request
  -> gateway, cache, trace, and debug log
  -> generated output and business record

Not every system uses every step. The point is that each arrow can create a new representation with a new boundary.

Context copy	Why it exists	Boundary operators must define
Source record	Holds the authoritative document, issue, customer record, or code	Source permissions, classification, and deletion owner
Crawl or connector snapshot	Makes content ingestible	Crawl scope, robots policy, refresh cadence, and snapshot retention
Chunks, embeddings, and index metadata	Makes retrieval fast	Index tenancy, encryption, access inheritance, and deletion propagation
Retrieval result	Selects material for one task	Query authorization, provenance, relevance threshold, and result logging
Persistent memory	Carries inferred facts or preferences across sessions	Memory scope, write rule, expiry, correction, and user-visible deletion
Active model context	Gives the model instructions and task data	Inference location, provider retention, model policy, and fallback route
Tool request	Lets the agent read or change another system	Client identity, audience, capability, approval, rate, and idempotency
Cache, trace, or debug log	Improves performance and diagnosis	Payload capture, redaction, sampling, access, and retention
Output or business record	Makes the result operational	Human review, downstream permissions, provenance, and record lifecycle

Calling the model locally only answers part of the active-model-context row. It says nothing automatically about the other rows.

Local inference is not local workflow execution

Ollama support is valuable. A team may choose it for offline development, model experimentation, lower marginal inference cost, data-location preferences, or access to a model that fits a particular machine. But provider = ollama is not a complete privacy statement.

A JetBrains coding workflow may still read a repository hosted on GitHub, recall information from persistent Copilot memory, call a remote MCP server, send telemetry, install or invoke a CLI, write debug logs, and open a pull request. Whether any particular feature transmits content depends on its configuration and implementation; the operator has to verify that path rather than assume the model choice settles it.

The reverse is also true. A hosted model is not automatically an uncontrolled route. A remote workflow can have narrow data classes, regional inference, payload logging disabled or redacted, strict tool scopes, short retention, and auditable deletion. "Local" and "hosted" describe deployment choices. They do not replace a data-flow review.

Use this decision rule:

Do not label an agent workflow local, private, or offline until every required context copy and network dependency can be named, tested, and deleted according to the same claim.

If the accurate description is "local inference with cloud repository access and remote MCP tools," say that. The narrower claim is more useful than a comforting but false binary.

Retrieval and memory solve different continuity problems

Cloudflare AI Search and Copilot memory can both reduce repeated context gathering, but they should not be merged into one conceptual bucket.

Retrieval should return evidence from a controlled corpus at query time. A good retrieval result has a source identifier, content version, retrieval time, access decision, and a way to refresh or delete the indexed copy.

Persistent memory carries a compact fact, preference, or learned project detail across conversations. It may be derived from earlier interaction rather than copied verbatim from one current source. That makes correction, expiry, and provenance harder.

For example, a retrieval system can return the current rule that quote requests outside a service area must be referred to a partner. Persistent memory might remember an older conversation saying those requests should be rejected. If the agent silently prefers memory, the workflow can behave consistently and still be wrong.

Use a precedence rule:

1. Current authoritative policy beats persistent memory.
2. Current business-system state beats a retrieved snapshot.
3. Retrieved evidence beats an unsupported remembered summary.
4. Memory can personalise or accelerate work only inside those constraints.
5. Conflicts should be surfaced, not blended into a confident answer.

Memory records should therefore carry at least learned_at, source_or_reason, scope, expires_at, and last_confirmed_at. A remembered coding convention may live for months. A customer consent state, price, stock level, booking slot, security policy, or employee access decision should usually be fetched from its source instead of remembered.

Search access must survive aggregation

AI Search can combine several owned surfaces and expose one query layer. That is useful for documentation, support knowledge, product information, developer references, and agent tools. Aggregation also creates a new authorization question: a user who can query the combined index may discover content from a source they could not browse directly.

Cloudflare describes two exposure patterns. A namespace can provide public /search and /mcp endpoints with no application deployment, or an operator can put Cloudflare Access in front of the domain so queries require login. The crawler follows robots.txt and identifies itself, which helps source operators understand ingestion traffic. Those controls address different boundaries:

• crawler identity governs whether and how content is ingested;
• source ownership governs which sites can become inputs;
• index tenancy governs which data is stored together;
• query authentication governs who can search it;
• result filtering governs which chunks a caller may receive;
• MCP tool scope governs how an agent can use the result.

A public product manual and a private support runbook should not enter one publicly queryable namespace merely because both help answer product questions. Either split the indexes, enforce caller-aware filtering, or place the aggregate endpoint behind an identity boundary that preserves the strongest source rule.

Stateless MCP removes transport state, not business state

The new MCP design removes protocol sessions from the core request path. Each request can carry the protocol version, client identity, and capabilities it needs. Gateways can use method metadata without parsing arbitrary JSON, while authorization improvements bind tokens more tightly to the intended server and issuer.

That simplifies deployment and scaling. It does not mean the workflow has no state.

A refund tool still has an order state. A deployment tool still has an approval state. A quote tool still has a CRM record. An elicitation can ask the client for missing information and then retry the operation. The application must decide which state belongs in the request, which belongs in an authoritative backend, and which must never be reconstructed from conversational memory.

For operators, stateless transport strengthens an important pattern: every sensitive tool request should be understandable without replaying an entire chat transcript.

{
  "request_id": "tool_01K2...",
  "client_id": "approved-agent-client",
  "principal_id": "user-or-service-identity",
  "tool": "create_quote_draft",
  "capability": "draft_only",
  "resource_audience": "https://quotes.example.com/mcp",
  "source_record_version": "lead_482_v7",
  "approval_state": "standard_intake_processing",
  "idempotency_key": "quote_lead_482_v7",
  "context_refs": ["policy_2026_08", "service_area_v14"],
  "memory_used": false
}

This envelope should carry references and decisions, not a dump of every prompt, retrieved document, or customer message. The receiving service can fetch authoritative records under its own access policy.

Observability can become the largest context copy

Cloudflare's unified AI control-plane announcement says a default gateway can be created on the first authenticated request and can record full request and response payloads, token counts, cost, latency, and errors. That visibility is excellent for debugging model behaviour and attributing spend. It also means the observability layer may receive a more complete copy of the interaction than the application database was designed to retain.

GitHub's update similarly makes Codex sessions visible in agent debug logs and gives administrators OpenTelemetry controls. Again, visibility is useful. The operating mistake is to enable rich diagnostics without deciding what those diagnostics may contain.

Apply a three-level logging policy:

Log class	Keep	Exclude by default
Operational metrics	Request ID, route, model, latency, token counts, cost, status, fallback reason	Prompts, retrieved chunks, tool arguments, generated free text
Debug sample	Redacted prompt shape, bounded response excerpt, tool names, error details	Secrets, customer messages, source code not needed for diagnosis, payment or auth data
Incident capture	Time-limited approved payload capture for a named investigation	Indefinite full-payload logging or broad analyst access

Prompt caching needs the same review. A cache read can reduce cost, as GitHub's new per-model token breakdown now makes more visible through separate input, output, cache-read, and cache-write token counts. But a useful cache is another retained representation. Cost efficiency and privacy policy have to agree on its key scope, contents, location, and expiry.

A context-copy register

Before enabling persistent memory, retrieval, local models, or new MCP tools, create one register for the workflow. This example is intentionally small enough for a repository policy file or operations runbook:

agent_context_register:
  workflow: local_service_quote_assistant
  claim: local_inference_with_controlled_cloud_context
  owner: growth_operations
  copies:
    - name: authoritative_lead
      kind: source_record
      location: crm
      data_class: customer_pii
      access: assigned_sales_and_quote_service
      retention: business_record_policy
      deletion_owner: privacy_operations
    - name: public_service_knowledge
      kind: retrieval_index
      location: managed_search
      data_class: public
      access: public_query
      refresh: daily
      provenance_required: true
    - name: staff_runbooks
      kind: retrieval_index
      location: separate_private_namespace
      data_class: internal
      access: staff_identity_required
      public_endpoint: false
      deletion_sla: 24_hours_after_source_removal
    - name: agent_memory
      kind: persistent_memory
      location: configured_client_service
      allowed_data:
        - writing_preferences
        - non_sensitive_project_conventions
      forbidden_data:
        - lead_content
        - credentials
        - prices_or_availability
      expiry: 90_days
      user_delete_test: quarterly
    - name: model_context
      kind: inference
      route: ollama_on_managed_workstation
      network_fallback: disabled_for_customer_data
    - name: quote_tool
      kind: mcp_request
      access: authenticated_principal
      capability: draft_only
      idempotency: required
      context_mode: references_not_transcript
    - name: operational_trace
      kind: telemetry
      payload_logging: false
      fields:
        - request_id
        - model
        - latency
        - token_count
        - tool_result_code
      retention: 30_days
  revocation_tests:
    - remove_runbook_then_confirm_index_deletion
    - revoke_staff_user_then_query_private_namespace
    - delete_memory_then_start_new_session
    - block_network_then_confirm_no_remote_inference_fallback
    - submit_canary_secret_then_confirm_logs_do_not_capture_it

The claim at the top matters. It prevents the team from describing the whole system as local when only inference is local. It also gives security, privacy, and support staff a testable statement instead of a product label.

Failure modes worth exercising

Local inference with remote leakage

The model runs through Ollama, but retrieval, memory, telemetry, a plugin, or an MCP tool sends sensitive context elsewhere. Capture outbound connections in a test environment and compare them with the register.

Deleted source, live index

A runbook, customer file, or obsolete policy is removed from its source while chunks remain searchable. Test deletion propagation, not only ingestion speed.

Correct retrieval, stale memory

The current source says one thing and persistent memory says another. Verify that the agent cites and follows the authoritative version, then corrects or expires the conflicting memory.

Public endpoint over private aggregate

A namespace that began with public documentation later gains an internal source. Require exposure review whenever a source is added; do not inherit the namespace's old public status automatically.

Debug mode becomes normal retention

Full payload capture is enabled during an incident and never disabled. Give incident logging an expiry, owner, access list, and closure check.

Remote fallback changes the privacy claim

A local model is unavailable and the client silently selects a hosted provider. For sensitive task classes, fail visibly or require approval before changing the inference boundary.

Tool retries duplicate business work

A stateless client retries after a timeout. The transport has no session, but the first request created the quote, issue, email, or booking. Require an operation-level idempotency key and reconcile against the business system.

The practical conclusion

Persistent memory, local models, searchable private knowledge, stateless tools, and rich observability can all make agents more useful. None is a complete context policy.

Start with the workflow claim, then enumerate every copy between the source and the business outcome. Keep retrieval provenance separate from remembered preference. Preserve source authorization when indexes are aggregated. Put explicit identity and capability on MCP requests. Treat caches, traces, and debug logs as data stores. Test deletion and fallback with the same seriousness as the happy path.

A local model is then genuinely useful for what it changes: the inference boundary. The rest of the context path remains visible, named, and governed instead of disappearing behind the word local.