Introduction to VS Code's Air-Gapped AI Mode

Microsoft has been aggressively integrating artificial intelligence into Visual Studio Code, primarily through GitHub Copilot. While Copilot offers deep integration and convenient features like inline autocomplete, it requires a GitHub account and constant internet connectivity, which is not suitable for all development environments. Many developers work in air-gapped or restricted networks where external API calls are impossible or prohibited. Additionally, some prefer to use other model providers or locally hosted large language models (LLMs) for reasons of cost, privacy, or customization.

Visual Studio Code 1.122 introduced a significant new feature called "Use BYOK (Bring Your Own Key) without a GitHub sign-in." This feature allows developers to use chat, tools, and Model Context Protocol (MCP) servers within air-gapped environments. More importantly, it enables fully offline workflows with local models such as those served by Ollama or LM Studio. This development is a major step toward making VS Code a more flexible and secure AI-assisted development platform, especially for enterprise users who cannot rely on cloud-based AI services.

Understanding BYOK Mode and Its Capabilities

The BYOK mode in Visual Studio Code essentially decouples the AI assistant from the requirement of a GitHub Copilot subscription or any internet-based authentication. Instead, developers can configure their own AI endpoints, whether they are hosted on the same machine, on a local server, or even on a cloud instance that is accessible within the network. The mode supports chat conversations, tool calls, and the integration of MCP servers, which can be used to extend functionality like file operations or API queries.

However, there is one important limitation: the built-in BYOK functionality does not support inline suggestions or code completions. Inline completions remain a feature exclusive to GitHub Copilot or third-party extensions like Continue. Microsoft has not indicated whether they plan to open this functionality to custom endpoints. As a result, developers who want a fully local AI coding assistant must use additional tools alongside VS Code's built-in chat and tool capabilities.

Choosing a Model for BYOK Mode

To use a local LLM with VS Code's bring-your-own-model system, you first need a way to host the model. VS Code itself does not include a model-hosting mechanism, but extensions may offer this in the future. For now, dedicated applications like LM Studio or Ollama are commonly used. These tools provide a user-friendly interface for downloading, serving, and managing LLMs on your own hardware. The model host does not have to be the same system as VS Code; it can be a separate server or cloud instance, as long as it is accessible over the network.

Choosing the right model is crucial for performance and usability. Many state-of-the-art models are too large to run on commodity hardware, so developers should select models that fit into existing VRAM while leaving room for a sizable token context. Additionally, the model should be fine-tuned or optimized for coding and development tasks. Some excellent options that run comfortably in 8GB VRAM include Gemma4 (effective 2 billion parameters version), Qwen3.5 9B, and Codestral 22B v.0.1 (which has a proprietary license but performs well). For those with more VRAM, larger models like DeepSeek-Coder or Code Llama can provide even better accuracy.

Setting Up BYOK Mode in VS Code

Once you have a model up and running on a hosting platform, integrating it with Visual Studio Code is straightforward. First, ensure that VS Code's AI features are enabled. The setting chat.disableAIFeatures must be set to false. You can find this in Settings -> Chat -> Miscellaneous.

Next, manage the language models list. Press Ctrl+Shift+P and type Manage Language Models. This opens a page showing all currently configured models. Initially, only the built-in models (such as GitHub Copilot) appear. To add a custom model, click Add Models at the top right and select Custom Endpoint.

You will then be prompted for several pieces of information:

• Group Name: Default is "Custom Endpoint," but you can rename it for organizational purposes. This name does not affect connectivity.
• API Key: If your model hosting platform requires an API key, enter it here. For local hosts without authentication, leave it blank.
• API Type: Choose from Chat Completions, Responses, or Messages. Responses is generally the best choice for compatibility with most models.

After providing these details, a modal editor opens with a JSON file containing the endpoint configuration. You will need to fill in three important fields:

• id: A unique identifier for this endpoint. It can be the model name or any arbitrary string.
• name: The exact model identifier as recognized by the server. In LM Studio, you can copy this by going to My Models, clicking the three-dot icon next to the model, and selecting Copy Default Identifier. For example, qwen2.5-coder-7b-instruct.
• url: The endpoint URL of the model server. For LM Studio, this is typically http://127.0.0.1:1234/v1. The /v1 suffix is essential for automatic discovery of model capabilities.

Other fields, such as vision, can be left at their default values. Set vision to false if the model does not support image inputs. Once the JSON is saved, the new endpoint appears in the Manage Language Models list. You can now open the chat window and select this model for conversation and tool usage.

Working with Local Models in VS Code Chat

After configuration, the chat interface works similarly to GitHub Copilot Chat. You can ask questions about your code, request refactoring suggestions, or use slash commands for common tasks. The model can also interact with tools such as file editing or terminal commands, depending on the MCP servers you have configured. One significant advantage is that all processing happens locally, ensuring data privacy and low latency. This is especially beneficial for sensitive projects or when working without internet access.

However, remember that the local model will not provide inline suggestions as you type. For that functionality, you must rely on third-party extensions like Continue, which can use the same local model for completions. Continue also offers advanced features such as multi-file editing and documentation generation, bridging the gap left by VS Code's built-in limitations.

Expanding the Use of Local AI in VS Code

While the current BYOK mode only covers chat and tools, developers can extend it significantly through customization. For example, MCP servers can be built to automate repetitive tasks, analyze codebases, or interact with databases. The combination of a local LLM and custom MCP tools creates a powerful, self-contained development environment that rivals cloud-based solutions in many respects.

Looking ahead, the community may develop extensions that add inline completion support for custom endpoints. Microsoft's roadmap hints at more flexibility in AI integrations, but for now, the air-gapped mode remains a valuable feature for those who need it. By choosing appropriate models and setting up a reliable hosting platform, developers can enjoy AI assistance without compromising on security or independence.

Source:InfoWorld News