When you buy through affiliate links in our content, we may earn a commission at no extra cost to you. Learn 
ChatGPT’s macOS app now supports code reading in popular programming-focused applications like VS Code, Xcode, and Terminal.
Launched in May, the app has been gradually enhanced, with the new “Work with Apps” feature eliminating the need to copy and paste code. When activated, this feature automatically transfers the currently active section as context with the prompt. But, unlike tools such as Cursor or GitHub Copilot, ChatGPT cannot input code directly into developer applications.
ChatGPT 🤝 VS Code, Xcode, Terminal, iTerm2
ChatGPT for macOS can now work with apps on your desktop. In this early beta for Plus and Team users, you can let ChatGPT look at coding apps to provide better answers. pic.twitter.com/3wMCZfby2U
— OpenAI Developers (@OpenAIDevs) November 14, 2024
OpenAI utilizes the macOS accessibility API and VoiceOver screen reader to capture text from apps, which works reliably with most common applications. Users must install a specific extension for tools like VS Code. However, the screen reader cannot interpret visual elements such as videos or photos.
OpenAI confirms that ChatGPT can only view, not alter, app content, and the feature requires manual activation. It’s available to Plus and Teams users, with plans for availability to Enterprise and Education users in the coming weeks. The arrival of “Work with Apps” on Windows is still unclear.
A Step Toward AI Agents
OpenAI is starting with coding apps, as AI coding assistants are widely used for LLMs, and plans to expand ChatGPT’s capabilities to text-oriented applications designed for writing tasks.
The “Work with Apps” feature is not yet an AI agent, but OpenAI views it as a crucial step toward developing autonomous systems, as AI agents still need assistance understanding the entire display, not just prompts or their own outputs.
This move is significant, especially since OpenAI reportedly plans to launch an all-purpose AI agent, “Operator,” in early 2025. The tool will compete with other emerging AI agents, such as Anthropic’s “Computer Use” and Google’s rumored “Jarvis.”
