Box CEO Says Companies Will Have to ‘Build Something That Agents Want’ as AI Agents Become New Consumers

Box Inc. (NYSE:BOX) CEO Aaron Levie argued that AI agents could fundamentally reshape consumer technology by forcing companies to design products that agents can navigate and operate on users’ behalf. AI Agents Could Become New Digital Gatekeepers On Saturday, in a post on X, Box CEO said the rise of personal agents would create a new model in which people delegate tasks to AI systems that can complete them from start to finish. "Personal agents are the ultimate manifestation of ‘build something that agents want,’" Levie wrote. He pointed to products such as Muse, saying users increasingly want to "hand off a task to the agent and ensure that it is fully completed end to end." For that to happen, agents need to interact directly with the tools and services required to complete tasks. Levie said they should be able to "use your MCP or CLI, easily navigate your site, be able to transact, and more." AI Agents Could Become New Digital Gatekeepers Levie argued that companies may soon have to compete for the attention of AI agents rather than directly for consumers. "The new attention you need to compete for is not from the user itself but instead for the agent," he said. He highlighted tools that allow agents to order food, conduct e-commerce transactions, book flights and interact with data as examples of services that could benefit from the shift. Levie said this transformation could represent "the biggest opportunity and shakeup in consumer tech since the App Store itself." Personal agents are the ultimate manifestation of "build something that agents want".The form factor of a product like Muse is you want to be able to hand off a task to the agent and ensure that it is fully completed end to end. To do this, the agent must be able to… https://t.co/33UJK9moAI— Aaron Levie (@levie) September 19, 2026 Read Also: Every AI Agent Needs a Gatekeeper: 3 Cybersecurity Stocks to Watch AI Agents Raise Safety and Control Concerns Last week, Anthropic said Claude performed 26% of its measured AI R&D work in August, up from 1% in March, while about 30,000 AI agents handled research and engineering tasks. More than 90% of the measured AI R&D work reached the “AI collaborates” level or higher. Anthropic also reported that about 6% of its AI R&D compute was devoted to safety research in a July snapshot. Meta Platforms, Inc. (NASDAQ:META) CEO Mark Zuckerberg said the company delayed its Muse AI agent for months to address safety and security concerns. He supported independent evaluators and said companies that failed to prioritize AI alignment could face liability if systems caused harm. Microsoft CEO Satya Nadella said AI development needed human control, rigorous testing and stronger safeguards. He warned that persistent AI agents could create new insider risks, including manipulating business records, and called for third-party evaluations and greater transparency. Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Read Also: Daniel Newman Praises Zuckerberg For 'Doing The Right Thing' With Muse AI Safety Despite Investor Pressure: 'Didn’t Feel The Need to Bring Bernie Sanders Into' This Photo courtesy: Shutterstock