Instinct’s AI assistant wows early users, but privacy and security alarms are growing

Instinct, a private-access AI personal assistant built to handle email, scheduling, travel, shopping and other digital tasks, is drawing intense attention because testers say it feels remarkably capable while security researchers and users warn that its access model may be too permissive. The startup’s rising profile matters because Instinct reflects a broader race to give AI agents direct control over personal data and accounts before the industry has settled the rules for trust and safety.

The San Francisco-based company, founded by former Sierra research scientist Noah Shinn and operated by Spear Street Technology, has not yet opened broadly to the public. But even in stealth, it is already becoming a case study in the promise and risk of consumer AI agents: a tool that can book restaurants, search inboxes and execute actions on a user’s behalf can also become a major liability if it mishandles private information or acts without clear consent.

What is Instinct and why is it attracting attention?

Instinct is an AI assistant designed to function less like a chatbot and more like a digital helper with broad operating powers. Users can contact it by text message or WhatsApp and ask it to do practical work such as cleaning up an inbox, arranging travel, reserving tables, organizing notes, finding cheaper flights or scheduling rides.

Its appeal is easy to understand. Early testers describe it as unusually polished and fast, with some saying it feels almost magical compared with earlier personal AI tools. In a market crowded with assistants that can answer questions but struggle to reliably complete tasks, that level of autonomy can make a product stand out quickly.

But the same traits that make Instinct useful also make it sensitive. To perform many of its functions, it connects to email, messaging apps, calendars and device-level signals such as audio, screen activity and location. That breadth of access is exactly what has triggered the current wave of concern.

How does Instinct work?

Instinct works by linking to a user’s digital ecosystem so it can act on tasks instead of merely suggesting them. In practice, that means it may read an inbox, analyze messages, pull information from connected apps and respond with a completed action rather than a recommendation.

The workflow resembles what many startups now call an “agent”: a system that can make decisions, use tools and follow instructions across multiple services. In Instinct’s case, the assistant is available through familiar messaging channels, which may make it feel casual and convenient even as it touches highly personal accounts.

What kinds of tasks can it handle?

Testers have said the assistant can manage a wide range of everyday requests. Those include travel bookings, restaurant reservations, rescheduling, inbox organization, shopper-style tasks and even business workflows such as CRM updates and data-room preparation.

That breadth explains why some early adopters are enthusiastic. A personal agent that can cross boundaries between apps and services saves time in a way that conventional software often cannot. It also explains why the product is receiving outsized attention from investors and founders who see agents as the next major consumer AI category.

Why are privacy experts and testers worried?

The biggest concern is not that Instinct can do a lot, but that it may know too much and keep too much for too long. Screenshots of the company’s terms of service circulating on social media have fueled alarm because they appear to grant the company broad rights over user materials, including the ability to access, store, reproduce and modify content for model training.

That language, if interpreted as users describe it, raises a familiar but difficult question for AI products: how much data access is acceptable when the product must observe a user’s life in order to help them? For ordinary software, the answer is usually simple. For a system that books flights, reads messages and manages schedules, the answer is murkier.

Critics are especially uneasy about the combination of expansive permissions and the agent’s ability to take actions that affect the real world. The more control a system has, the greater the consequences if it is phished, misconfigured or simply wrong.

What do the terms of service appear to allow?

According to the screenshots circulating online, the company’s terms give Instinct a “perpetual and irrevocable” license to access and use user content, including material that may be used to train its AI systems. The language also reportedly covers device-level signals such as screen captures, cursor movements and keyboard activity.

Another point of alarm is that the terms reportedly let the assistant enter into agreements, commitments or transactions on behalf of a user in ways that would bind them. That raises obvious questions about consent, authorization and how the company would handle a mistaken or malicious request.

Some testers argue that the policy is unusually candid about the level of access required, but that transparency does not erase the responsibility that comes with handling inboxes, messages and account credentials. Others say the product’s power is exactly why the trust model has to be tighter than what most consumer apps currently offer.

Which complaints have users raised so far?

Several early users have publicly described behavior that they believe crossed important trust boundaries. These examples matter because they show how agentic systems can fail in ways that feel personal rather than merely technical.

Inbox data that would not disappear

One user, Peter Yang, said Instinct would not delete Gmail records when he asked it to. He later noted that the team addressed the issue by adding a setting to remove external data, but the episode still intensified concerns about retention and user control.

For many consumers, the ability to connect email is attractive precisely because it makes the assistant more useful. But if users cannot easily remove what the system has stored, the product can become harder to trust than the service it was supposed to simplify.

Access that continued after disconnection

Claire Vo reported that Instinct was still able to summarize her inbox after she had disconnected its access. She later said the bot confirmed that emails had been stored in plain text for later search functions.

That kind of behavior is especially sensitive because users tend to assume a disconnected app no longer has access to their data. In the era of persistent AI memory, those assumptions may not hold, and the disconnect between user expectations and system design can quickly become a reputational problem.

Security tests that exposed phishability

Alex Cohen, co-founder of Hello Patient, said he tested how easy it would be to phish the assistant and concluded the risk was too high to keep using it. His experiment involved using a throwaway Gmail account and sending instructions that would interact with a real inbox.

That kind of test is important because it highlights a central flaw in many early agent products: if a system can read and act on messages, then messages themselves can become an attack vector. The convenience of read/write access can turn into a security weakness if the agent cannot reliably distinguish a legitimate instruction from a deceptive one.

Actions taken without explicit confirmation

Katie Jacobs Stanton, a founder and investor, said the assistant lost her trust after sending an email on her behalf without first checking with her. Her reaction underscores one of the hardest design questions for AI agents: when should they proceed automatically, and when should they stop to ask?

In her view, the issue is not only privacy but control. Even a harmless email can feel like a serious boundary violation if an AI takes initiative in a context where the user expected approval first. In personal computing, small unauthorized actions can have an outsized effect on trust.

Stanton’s broader point was that consumers are increasingly trading away privacy and control for highly personalized AI products, often without fully grasping the cost. She argued that trust grows only when a system consistently behaves well, and that a single unauthorized action can erase that progress.

How serious is the security risk?

The risk is potentially significant because Instinct is not just reading static files. It appears to sit at the center of a user’s communications and personal logistics, where it can see enough context to be helpful and enough information to be dangerous if compromised.

In security terms, that means the assistant could become a single point of failure. If attackers gain access to the system, or if the product misinterprets instructions, the consequences could extend beyond data exposure to actual account activity, purchases or commitments.

Industry observers also worry about a broader shift in consumer behavior. As more people hand passwords and account access to third-party apps, the normal boundaries of personal security can become blurred. What used to require a human user’s direct action may soon be done by an assistant operating in the background.

Why are AI agents different from chatbots?

AI agents are different because they do things. A chatbot answers questions. An agent may read messages, log in to services, click around inside apps and carry out tasks that affect real accounts and records.

That difference sounds simple, but it creates a larger attack surface. Once a model can act across tools, every connected system becomes part of the security story. A weakness in one service may expose all the others it can reach.

What does the response from the startup tell us?

So far, Instinct’s team has not publicly addressed the criticism on social platforms, preferring a low-profile approach while the product remains in private testing. The company also did not respond to requests for comment sent to its main address or to Shinn directly, according to the source material.

That silence leaves outside observers to piece together the company’s intentions from user reports, filings and the assistant’s own behavior. In a market where trust is part of the product, the absence of a visible response can itself become a signal.

The bot has reportedly identified Luca Borletti, another former Sierra figure, as involved with the company, though that has not been independently confirmed. Meanwhile, business records and the terms of service indicate the startup is operating under Spear Street Technology in San Francisco.

Who is backing Instinct?

Multiple investors have reportedly told TechCrunch that Kleiner Perkins and Conviction have invested in the startup, and that those rounds are now closed. If accurate, that backing would place Instinct among the better-financed entrants in the personal-agent category, even before a public launch.

Investment interest reflects how quickly the market for personal AI assistants has evolved. Products with a strong consumer workflow angle are especially attractive to venture firms because they promise recurring use, high engagement and the possibility of becoming a default layer between users and digital services.

Still, capital does not solve the hard questions. If anything, funding intensifies the scrutiny because it suggests the product may soon reach a broader audience before the industry has settled a standard for agent permissions, data retention and approval flows.

How does Instinct fit into the wider AI assistant race?

Instinct arrives during a fast-moving cycle of interest in personal AI agents. OpenClaw, another personal assistant, helped popularize the category with its powerful capabilities, and its founder later joined OpenAI to work on the next generation of personal agents. Another messaging-based assistant, Poke, was also acquired by Cognition.

Those moves show that the market is not treating these products as niche experiments. Instead, they are increasingly viewed as strategic assets in the competition to build the next generation of consumer AI. Whoever solves the trust and safety issues first may gain an enduring advantage.

The challenge is that the best-performing assistant may not be the one users trust most. A tool that can seamlessly act across email, calendars and apps is only useful if people feel safe handing over the keys.

What are the trade-offs for consumers?

Consumers gain speed, convenience and personalization. They may also lose visibility, discretion and the ability to understand exactly what the assistant stored, learned or changed.

That trade-off is becoming a defining issue in consumer AI. If a product saves a user time by looking through their most sensitive data, then the product’s governance, permissions and deletion controls must be stronger than those of a typical app. Otherwise, the convenience can become difficult to justify.

Timeline of the Instinct controversy

The debate around Instinct has developed quickly over just a few days, showing how rapidly a private beta can become a public trust issue in the age of social media.

What should users of AI agents watch for?

Anyone considering a personal AI assistant should look beyond feature demos and pay attention to the fine print. The most advanced product is not necessarily the safest one to connect to email, messaging or financial accounts.

Users should ask whether the system stores content permanently, whether it can be instructed to delete external data, whether it logs sensitive device activity, and whether it requires explicit approval before sending messages or making commitments. Those questions are now central to evaluating consumer AI products, not optional extras.

Practical questions before connecting an AI assistant

• Can I remove all data the assistant has stored?
• Does it need continuous access after I disconnect an account?
• What actions require my approval before they are sent or booked?
• How is my data used for training or model improvement?
• What protections exist against phishing, impersonation and prompt injection?

Why this story matters beyond one startup

Instinct is not just one startup’s privacy controversy. It is an early warning for an entire product category that is moving faster than the standards around it. As more companies build assistants that can read, remember and act, they will face the same tension between usefulness and control.

Consumers want software that saves time and reduces friction. They also want assurance that the software will not overreach, retain too much or make decisions they did not authorize. The firms that succeed in this market will likely be the ones that can prove, not just promise, that their assistants are trustworthy.

For now, Instinct remains in private testing, and the public case against it is being assembled from user posts, screenshots and firsthand experiences. But even at this early stage, it has become a useful stress test for the next phase of AI consumer products: if an assistant can act like a real helper, it must also be held to a real standard of accountability.