This interview between tech legend Guy Kawasaki and Meredith Whittaker, president of encrypted messaging app Signal drew some attention recently among the AI-skeptical. At the 50-minute mark of the linked video, Whittaker raises the issue of Agentic AI, and describes the multiple points of vulnerability encountered in a single agentic transaction, positing a scenario where the agentic AI looks up a concert, books a ticket, schedules it in your calendar, and messages your friends to tell them you’re attending. Such transactions would require accessing your browser, credit card information, calendar, and messaging app, which creates, according to Whittaker, “a profound issue with security and privacy.”
Whittaker oversees an app devoted to placing security above all other considerations, so it’s not surprising that she’d raise this alarm. Wired called her “one of the world’s most prominent tech critics,” and her own view can be inferred from her offhand remark in the video above about agentic AI letting users “put our brains in a jar” while the agentic AI serves as a “magic genie bot that’s going to take care of the exigencies of life.” Nevertheless, I think it’s incumbent on the technology providers, whether enterprise- or consumer-focused, to explain in detail why users shouldn’t be concerned about the level of access Whittaker describes, and the potential for catastrophic security breaches it threatens.
Yet there are signs that users aren’t worried enough about this. A recent Boston Consulting Group survey reveals a level of concern about agentic AI that I found shockingly low. BCG reports that the top 3 concerns about agentic AI among users – in a survey base equally divided among frontline employees, managers, and leaders -- are:
- Decisions taken without human oversight: 46%
- Unclear accountability when mistakes happen: 35%
- Bias or unfair treatment introduced: 32%
The kinds of security and privacy issues Whittaker mentions don’t even make the top 3, but that’s only the beginning. I find it incredible that less than half of those surveyed were concerned that they might lack sufficient oversight on agentic AI. So, more than half of all users presumably believe some combination of:
- They actually will have some human oversight (hope they’re right, but it kind of seems counter to the agentic AI promise)
- They won’t need human oversight; the technology will be just that good (seems naïve; could be a career bet)
And only a third seem to be worried about any sort of audit trail or bias, which also seems…problematic.
We’re in early days with agentic AI. BCG’s results show fairly limited agentic AI adoption: Just 13% say AI agents are already integrated into broader workflows in their company; 56% are piloting, and 31% aren’t using the technology at all. It’s critical for IT leaders to get in front of the agentic AI hype, set clear expectations with users, and fight for strong governance among business and security leadership. Users will be a critical focus, if for no other reason than the emergence we’re already seeing of Shadow AI. Indeed, more than half (54%) of BCG’s respondents said they’d use AI tools even if not authorized by the company; for Millennials and Gen Z that figure is 62%.
Agentic AI offers promising applications on the enterprise side, and maybe the more extreme concerns that Whittaker expresses will prove to be more hype than reality. It's definitely an area for due diligence on the part of both vendors and their enterprise customers.
Stay Ahead with Enterprise Connect
Enterprise Connect is your premier destination for staying ahead in enterprise collaboration and CX. What role will AI play in shaping the future of enterprise communications? The answer is that AI will undoubtedly have a significant impact, but as always, our event goes beyond just AI. We aim to place AI in the broader context of the many challenges and opportunities enterprise collaboration and CX professionals navigate daily.
Our mission is to equip you, as an enterprise leader, with essential tools and actionable insights to craft a strategy tailored to your organization's unique needs. Alongside a deep dive into AI, our comprehensive program includes dedicated tracks on AV/Video, Collaboration Platforms/Unified Communications, CPaaS & Programmable Communications, CX/Contact Center, and Security/Compliance.
We’re also excited to offer a Strategy & Management track, focusing on real-world applications and solutions. Most sessions in this track are case study discussions led by enterprise leaders, providing unparalleled opportunities to learn from and connect with your peers.