The city of Kelowna once asked its AI chatbot which beaches had lifeguards.
The response was returned in false confidence.
The bot listed lifeguards at the beaches that didn’t have them because somewhere, deep on the city’s website, there was an old PDF from when Kelowna’s beaches had lifeguards.
No one had touched that document in years, but HE found it anyway.
You can bring Jurassic Park and “life finds a way”, but that’s a whole other kettle of fish with AI and the human thoughts and emotions of it all. It is, of course, quite an anecdote.
“It’s a funny example,” said James McGregor, chief technology officer (CTO) for the city of Kelowna, in a CIOCAN Peer Forum session on moving from AI pilots to enterprise intelligence, “but it’s also a pretty critical mistake.”
Aside from being a laugh, it’s also a pretty accurate example of why AI transformation in the public sector is all about plumbing.
Canadian municipal technology leaders carrying decades of accumulated technical debt need to know how to incorporate artificial intelligence without ancient PDFs that eat away at your credibility.
To explain how Kelowna does it, especially budget-wise, McGregor turns to baseball.
He compares their approach to ‘small ball’ or hitting singles and doubles. In practice, this means targeting areas where data is clean enough to be trusted, processes stable enough to be automated, and the blast radius of a bug is manageable.
Kelowna’s chatbot program started as a way to answer common service questions after hours. It now handles approximately 180,000 inquiries per year, with a conversion rate of about 40% on interactions that do not require human intervention. It’s certainly not a moonshot because it’s a measurable, repeatable win that buys you the credibility to go after bigger things.
The approach McGregor and his team took has lessons for any tech leader who’s ever had to tell a CEO that the foundation needs work before the bigger stuff hits the ground.
The 30-year problem sitting in your infrastructure
Kelowna’s journey with artificial intelligence began earlier than most people would expect. McGregor’s predecessor began exploring AI-assisted citizen services in 2018, long before LLMs were a board-level conversation.
City offices were open from 08:00 to 16:00, but citizens need information around the clock. Chatbots can answer questions and service requests after business hours. These small, focused applications were designed to expand the reach of the city without additional employment that might otherwise be needed. proportional increase in the number of employees.
What McGregor inherited when he joined some 18 months ago was a start and an account.
“We’re dealing with these last elements of AI,” he said. “But we have this 30-year-old ColdFusion thing sitting out there.”
Modernization of legacy infrastructure also deploying new AI capabilities is a problem facing technology leaders in mid-market organizations across Canada.
Boards and suites go Marco Polo and “discover” AI, but the infrastructure just isn’t there.
McGregor’s approach is to run both tracks simultaneously rather than sequencing them. Deploy artificial intelligence in areas where data collaborates and drive fundamental modernization in parallel.
The city is in the midst of a major enterprise resource planning (ERP) implementation, and data governance work is directly related to that project.
Theory, meet practice.
“Part of my challenge is moving from theory to practice,” McGregor said. “What does it actually mean? How does that show up in a meaningful way?”
More technology leaders need to ask their teams this question out loud.
Playing small ball and why it’s a strategy
In a session nominally about scaling AI, McGregor spent a fair amount of time making the case for not scaling.
At least not yet, and not everything everywhere at once.
He describes it as being kind to your future self. Large, decentralized AI implementations without proper governance create the exact technical debt you’re trying to escape.
For example, agents end up interacting with other agents in ways that produce unintended consequences, or processes that engage the business in ways that IT owns but cannot sustainably support.
Continuing with the baseball metaphor, he says, “Trying and hitting home runs creates tremendous risk and is probably inconsistent with where we are with this technology.”
The success of Kelowna’s chatbot program underscores this point. The city is not yet ready to declare victory in citizen service automation, but the existing deployment acts as a proof of concept for a larger central city services project.
It’s similar to a 311 service that Kelowna doesn’t currently have. The chatbot’s data informs what that hub should look like, where the demand is focused, which questions need human judgment, and where the delivery points should be.
Robbie Beyer, director of data science and AI at RSM, worked with Kelowna on the implementation.
As AI models continue to advance in capability, what will differentiate organizations is the organizational data to which those models have access.
“They’re being trained on data that’s on the open Internet, but they’ve never seen data specific to your organization,” Beyer said. “It’s going to set you apart from the organization down the road. So as you double down and focus on getting the database right, it’s going to be continually more and more valuable as features continue to advance.”
For public sector organizations in particular, this fundamental work is a competitive gap.
A city that has clean, well-governed data about its permitting process, citizen service models, and operational workflows will find value in future AI capabilities. Contrast this with the city that chased every shiny object and now has a governance problem layered on top of a data problem layered on top of a legacy infrastructure problem, like a civic turducken.
What the private sector keeps getting wrong
The question of failure rate came up during the hearing and McGregor didn’t shy away from it. It puts the failure rate of AI projects at roughly double that of other technology initiatives.
Research backs up his assessments. In partnership with Lenovo, IDC found this out 88% of proof of concepts observed don’t switch to large-scale deployment. An MIT report from 2025 found that 95% of organizations had an ROI of zerodespite enterprise investment in GenAI reaching $30-40 billion.
His theory is that they are nebulous in nature. The definition of success is vague and its value is difficult to measure. Not to mention that process changes often end up being much more involved than originally thought.
“Having a partner who’s been through it (and) can guide you through that process is really key,” McGregor said. “But you don’t want to give up the strategic value of the work. You don’t want to give up the strategic understanding, and you want your teams to be involved in those kinds of things.”
Kelowna’s model is a hybrid, with partners when they don’t have in-house expertise or when they need to move quickly, and in-house ownership of strategy and architecture decisions, adding staff for specialized skills that can’t be hired in a reasonable timeframe.
McGregor was frank that municipal hiring timelines are not designed for the pace of technology change. Planning six months ahead for a role is not a realistic time when the need may have evolved by the time the new employee is signing internal paperwork.
The broader lesson is one that the private sector is slower to absorb than it should be.
Transformation programs, with their promises of leapfrogging from legacy to leading edge in a single initiative, consistently underperform compared to targeted pilots that build institutional knowledge alongside the technology itself.
Each failed pilot is a hidden data management check. The organizations that handle it that way are the ones that get somewhere.
The last shots
- 180,000 citizen questions were handled by Kelowna’s chatbot last year, 40% without human intervention, and the city still calls it a proof of concept.
- A 30-year technological footprint will not be a barrier to deploying AI. You just have to start where the data cooperates.
Digital Journal is the national media partner for the CIO Association of Canada.
