Enterprise leaders are swarmed by generative AI service providers and their lofty promises as interest in the technology spreads across industries. But transparency around the risks and limitations of AI tools is often limited.
As CIOs turn to generative AI to improve user experience, speed up work and reduce administrative burdens, it’s up to leaders and procurement teams to know when to walk away during solution selection — and what mitigation techniques to deploy as they push forward on AI initiatives.
“A vendor might claim to use AI when it is minimal or non-existent,” Maxime Vermeir, senior director of AI strategy at ABBYY, said in an email. “To avoid falling for superfluous claims of AI integration, IT leaders should question if a solution can genuinely solve a relevant business challenge and if the vendor can provide evidence of its success.”
On a high level, Vermeir recommends IT leaders ask four questions to identify potential risk areas:
- Does the AI solution address the business problem or opportunity?
- Can the AI vendor provide evidence of success?
- Is the AI solution rooted in ethical and responsible practices?
- Does the vendor offer ongoing support and improvement post-adoption?
As tech leaders carry out due diligence on vendors, they could run across red flags.
CIOs should assess agreement terms, model specifics and expected benefits, among other details. However, not all risks outlined by vendors or uncovered along the way should be weighed evenly, according to Amy Caiazza, partner, fintech and financial services at Wilson Sonsini Goodrich & Rosati.
Every generative AI solution will have pros and cons.
There’s a certain amount of risk that comes with most generative AI solutions. Inaccurate outputs represent one common concern for executives. But CIOs can respond by upskilling employees in fact-checking and encouraging them to deploy a level of skepticism when assessing generated outputs. Other risks, though, aren’t as easily resolved.
“If there’s a red flag or problems and you can’t address them, then is the technology really that important that you should be engaged in this relationship and should you be using that service provider?” Caiazza said.
Not all AI solutions hit the mark
To enter the procurement process with a clear understanding of the different risks associated with a variety of vendors, CIOs can perform a thorough market analysis.
Solutions that are popular now are likely to change standings as competition in the marketplace continues, Maneesha Mithal, partner, privacy and cybersecurity at Wilson Sonsini Goodrich & Rosati, told CIO Dive. Nearly all venture capital-backed companies laid out plans to launch generative AI in their products last year, according to a Productboard survey of more than 300 CEOs.
The product deluge has muddied the waters.
“It’s creating a lot of stress for practitioners and for our clients,” Bret Greenstein, data and analytics partner and generative AI leader at PwC, said. Buyers should consider whether a solution is worth the time and resources need to make sure it’s safe, certified and differentiated, Greenstein said.
The upside of keeping up with the ever-expanding generative AI marketplace is that it can help teams better ground their understanding of the impact and severity of risks through comparison, Mithal said.
“Companies should have their own kind of risk tolerances, and there may be certain areas where something is just a no-go, but there are some areas where there can be some mitigations where you use it for one purpose and not another purpose,” said Mithal.
While not all risks are weighted the same, CIOs need to know when to stop further conversations with a vendor.
“As companies build AI solutions or capabilities, we should expect that not all will hit the mark,” Sharon Mandell, SVP and CIO at Juniper Networks, said in an email.
Enterprises waded deeper into the generative AI pool last year, and the majority of businesses aren’t thrilled with where they’ve ended up. Two-thirds of C-suite leaders were ambivalent or dissatisfied with their organization’s progress on generative AI in 2023, according to a Boston Consulting Group poll of more than 1,400 C-suite executives published this month.
The discontent isn’t stopping enterprises from pushing to deliver on generative AI aspirations this year. More than 4 in 5 executives plan to increase spending on AI and generative AI in 2024, according to BCG data.
“That’s why every company needs a reliable process for vetting AI vendors,” Mandell said.
CIOs that find solutions are not meeting expectations, should take a systematic and constructive approach to documenting the issue, communicating concerns with the vendor, reviewing contractual agreements and discussing options for customization, according to Mandell.
“Consider building legal protections into contracts like including special terms regarding solution performance or structuring your deals for an initial year with the intent to renew, which give you options should the solution not deliver the expected outcomes,” Mandell said.
