The most significant failures may not arise from technical problems alone, but from adopting AI without clear responsibilities, controls and decision boundaries.
Artificial intelligence has rapidly moved from experimentation into everyday business, supporting everything from data analysis and report drafting to customer communication, recruitment and routine decision-making.
As adoption grows, the conversation is shifting from what AI can do to who is accountable when AI-driven decisions have unintended consequences. Governance often lags behind, raising urgent questions about oversight, accountability and management’s role in controlling AI-related risks.
Norbert Kovács spoke with Pawel Piwowarski, Regional D&O Practice Leader at Colonnade Insurance, to discuss how AI governance may signal organisational maturity and influence assessments by insurers and other stakeholders.
The Governance Gap
Kovács: AI is moving from supporting decisions to shaping them. What change does that create in the corporate risk landscape?
Piwowarski: It has transformed decision-making risk. Technology traditionally supported business decisions; AI increasingly influences or shapes them directly, creating exposures involving inaccurate outputs, bias, regulatory non-compliance, intellectual property and operational decisions based on AI-generated recommendations.
That makes AI risk an enterprise-wide governance issue, not merely an IT concern. Yet many organisations are adopting AI faster than they’re building the structures to govern it. Business value is immediately visible; accountability, oversight and control often follow later.
Kovács: Does that mean the greatest risk lies in the technology itself, or in the organisation’s ability to oversee it?
Piwowarski: The most significant failures may not arise from technical problems alone, but from adopting AI without clear responsibilities, controls and decision boundaries.
A system may function as designed yet produce a poor outcome if it is used for the wrong purpose, developed, configured or operated using unsuitable data, or trusted without sufficient human review. AI risk increasingly becomes a question of management quality: does the organisation know where AI is used, which decisions it influences and when human judgment must prevail?
Learning from Cyber Before the Crisis
Kovács: Cyber risk became a board concern only after costly disruption. What can companies learn from that experience before AI follows the same path?
Piwowarski: Cybersecurity was once treated predominantly as a technical issue. Major incidents exposed its legal, financial, operational and reputational consequences, forcing boards to assume greater responsibility.
AI may follow a similar trajectory, but companies can act earlier. They need not wait for major losses or regulatory intervention before treating AI oversight as a strategic responsibility.
As AI becomes embedded in operations, senior management must show that controls are proportionate to the decisions involved. A low-risk administrative tool should not be governed like an AI system influencing recruitment, customer advice or financial decisions.
From Human Oversight to Meaningful Control
Kovács: As AI recommendations become more persuasive, how should directors preserve independent judgment and where should accountability ultimately sit?
Piwowarski: Accountability should sit with people. AI can analyse data, recognise patterns and recommend actions, but it should remain a tool, not the decision-maker.
Yet saying that a human remains responsible is not enough. Oversight must be meaningful. If someone routinely approves an AI recommendation without understanding its basis, challenging its assumptions or recognising its limitations, human involvement becomes little more than a formality.
Using AI does not remove directors’ duties, management responsibilities or the need for independent judgment and reasonable oversight. The key test may be not whether an AI system made a mistake, but whether the organisation maintained a reasonable, documented system of oversight proportionate to the risk.
Shadow AI Reveals the Limits of Traditional Governance
Kovács: Formal governance may cover approved systems, while everyday experimentation remains invisible. How serious is the Shadow AI gap?
Piwowarski: It is one of the most immediate AI risks because organisations cannot govern what they cannot see.
Employees may use external tools to draft contracts, analyse internal documents, prepare presentations or develop professional opinions, outside established security, legal, data-protection and management controls.
But the deeper issue is not simply unauthorised technology. Shadow AI may show that formal governance lags behind how people work. Prohibition can push usage further out of sight without removing demand.
Organisations therefore need visibility into actual AI use, not merely a policy on approved use. An organisation-wide inventory of applications and use cases could show which systems are used, what data they process and which decisions they influence.
AI Governance as a Measure of Organisational Quality
Kovács: Could AI governance eventually become an indicator of management quality rather than simply a compliance requirement?
Piwowarski: Potentially, yes. Stakeholders may increasingly ask not whether an organisation uses AI, but whether it can demonstrate appropriate governance.
They may expect companies to know where AI is used, what risks each application creates, which decisions require human review and how incidents are escalated. A documented framework, an AI inventory and clear responsibilities could become evidence of broader organisational maturity.
The distinction will be between companies that merely have an AI policy and those that can show how governance works in practice.
The Next Underwriting Question
Kovács: How could this influence insurance underwriting?
Piwowarski: Insurers may increasingly assess AI governance maturity, much as cyber insurers now examine controls, incident management and governance frameworks.
Relevant evidence could include board and executive oversight, AI inventories, risk assessments for significant use cases, human review of material decisions, incident escalation, employee training and regulatory monitoring.
AI governance may therefore become an underwriting factor. Companies demonstrating mature, proportionate and effective oversight may present a different risk profile from those unable to explain how AI is used across their operations.
The future underwriting question may not be simply, “Does the company use AI?” but, “Can it demonstrate control over the decisions AI helps it make?”
Governance as an Advantage
The defining question for leaders is whether AI governance can keep pace with its influence. The strongest companies will treat oversight not as a constraint imposed after adoption, but as part of how they design, deploy and improve AI from the outset.
This demands more than policies. It requires leaders to make responsibility visible: knowing where AI affects decisions, setting clear boundaries and ensuring that people retain the authority and confidence to challenge its output.
The next competitive advantage may therefore come not from adopting AI fastest, but from earning trust in how it is used. Organisations that can combine experimentation with disciplined accountability will be better placed to innovate at scale and to defend the decisions that follow.
Some tasks can be delegated to AI. Responsibility cannot.
About Pawel Piwowarski
Pawel Piwowarski is Financial Lines Director at Colonnade Insurance’s Polish branch and Regional D&O Practice Leader. Working in insurance since 2000, he specialises in financial lines and Directors & Officers liability insurance.
About Colonnade Insurance
Colonnade Insurance S.A. is a Luxembourg based ‘A-‘ rating non-life insurer owned by Fairfax Financial Holdings. Colonnade offers products for consumer and corporate clients across Central and Eastern Europe with commercial and consumer insurance products. Colonnade has offices in Hungary, Poland, Czech Republic, Slovakia, Bulgaria, and Romania with more than 600 employees in the CEE regio

