Learning Objectives

By the end of this lesson, the learner should be able to:

  1. Define AI governance and explain its importance in business analytics.
  2. Explain the relationship between AI governance, data governance and corporate governance.
  3. Identify major risks associated with artificial intelligence and analytical systems.
  4. Explain key privacy principles relevant to business analytics.
  5. Describe methods for identifying, assessing and mitigating analytical risks.
  6. Explain the importance of accountability, transparency and human oversight.
  7. Apply international AI and risk-management frameworks to business situations.
  8. Develop a basic AI and analytical risk-management approach.

1. Introduction to AI Governance

Artificial intelligence is increasingly being incorporated into business operations and decision-making. Organizations use AI and advanced analytics for forecasting, fraud detection, customer segmentation, credit assessment, recommendation systems, automation and strategic planning.

However, AI systems can create significant risks if they are poorly designed, incorrectly implemented or used outside their intended purpose.

AI governance refers to the policies, structures, processes, responsibilities and controls established to ensure that AI systems are developed, deployed, monitored and retired in a responsible and controlled manner.

AI governance seeks to balance:

Innovation + Business Value + Risk Management + Responsible Use

2. Importance of AI Governance

AI governance is important because analytical and AI systems can affect:

  • Customers.
  • Employees.
  • Suppliers.
  • Investors.
  • Regulators.
  • Communities.
  • Organizational reputation.

Poor governance can result in:

  • Privacy violations.
  • Discriminatory outcomes.
  • Incorrect decisions.
  • Security breaches.
  • Financial losses.
  • Regulatory violations.
  • Reputational damage.
  • Loss of stakeholder trust.

The OECD AI Principles emphasize human-centred values, fairness, transparency, robustness, security, safety and accountability throughout the AI lifecycle.

3. AI Governance and Data Governance

AI governance and data governance are closely connected but are not identical.

Data Governance

Focuses on:

  • Data ownership.
  • Data quality.
  • Data access.
  • Data security.
  • Data definitions.
  • Data lifecycle management.
  • Appropriate use of data.

AI Governance

Focuses on:

  • AI use cases.
  • AI system development.
  • AI risk.
  • Model validation.
  • Human oversight.
  • Transparency.
  • AI accountability.
  • Monitoring and continuous improvement.

Good AI governance therefore requires strong data governance.

4. AI Governance and Corporate Governance

AI governance should form part of the organization’s wider governance framework.

A simplified structure is:

Board and Executive Leadership

↓

Enterprise Governance

↓

AI and Data Governance

↓

Risk and Compliance

↓

Technology and Security

↓

Business Operations

This structure helps ensure that responsibility for AI does not remain exclusively within the IT or analytics department.

5. International AI Governance Frameworks

Important international frameworks include:

  • NIST AI Risk Management Framework (AI RMF).
  • ISO/IEC 42001:2023.
  • OECD AI Principles.
  • ISO/IEC 23894:2023.

The NIST AI RMF provides a voluntary framework for organizations designing, developing, deploying or using AI systems. Its four core functions are Govern, Map, Measure and Manage.

ISO/IEC 42001:2023 provides requirements for establishing, implementing, maintaining and continually improving an Artificial Intelligence Management System.

6. AI Risk

AI risk is the possibility that an AI system may produce harmful, incorrect, unfair, insecure or otherwise undesirable outcomes.

AI risks can arise from:

  • Data.
  • Models.
  • Algorithms.
  • People.
  • Processes.
  • Technology.
  • Security.
  • Business context.
  • External changes.

Risk management should therefore cover the entire AI lifecycle.

7. Major Categories of Analytical Risk

7.1 Data Risk

Problems associated with:

  • Poor-quality data.
  • Missing data.
  • Inaccurate data.
  • Outdated data.
  • Biased data.
  • Incomplete datasets.

7.2 Model Risk

Problems caused by:

  • Incorrect assumptions.
  • Poor model selection.
  • Overfitting.
  • Poor validation.
  • Inappropriate variables.
  • Model deterioration.

7.3 Privacy Risk

Risks associated with:

  • Unauthorized collection.
  • Excessive collection.
  • Improper sharing.
  • Re-identification.
  • Unauthorized access.
  • Inappropriate retention.

7.4 Security Risk

Includes:

  • Unauthorized access.
  • Data theft.
  • Model manipulation.
  • Data poisoning.
  • Adversarial attacks.
  • Credential compromise.

7.5 Ethical Risk

Includes:

  • Discrimination.
  • Unfair treatment.
  • Lack of transparency.
  • Excessive automation.
  • Harm to affected individuals.

7.6 Regulatory Risk

Organizations may face consequences when analytical systems fail to comply with applicable laws and regulatory requirements.

7.7 Reputational Risk

Poor analytical decisions can reduce stakeholder confidence in an organization.

8. Privacy in Business Analytics

Business analytics often involves personal, financial, behavioural or commercially sensitive information.

Privacy governance should consider:

  • Why the data is collected.
  • Whether collection is appropriate.
  • How the data will be used.
  • Who can access it.
  • How long it will be retained.
  • Whether it can be shared.
  • How it will be protected.

The OECD principles specifically recognize privacy and data protection as part of human-centred and trustworthy AI.

9. Data Minimization

Data minimization means collecting and processing only information that is reasonably necessary for the intended purpose.

For example, an organization developing a customer-service analytics dashboard should not automatically collect unrelated personal information simply because its systems can technically capture it.

Data minimization can reduce:

  • Privacy exposure.
  • Security exposure.
  • Storage requirements.
  • Governance complexity.

10. Purpose Limitation

Data should generally be used consistently with its legitimate and defined purpose, subject to applicable law and organizational requirements.

For example:

Data collected to process a customer transaction should not automatically be assumed to be appropriate for unrelated analytical purposes.

Business analysts should therefore understand the intended purpose of data before using it.

11. Privacy by Design

Privacy should be incorporated into analytical systems from the beginning.

Possible controls include:

  • Access restrictions.
  • Data minimization.
  • Encryption.
  • Pseudonymization.
  • Retention controls.
  • Audit trails.
  • Appropriate authentication.

Privacy should not be treated solely as a problem to solve after deployment.

12. Human Oversight

Human oversight is particularly important where analytical systems can significantly affect individuals or organizations.

Human oversight may involve:

  • Reviewing high-risk decisions.
  • Investigating unusual outputs.
  • Approving important actions.
  • Overriding inappropriate recommendations.
  • Monitoring system performance.

The OECD principles specifically emphasize human agency and oversight where appropriate.

13. Transparency and Explainability

Stakeholders may need meaningful information about:

  • What an AI system does.
  • What data it uses.
  • Its capabilities.
  • Its limitations.
  • How outputs are generated.
  • How decisions can be challenged.

Transparency is particularly important where AI outputs have significant consequences.

14. Model Validation

Before an important analytical model is deployed, it should be appropriately tested and validated.

Validation may examine:

  • Accuracy.
  • Robustness.
  • Generalization.
  • Bias.
  • Stability.
  • Security.
  • Suitability for the intended business purpose.

A model should not be judged solely by its technical accuracy.

15. Model Drift

Model performance can deteriorate when the environment changes.

Examples include:

  • Changes in customer behaviour.
  • Changes in economic conditions.
  • Changes in fraud patterns.
  • Changes in market conditions.
  • Changes in data distributions.

Organizations should therefore monitor important models after deployment.

16. Generative AI Risk

Generative AI introduces additional analytical risks, including:

  • Hallucinations.
  • Inaccurate information.
  • Confidential-data leakage.
  • Bias.
  • Intellectual-property concerns.
  • Prompt injection.
  • Misuse of generated content.

NIST’s Generative AI Profile provides additional guidance for identifying and managing risks associated with generative AI.

17. AI Risk Assessment

A business analyst can structure an AI risk assessment around the following questions:

  1. What is the purpose of the system?
  2. Who will be affected?
  3. What data will be used?
  4. What could go wrong?
  5. How likely is each risk?
  6. What could be the impact?
  7. What controls already exist?
  8. What additional controls are required?
  9. Who is responsible?
  10. How will performance and risk be monitored?

18. Risk Matrix

Organizations may classify risks using likelihood and impact.

Likelihood

Impact

General Risk

Low

Low

Low

Low

High

Medium

High

Low

Medium

High

High

High

Organizations may use more sophisticated risk scales depending on their needs.

19. Risk Mitigation

Common approaches include:

Avoid

Do not implement the proposed use case where risk is unacceptable.

Reduce

Introduce controls that lower likelihood or impact.

Transfer

Use appropriate contractual or other risk-sharing arrangements.

Accept

Accept the risk when it falls within approved organizational risk tolerance.

20. AI Risk Register

An AI risk register may contain:

Risk

Likelihood

Impact

Owner

Control

Biased data

Medium

High

Data Team

Bias assessment

Privacy breach

Low

High

Security Team

Access controls

Model drift

Medium

Medium

Analytics Team

Performance monitoring

Incorrect output

Medium

High

Business Owner

Human review

21. AI Governance Roles

Responsibilities may be distributed among:

Board / Executive Management

Provides strategic oversight.

AI Governance Committee

Coordinates AI policies and risk management.

Data Governance Team

Oversees data quality, access and appropriate use.

Analytics Team

Develops, validates and monitors analytical systems.

IT and Security

Protect infrastructure and information.

Legal and Compliance

Provides regulatory and legal guidance.

Business Owner

Remains accountable for the business purpose and outcomes.

22. Third-Party AI

Organizations increasingly obtain AI capabilities from external providers.

Before adopting such systems, organizations should consider:

  • Vendor security.
  • Data processing.
  • Privacy.
  • Contractual obligations.
  • Performance.
  • Reliability.
  • Transparency.
  • Regulatory requirements.
  • Exit arrangements.

Using a third-party system does not automatically eliminate organizational responsibility.

23. AI Procurement

Before acquiring an AI solution, decision-makers should ask:

  • What problem does it solve?
  • What data does it require?
  • Where is data processed?
  • How is information protected?
  • What evidence supports its performance?
  • What are its limitations?
  • How are updates managed?
  • How can the system be audited?
  • What happens if the provider becomes unavailable?

24. Continuous Monitoring

AI governance must continue after deployment.

Organizations should monitor:

  • Model performance.
  • Data quality.
  • Bias.
  • Security.
  • Privacy incidents.
  • User feedback.
  • Unexpected outcomes.
  • Changes in the operating environment.

NIST describes AI risk management as a continuing activity across the AI lifecycle rather than a one-time exercise.

25. AI Governance Lifecycle

A practical lifecycle is:

Identify Use Case

↓

Assess Risk

↓

Assess Data and Privacy

↓

Develop or Procure

↓

Test and Validate

↓

Approve

↓

Deploy

↓

Monitor

↓

Review

↓

Improve or Retire

26. Case Study: Credit Risk Analytics

A financial institution wants to use machine learning to assess credit risk.

Potential benefits include:

  • Faster decisions.
  • Consistent analysis.
  • Better risk identification.
  • Reduced processing costs.

Potential risks include:

  • Historical bias.
  • Privacy concerns.
  • Incorrect predictions.
  • Lack of transparency.
  • Model drift.

Appropriate controls could include:

  • Data-quality testing.
  • Bias testing.
  • Model validation.
  • Human oversight.
  • Documentation.
  • Continuous monitoring.

27. Role of the Business Analyst

The business analyst can support AI governance by:

  • Defining the business purpose.
  • Identifying stakeholders.
  • Documenting requirements.
  • Identifying risks.
  • Assessing data requirements.
  • Supporting validation.
  • Communicating limitations.
  • Monitoring business outcomes.

Lesson Summary

AI governance provides the structures necessary for responsible use of AI and advanced analytics.

The major principles include:

  • Accountability.
  • Transparency.
  • Privacy.
  • Security.
  • Human oversight.
  • Risk management.
  • Model validation.
  • Continuous monitoring.
  • Proportionality.
  • Responsible innovation.

The central principle is:

AI should be governed throughout its lifecycle so that organizations can obtain its benefits while managing risks to individuals, organizations and society.

References 

PreviousNext