Content Detail

• Content META

  Title	Augmenting Third-Party Risk Management with Enhanced Due Diligence for AI
  ID	6005171
  External ID	7476888106
  Author	Unknown
  Published Date	Jan. 17, 2025, 11:12 p.m.
  Summary	Since the June 2023 release of U.S. interagency guidance (IAG) on third-party risk management , financial institutions have revisited their TPRM programs' structure, methodologies and tools. Much of t
  Tags	None
  URL	https://www.garp.org/risk-intelligence/operational/augmenting-third-party-250117
  Sentiment	Negative: 0.01543    Neutral: 0.90910    Positive: 0.07547
  Relevance	0.01498

• Content Details

  Content ID	3668598
  Key	cdb8404c-83f1-4efb-af17-a02803d28c65
  Content	Since the June 2023 release of U.S. interagency guidance (IAG) on third-party risk management , financial institutions have revisited their TPRM programs' structure, methodologies and tools. Much of the recent refinement effort we observe has focused on enhanced due diligence (EDD) processes that add subject-matter-expert depth across such key risk types prevalent in TPRM as cybersecurity, business continuity and disaster recovery, and regulatory compliance. Fewer institutions have augmented their EDD set to include a module for risks related to artificial intelligence. Here we look at key considerations for adding an EDD module for vendors providing AI systems through each of the five TPRM lifecycle stages defined in the IAG: planning, due diligence and third-party selection, contract negotiation, ongoing monitoring, and termination. Planning Two key controls are involved in the planning step. First, there should be a control governing whether any EDD processes should be invoked during onboarding which requires augmentation for AI systems. The decision to invoke EDD for AI should apply to any related technology, including machine learning (ML), deep learning (DL), natural language processing (NLP) and large language models (LLMs), that support generative AI models. In-scope types should be enumerated in policy and periodically reassessed. The next step is an independent risk/benefit analysis of the decision to pursue a vendor solution that incorporates AI. A rationale to believe that it will be reliable for the use case must be assessed, and the appropriate role the AI system will play in the business process must be considered up front. For example: will the AI system be entrusted to drive outcomes, or only provide a signal to a human decision-maker? If the latter, where is the human in the loop? Does its use align with strategy, risk appetite, and broader corporate policies? Additionally, organizational impact must be considered during planning. How will the company perform risk oversight for this third party, and does the talent needed exist in-house? Does the company have the bandwidth for augmented activities across the lifecycle? Paul Feldman The vendor (or the company) must have a viable, representative and fair set of training data, i.e., the data used to train and calibrate the AI system, drawn from actual transactions. Data should be free of historical bias and discrimination, representative, and adequately varied across the range of products, services, customer types, channels and jurisdictions covered by the business(es) the model will serve. Independent challenge must look at whether datasets were deliberately curated to be unrepresentative of a population that will lead toward a biased outcome, or if runtime prompts could have a similar nefarious effect (“prompt engineering”). Data privacy controls must protect against the risk of confidential data (personally identifiable information [PII], intellectual property [IP], copyrighted material, etc.) used for training being inadvertently revealed in later AI system outputs. Due Diligence and Third-Party Selection Model validation must either be undertaken in-house or properly evaluated if performed by a third party. These validation areas require additional effort for an AI system: Conceptual soundness: identify and assess bias and discrimination risks; clustering and uniformity risks; confabulation/virtual input risk (a.k.a. hallucination risk); and explainability (or black box) risks. Also look for cross-risk exposures to traditional risk types, including regulatory compliance risk, fraud/conflict-of-interest risk, data privacy risk and reputational risks. Outcomes analysis: traditional model validation in this step focuses on backtesting, which may be feasible in some AI cases. An internal study may be needed to review model outputs for the risks enumerated above. Explainability of outcomes must be adequate, particularly involving cases where a regulatory requirement around explainability exists (e.g., for loan denials). Ongoing monitoring: covered in the TPRM ongoing monitoring phase (see below). Other proposed elements enumerated in the IAG's due diligence section may also require heightened scrutiny in the EDD, including information security, management of information systems, reliance on subcontractors, contractual arrangements with other parties, and other clauses on a case-by-case basis. Contract Negotiation Within the core terms and conditions of contracts, the EDD for AI may require a review of performance measures and benchmarks and indemnification and limits on liability. Tilcia Toledo The IAG's proposed clauses concerning remote oversight rights might also require review in some cases: responsibilities for providing, receiving and retaining information and the right to audit and require remediation. Even in cases where software is provided as-is, and the bulk of the review happens during initial due diligence, these rights can still be important if a significant issue arises later in ongoing monitoring. Clauses from the guidance regarding risk management capabilities generally requiring a review in the EDD for AI include: responsibility for compliance with applicable laws and regulations and confidentiality and integrity. If a contractual clause regarding customer complaints is included, the EDD may require contextual language. The third party's capabilities regarding regulatory supervision are absolutely of interest, and periodic updates on any discussions with regulators should be provided in due course. Ongoing Monitoring Production outputs must be regularly checked against reliable benchmark(s): numerical values where feasible, but human expert judgment at a minimum. Specific tests and benchmarks, monitoring frequency, and timely response plans for deviations should be established. Moreover, issues identified by employees and customer complaints must be regularly evaluated. Key risk indicators (KRIs) should be identified, such as the count of instances requiring human reversal of model output, count of instances of examiner criticism, critical system uptime, etc. Controls should help prepare for examiner reviews by requiring documented proof of the due diligence, contract negotiation, and ongoing monitoring steps, and require the TPRM process owner(s) to describe and defend the overall methodology. Model validation activities are usually a key focus area for examiners. Termination In the termination phase, a key consideration for the EDD for AI process is around use of data provided by third parties to train models. If the data provider's contract includes a requirement that all data must be returned if the contract is terminated, then the company should protect an already-trained system using that data as the property of the company. The data provider might argue that their data is still embedded in some refracted way within the trained AI system. A stressed exit of a relationship with an AI-providing vendor could be risky and costly. Will it be feasible to identify an alternate AI system that will be able to step into the shoes of the exiting provider, or will the activity have to come in-house? If another system is viable, switching costs may be substantially higher than with other types of software. Parting Thoughts Developing a thorough understanding of the range of idiosyncratic risks resulting from the use of AI is an essential prerequisite to building out an effective EDD process to protect the organization. Both banks and non-financial institution organizations that use AI tools provided by third parties would benefit from heightened coverage of related risks in their TPRM programs. Those that already have EDD processes related to technology, model validation, cybersecurity or information security will have a leg up in designing an additional EDD process for AI risks. Some firms may choose to integrate AI directly into model validation EDD. Paul Feldman is a Senior Director at FTI Consulting . He has more than 20 years of experience in implementing and validating analytics-driven solutions to problems facing financial institutions involving risk, finance, capital, strategy and regulatory compliance. Tilcia Toledo , a Senior Managing Director at FTI Consulting, has over 25 years of combined experience in leverage finance, regulatory compliance and consulting. Previously, she worked for JPMorgan, BNP Paribas, Scotiabank and the FDIC. She advises boards of directors on risk, corporate governance and compliance. She also leads the delivery of financial crime compliance risk assessments and has served as risk management subject matter expert on multiple monitorships. The views expressed in the article above are those of the authors and not necessarily of FTI Consulting Inc., its management, its subsidiaries, its affiliates or its other professionals.
  Content Reference URL	None

• Publisher

  Publisher Name	GARP (#11054)
  Publisher URL	garp.org
  Description	None
  Trust Flow Score	24
  Citation Flow Score	46

• Data Source

  Data Source Name	Quantexa News (#5)
  Data Source Type	news
  Data Source URL	https://app.aylien.com/dashboard
  Description	Welcome to Quantexa News API — the most powerful way to aggregate, filter, and integrate global news data into your apps and models.
  Weight	10