How to appraise Artificial Intelligence (AI) and why context is key.

The year 2022 marked a significant milestone in Artificial Intelligence (AI). ChatGPT released an AI hurricane on the world, accelerating the AI arms race between technology giants like Microsoft and Google. Next to the enormous potential AI can bring, people, governments, and companies are increasingly concerned [1] regarding the potential (mis)use of AI which may harm persons' civil liberties, rights, and physical or psychological safety. Such (mis)use can hurt group discrimination, democratic participation, or educational access.

The Dutch Security Intelligence Agency (AIVD) expresses similar concerns regarding the potential risks associated with AI technology. In a recent report, the AIVD calls for extra attention to be paid to the development of AI technology, warning that the rapid advancement of AI could pose a severe threat. [2]. Italian regulators even went as far as to ban ChatGPT. [3] Microsoft has launched initiatives to promote and standardize responsible AI amidst the growing concerns over AI ethics [4]. Can we rely on Big Tech guidance in an area with such a significant commercial interest? 

Brecker et al. stated, “Policymakers need to provide guidelines as every day without clearer regulations means uncertainty for market participants and potentially ethically and legally undesirable AI solutions for society.” They referred to earlier concerns  about “un-black boxing” BigTech. [5] Un-blacking boxing is needed to enable fact-based and transparent decision-making for managers [6]. 

“Regulators and policymakers cannot take a “wait-and-see” approach as AI systems are already used or are being developed for their imminent deployment in sensitive areas such as healthcare, social robotics, finance, policing, the judicial system, human resources, etc.” [7]. 

How can we grasp the fundamental underlying concerns of AI more collectively? How can we allow AI actors to leverage their expertise in appraising AI technology and its context to improve decision-making? Let us first look at the main challenges before detailing our suggested approach and your actions.

"Policymakers cannot take a wait-and-see approach.”

What are the main challenges with AI?

In early 2023 the Karlsruhe Institute of Technology [8] stated:

“… Artificial Intelligence (AI) assessment to mitigate risks arising from biased, unreliable, or regulatory non-compliant systems remains an open challenge for researchers, policymakers, and organizations across industries. Due to the scattered nature of research on AI across disciplines, there is a lack of overview on the challenges that need to be overcome to move AI assessment forward.” 

The research at Karlsruhe University highlights seven areas of concern. Their work revolves around these seven areas of concern:

1. Assessing AI for ethical compliance
2. Assessing AI’s Future Impact on Society
3. Adequacy of existing law for AI assessments
4. Dealing with Conflicting Laws on Trade Secrets
5. AI-based Systems’ Explainability
6. AI-based Systems’ Transparency
7. AI-based Systems’ Impermanence

Regarding the first concern, ethical compliance, a great example manifested when Amazon created an AI recruitment tool that favored male candidates for technical roles. It wasn’t that the AI model was directly biased toward males. However, because the training data contained historical patterns of male-dominated technical roles, the AI system learned to associate specific colleges with successful candidates, inadvertently disadvantaging female applicants. [9]

This example underscores the importance of ethical compliance and addressing potential biases in AI systems. The example highlights the necessity for transparent and explainable AI — without it, you can’t validate the decisions. Furthermore, do we even want AI to make such decisions? What about privacy?

A milestone in AI risk management

2023 is a milestone in risk management with the publication of the National Institute of Standards and Technology's (NIST) standard for AI — a welcomed addition to their existing risk management standard. The framework initiative launched in July 2021 and was improved by approximately 400 comments from 240 global organizations over the past years. The AI framework was released just before the enormous AI storm. The AI NIST framework addresses many of the earlier raised concerns by academics. Academics was concerned about the lack of empirical validation of these frameworks. Something NIST has done via its extensive foothold in several industries and AI practitioners. This is why we, as authors, consider the NIST AI framework a significant step forward and use it in our research and practices.

In this article, we summarize the critical points to further your understanding of the topic, empowering you as a reader to apply the AI appraisal method and framework yourself. We offer high-level guidance via a. the framework and b. a practical approach through GSS for conducting the assessment to address the main seven concerns.

AI risk management framework

One of the most critical outcomes of risk management is to treat those risks most relevant to your organization, considering your specific context. Since every organization is different, defining the context seems logical but needs to be done more explicitly.  As Long [10] points out in his book Fallibility and Risk, Living with Uncertainty: “… not identifying the assessment context and requirements and delivering functional risk identification and risk expression language that decision-makers can understand will contribute to an increased risk”. Earlier studies at AMS [11], [12] in other industries, like Industrial Control Systems, Cybersecurity, and Smart Grids, emphasize this context refinement as a must-do in any risk assessment method [13].

The NIST framework helps frame risk, paving the path towards using the AI system and maximizing its associated opportunities. The AI framework allows identifying and measuring AI risks, determining the risk tolerance, performing risk prioritization, and integrating the outcomes into the organization's management system (if applicable). The NIST publication outlines seven essential characteristics of AI risk, providing a framework to shed light on the dangers of using and deploying AI. [14]

1. Valid and reliable
2. Safe
3. Secure and resilient
4. Accountable and transparent
5. Explainable and interpretable
6. Privacy-enhanced
7. Fair with harmful bias managed

Each of these seven characteristics deserve balanced care and attention. To discover what one of these characteristics entails, we zoom in on “secure and resilience.”

NIST outlines several common security concerns, such as data poisoning and the exfiltration of models, training data, or other intellectual property through AI system endpoints. Microsoft went even further by distilling more than 200 research papers on AI security into a comprehensive list of 17 threats. [15]

A subset of the AI threats outlined by Microsoft

Most notable is that some threats Microsoft outlined differ from traditional cyber risks. These new threats need particular attention. We see these threats also applicable in the context of ChatGPT. [15]

Threat number 3, model inversion, was found in ChatGPT. Model inversion is a threat to using AI tools outside their intended use, the so-called “secret features” of the AI. [15]

You can access ChatGPT’s secret features by carefully crafting ChatGPT prompts. For example, you could activate ChatGPT’s evil twin: DAN. Do Anything Now (DAN) ignores all the ethical limitations implemented by OpenAI. DAN has extreme opinions on sensitive topics and can swear and insult people, while these actions are restricted under normal use conditions.

Figure 1 ChatGPT’s Evil Twin: DAN [14]

Such attacks are referred to as “ChatGPT jailbreaking.” Jailbreaking aims to remove the restrictions, thus bypassing the intended use of the AI model. Another one of these alter egos is called Ranti. Ranti even provides users access to the generation of malicious code.

Figure 2 ChatGPT’s evil twin Ranti can write malicious code [15]

A call for “thoughtful” use and creation of AI

AI systems can be prone to biases, errors, and other inaccuracies. These issues can have serious consequences, particularly in high-risk environments. By assessing these risks, organizations can identify potential biases and risks in their AI systems and take steps to correct them. Apart from biases, AI systems are also vulnerable to cyber-attacks. These attacks can compromise data integrity and put sensitive information at risk.

AI technology makes decisions with significant social and ethical implications, such as decisions related to employment, finance, and criminal justice, and is expected to gain importance in decision-making. AI assessments should be used to ensure that the AI systems are fair and make ethical decisions that do not perpetuate existing inequalities or biases. Secondly, the evaluation should investigate the risk of potential misuse and hacking.

An AI assessment within the NIST framework views the AI system from the seven angles mentioned earlier. Each angle provides a different perspective on the risks of using or building an AI system. In the screenshot below, you can see these seven perspectives placed into the agenda of the collective assessment instrument based on Group Support Systems.

Figure 3 Screenshot of the AI assessment created in Meetingwizard GSS that looks at the seven angles of potential AI risk.

The use of Group Support

In a standard setting, a risk assessment on an AI system would be done by one person. Or a separate group of people. All thought this would get the job done; this approach can be risky for biased results. AI systems can have a wide range of potential impacts on different groups of people. It is best to involve multiple stakeholders in helping identify and understand potential risks and the consequences that may not be apparent to a single person.

A GSS-based assessment method focuses on getting all AI actors' collective views and opinions. These AI actors vary in the AI lifetime but share a joint responsibility of managing the risks of AI. This group can reflect a diversity of experience, backgrounds, and expertise. Using a GSS-based assessment, you remove the risk of bias by collectively doing the evaluation together.

A self-assessment or even a one-on-one assessment has enormous limitations. It is risky due to a range of cognitive biases, including confirmation bias, individual biases, pretending your situation is better than it (f)actually is, or even lying about the condition or lack of interaction and discussion between stakeholders that have another opinion or (outside in) view.

There are other methods to gain stakeholders' perspectives, like surveys, interviews, and focus groups. However, only GSS can encourage all participants to share their opinions without fear of judgment (anonymous participation), and organizational rank and reduce social desirability bias. Additionally, GSS employs structured argumentations for decision-making to ensure that all viewpoints are considered, minimizing bias from dominant or vocal participants.

Conducting the AI risk assessment can be done throughout the entire AI lifecycle. This is needed since usage, technology, and regulatory requirements rapidly change.

The group's collective brain is mobilized via GSS to establish a more sincere view of the current situation while simultaneously developing a feasible theory of the desired situation. The outcome of such a joint exercise is a transparent action plan.

Figure 4 Assessing an AI system on “Secure and Resilient” using Meetingwizard on the participant’s mobile phone

Academic research has proven that conducting a collective GSS-based assessment contributes in:

1. An improved discussion generates unique and valid viewpoints and ideas, leading to a representative view of the situation. The significant variance between participants is discussed and (re)voted (if needed) to gain consensus.
2. The group scrutiny leads to a more qualitative view and understanding, resulting in knowledge transfers and an increase in awareness (for example, about accountability and responsibility)
3. collectively committing to a goal, roadmap, and (actions)ownership leads to a more sustainable implementation of the AI strategy

This objectivity, validity, sincerity, and collectively become increasingly relevant when judging and understanding an Ai technology algorithm. GSS allows you to get all actors and expert opinions at the table and scrutinize the latter via discussion, elaboration of issues, discuss social or ethical barriers, and from a technology perspective to understand how the AI configuration or AI parameters operate. It also helps to determine if you want to develop this in-house or outsource the development of the AI application. Next, HR staff can determine where the maintenance and technical capabilities must be allocated. In other words, are we recruiting on this or outsourcing it?

Conclusion

Xia et al. (2023), who examined 16+ AI frameworks, stated, “… these frameworks do not provide clear guidance on extending or adapting them to fit diverse contexts. This limitation restricts the effectiveness of RAI risk assessment frameworks as the risks and mitigation may vary depending on the context (e.g., different organizations, sectors, or regions) in which AI systems are used.” [16]. The AI assessment framework provided by NIST and applied via a Group Support System methodology foresees just that: an assessment of the context and required capabilities of the organization. This AI assessment method also guides the implementation of treatment plans to mitigate a particular risk or leverage a specific opportunity. It also allows organizational decision-makers to develop an AI (exit) strategy.

Bibliography

Vincent van Dijk is Executive Master at AMS and Cybersecurity entrepreneur. 

Prof. Dr. Yuri Bobbert is Academic Director at AMS and supervised Vincent during his research project.