Glossary

The definitive glossary of terms related to auditing AI.

AI	The capacity of computers to exhibit or simulate intelligent behavior in the broadest sense, or the systems that produce such behavior. Also called an AI system.
AI audit	A rigorous and systematic evaluation of the core claims, expectations, or standards pertaining to an AI system. Also called an algorithm audit.
algorithm	A procedure used by a computer to solve a problem, or, more generally, a computer system as a whole that is employing such a procedure.
auditor	The person or group conducting an AI audit.
audit study	As distinguished from a financial or tax audit, in social science, changing the inputs to a situation (e.g., by employing different testers) in order to evaluate any resulting changes in the outputs. For computing, see AI audit.
audit-washing	Using an ineffective AI audit to provide a system with the appearance of accountability and respectability, without the substance. Used with negative connotations.
bias	To influence or distort a system away from an expected, widely agreed-upon, neutral, ideal, natural, or objective behavior. Also the amount of such influence.
bias bounty	A reward offered, often to the public, for the discovery of a previously unknown bias. Cf. bug bounty.
black box	Any component of a system whose internal mechanism may not be readily known. Cf. glass box.
bug bounty	A reward offered, often to the public, for identifying errors in software. Usually intended to detect security vulnerabilities. Cf. bias bounty.
correspondence audit	A social science audit study that was once accomplished by sending letters in the mail (historically, cover letters and resumes). More broadly, shorthand for any audit study.
Crandall’s Complaint	No reasonable person should expect a system to be unbiased. Named after the former CEO of American Airlines who defended the SABRE airline reservation system by admitting it was intentionally biased.
four-fifths rule	In employment law, if the selection rate for one group is less than 4/5 (or 80 percent) of the rate for the largest group, this can be taken as evidence of a substantial difference, or of discrimination. This rule was developed as a simple, practical test, but it is also arbitrary.
generative AI	A machine learning system focusing on pattern creation that is designed to produce output that has never existed before, including text, images, audio, and/or manufactured data.
glass box	Any component of a system whose internal mechanism is intentionally revealed. Also called a white box. Cf. black box.
ground truth	In machine learning and AI, data chosen to represent the standard that the system is tested against, or the data defined as true for the purpose of developing the system. These data are unlikely to be actually true, but they may be the best data available.
harm	Damage, injury, or loss suffered by a person, group, system, company, institution, or object. Also, to do harm.
impact assessment	A formal procedure that assesses the social, environmental, and/or economic consequences of some action. May be required by law or as an industry standard. Also called an impact report.
large language model (LLM)	AI that uses machine learning for language generation (e.g., OpenAI’s GPT-4). Used in contemporary AI systems that interact with people using language.
machine learning	AI that infers patterns in a set of data to develop, change, or adapt without following explicit instructions.
muffin–Chihuahua problem	A situation where an AI’s performance may differ substantially from a human’s, sometimes in a way that is not obvious. For example, a child sees the difference between a muffin and a Chihuahua, but an image recognition algorithm may not.
normal accidents	The sentiment that failures should be expected and may be unavoidable. Broadly, the position that planning for and mitigating failures is more useful than trying to design systems that will not fail. Named after sociologist Charles Perrow’s book of the same name.
redlining	Systematic, often intentional, racial discrimination. Refers to the historical practice in banking of literally drawing a red line on a map to indicate Black neighborhoods where loans, mortgages, or insurance would not be offered.
red team	From computer security, a group pretending to be an adversary, usually hostile hackers, as part of an exercise.
scraping	Using a computer program to copy a large amount of information, often information that is publicly available. As web scraping, downloading a large number of web pages automatically.
sexy construction worker problem	The situation where members of a group are underrepresented and they are also misrepresented. Named for the finding that AI-produced images of female construction workers are sexualized and unrealistic, while images of male construction workers are not.
sock puppet	A computer script or program masquerading as a person. In an AI audit, an automated tester.
source code	The most human-readable form of a computer program.
target	The person, system, or company being investigated by an audit study. Also called the auditee.
tester	Historically, a human investigator who pretends to want an apartment or a job in order to conduct an audit study of a landlord or employer for bias. Later audits used letters (correspondence studies) or sock puppets instead of humans.
training data	Inputs initially given to a machine learning system from which it can infer patterns, sometimes described as a set of examples. Distinct from data used for validation, testing, or any other purpose.