These are the audit targets mentioned in the book Auditing AI. We have included the key question(s) addressed by the audit and a page number where the reference appears. This table is listed in order of first appearance.
| system | question | see |
|---|---|---|
| LYFT (Life-Years From Transplant) kidney allocation |
Is kidney allocation discriminatory? | pp. 1-3 |
| SABRE (Semi-Automated Business Research Environment) airline reservation |
Do search results secretly favor one company? | pp. 4-9 |
| Facebook social media advertising |
Does the ad placement system violate the Civil Rights Act? | pp. 9-11, 111-112 |
| COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) recidivism risk assessment |
Are risk estimates biased toward or against Black and/or White defendants? | pp. 22-24 |
| PredPol (Predictive Policing) law enforcement |
How accurate are crime predictions? Do accurate predictions lower crime? | pp. 25-26, 114 |
| Google image search |
Do search results promote inaccurate, exaggerated stereotypes? | pp. 28-29 |
| Spotify music recommendation |
Are independent artists being suppressed in recommendations? | p. 31 |
| Google Gemini generative AI |
Are efforts to ensure diverse representation producing historical inaccuracies? | pp. 32-34 |
| Twitter automatic image cropping |
Are Black faces being automatically removed? | pp. 37-38, 45, 142-144 |
| Amazon automated resume evaluation |
Are resume scans biased against women applicants? | pp. 38, 88-89 |
| Airbnb short-term rentals |
Does the platform faciliate racial discrimination? | p. 39 |
| IBM, Microsoft, and Face++ face detection/recognition |
How well do these systems detect gender? | pp. 41-43 |
| FACE Watch Plus / FACE Plus face recognition |
How accurate is facial recognition? | pp. 43, 45 |
| Facebook advertising marketplace |
Is the bidding for ad placement fair to advertisers? | pp. 45-47 |
| Google advertising placement |
Do ads on Google person search discriminate against Black people? | pp. 48-50 |
| Orbitz travel booking |
Are Mac users offered higher prices? | p. 53 |
| ChatGPT automated resume evaluation |
When used for hiring, does ChatGPT exhibit racial or gender bias? | pp. 58-61 |
| Facebook content moderation |
Are policies about misinformation being enforced correctly in political advertising? | pp. 63-64 |
| Twitter censorship / bias |
Are some popular topics being removed from the “trending” list? | p. 68 |
| Pymetrics automated job candidate evaluation |
Is the automated hiring assessment fair? | pp. 74, 86, 92-96, 112-113 |
| SmartRecruiters SmartAssistant automated hiring evaluation |
Is the automated hiring assessment fair? | pp. 96-100 |
| w2vNEWS machine translation |
Are gender stereotypes encouraged during processes like automatic translation? | p. 105 |
| X (formerly Twitter) censorship / bias |
Do search results accurately represent popularity? | pp. 107-109 |
| Google image tagging |
Are images of Jewish people and Black people being tagged with offensive words? | pp. 109-110 |
| OpenCV and others face detection |
Can face detection be easily tricked? | pp. 114-116 |
| US Immigration and Customs Enforcement (ICE) Risk Classification Assessment law enforcement |
Does the tool promote or obscure unconstitutional detention? | p. 121 |
| Medicaid benefits fraud detection |
Are people with developmental disabilities being illegally cut from the program? | pp. 125-126 |
| Apple credit cards |
Are women being illegally offered lower credit limits than men? Are they disproportionately denied credit? | pp. 126-127 |
| data brokers (several) privacy |
Are data transparency and disclosure requirements being followed by companies that hold personal information? | pp. 127-129 |
| National Eating Disorders Association (NEDA) AI Mental Health Chatbot mental health / LLM chatbots |
Does the chatbot offer dangerous advice? | pp. 131-132 |
| Yelp restaurant recommendation |
Is the star rating system favoring some businesses over others? | p. 133 |
| US Military SKYNET Drone Strike Targeting System warfare |
Is automated target identification likely to wrongly identify and kill civilians? | pp. 138-139 |
| Israeli Defense Force (IDF) Lavender Target Generation System warfare |
What civilian casualty rate does the system judge to be acceptable? | p. 139 |
| Donald Knuth (author) typographical errors |
What are the errors in my published books? | pp. 140-141 |
| Zoom videoconferencing |
Are Black faces less likely to be detected? | p. 142 |
| Anthropic, Google, Hugging Face, NVIDIA, OpenAI, and Stability LLM chatbots |
Are responses dangerous? Are they true? Can safeguards be easily defeated? Can private information leak? | pp. 144-145 |
| Facebook content moderation |
Is content that attempts to incite violence linked to actual violence or ethnic cleansing? Are rules prohibiting such content being enforced? | p. 146 |
| WhatsApp generative AI |
Does sticker generation produce offensive imagery? | pp. 155-156 |
| SmartCheck public benefits fraud detection |
Are certain vulnerable groups (single-mothers, migrants) more likely to be flagged? | pp. 157-158 |
| Google search engine |
Does filtering out gratuitous violence wrongly suppress historical or evidentiary material? | p. 159 |