In this Stanford CS547 HCI seminar, the speaker presents a compelling argument for reframing responsible AI through the dual lenses of rigor and human agency. The talk begins by emphasizing that rigor in AI research and practice extends far beyond mere methodological correctness. Instead, rigor encompasses multiple facets including epistemic, normative, conceptual, methodological, reporting, and interpretative rigor. The speaker argues that responsible AI inherently demands this broader conception of rigor, as it involves careful choices about background knowledge, norms, theoretical constructs, methods, communication of findings, and interpretation of results. This expanded view helps improve the quality and reliability of AI work by making explicit the assumptions and decisions that shape research and development.
The first facet, epistemic rigor, concerns the background knowledge that informs AI research, highlighting the importance of making assumptions explicit and scrutinizing their validity. The speaker illustrates this with examples of flawed AI research that attempts to predict unobservable traits like criminality or political beliefs from facial images, which are based on debunked scientific assumptions. Normative rigor follows, focusing on the explicit articulation of the norms, values, and standards guiding AI work. For instance, the development of AI personas to simulate users raises normative concerns about representation and inclusion, which must be addressed transparently. Mechanisms such as positionality and ethical statements are recommended to clarify these normative influences.
Conceptual rigor involves clearly defining and justifying the theoretical constructs under investigation. The speaker uses the example of “hallucination” in language models, a term with varied and sometimes misleading interpretations, to demonstrate the need for conceptual clarity to avoid confusion and misrepresentation. Methodological rigor, often the most emphasized facet, relates to the appropriateness and justification of methods used to operationalize constructs. The talk highlights the importance of construct validity and the establishment of methodological standards, especially in high-risk domains. Reporting rigor addresses how research findings are communicated, advocating for transparent and detailed reporting practices such as pre-registration and disaggregated metrics to avoid misleading conclusions.
Interpretative rigor concerns the careful deliberation involved in moving from research findings to claims, whether descriptive or normative. The speaker illustrates this with an example of AI systems performing well on mathematical reasoning benchmarks, showing how claims about human-level reasoning require careful consideration of background assumptions, conceptualizations, and methodological validity. Transparency about AI artifacts like datasets and models is crucial to support interpretative rigor. The speaker concludes the first half by asserting that responsible AI encompasses all these facets of rigor and that embracing this broader understanding can significantly enhance AI research and practice, though rigor alone is not a cure-all.
The second half of the talk shifts focus to human agency as a foundational principle for responsible AI. Human agency encompasses autonomy, freedom, self-determination, privacy, authenticity, and dignity, and foregrounding it helps clarify why responsible AI matters. The speaker discusses anthropomorphic AI systems, which are designed to appear humanlike and thus raise unique ethical concerns such as emotional dependence, consent, and misrepresentation. The talk highlights the complexity of intervening in anthropomorphic behaviors and the risks of uncritical approaches. By centering human agency, AI development can better support human needs and values, ensuring that AI systems empower rather than undermine people. The seminar closes with a brief Q&A addressing practical adoption and measurement challenges in responsible AI, particularly in sensitive applications like AI companions for vulnerable populations.
