Centering Knowledge Along the Responsible LLM Supply Chain: An Empirical Study & Multi-Stakeholder Taxonomy

Abstract

Framing LLMs as products of complex supply chains rather than monolithic entities facilitates the creation of nuanced approaches to developing responsible LLM systems. Existing HCI, psychology and organization management literature suggests that \textbf{knowledge} is a central construct when developing, deploying, and using technological systems that are embedded in multi-stakeholder supply chains. This literature advocates the need for a unifying knowledge taxonomy that can support the collective effort of diverse stakeholders in producing responsible systems. Answering this need, we empirically study knowledge needs along the \emph{LLM supply chain} by conducting semi-structured interviews with 71 LLM practitioners. We conceptualize LLM knowledge by developing a multi-dimensional taxonomy that characterizes the knowledge needed based on its abstraction, theme, and nature. We then illustrate how this taxonomy can be used to reframe, revise, and disambiguate research and policy works on responsible AI notions related to knowledge (viz., AI literacy, transparency, and explainability). We hope to inspire future work in the CSCW community by outlining research opportunities to support \emph{practitioners} in accessing, exploiting, and sharing LLM knowledge.