Michael Ekstrand

What, Why, and How?

⮴ Talks
Title slide for What, Why, and How

This is the resource page for my seminar talk at Radboud University on Sep. 12, 2026 entitled “What, Why, and How do We Retrieve?”.

Abstract

Recommender systems, search engines, and other modern information discovery and access technologies have broadened both access to information and the ability to disseminate new information. Every day, people use these tools to locate, collate, and understand information, and they in turn shape their users’ understanding of the information space, and through it the world and society they inhabit. Information discovery has profound possibilities for connecting people with information, products, culture, and people they wouldn’t encounter in other ways, but can also reproduce a range of harms including discrimination, negative or unnecessary stereotypes, information silos, and more.

In this talk, I discuss the goals of information retrieval, access, and discovery, and how those goals translate to specific measurement and evaluation designs. My central thesis is that effective, ethical, and pro-social technologies for information discovery must be grounded both theoretically and empirically in a clear understanding of system goals, and robust operationalizations of those goals into system and evaluation designs.

Slides

Papers

Sushobhan Parajuli, Samira Vaez Barenji, and Michael D. Ekstrand. 2026. On the Convergent Validity of Offline Evaluation Designs for Recommender Systems. To appear in Proceedings of the 20th ACM Conference on Recommender Systems (RecSys ’26), Sep 28–Oct 2, 2026. 8 pp.  DOI 10.1145/3773078.3831818. arXiv:2607.25097 [cs.IR]. Acceptance rate: 18%.

Michael D. Ekstrand, Aleksandra Sarcevic, Afsaneh Razi, Maria Soledad Pera, Robin Burke, and Katherine Landau Wright. 2026. Recommending With, Not For: Co-Designing Recommender Systems for Social Good. Transactions on Recommender Systems 5(1) (August 2026; online Aug 5, 2025), 3:1–24. Selected for presentation at RecSys 2026. DOI 10.1145/3759261. arXiv:2508.03792 [cs.HC]. Cited 19 times.

Fernando Diaz, Michael D. Ekstrand, and Bhaskar Mitra. 2025. Recall, Robustness, and Lexicographic Evaluation. Transactions on Recommender Systems 4(1) (July 2025; online Apr 3, 2025), 13:1–50. DOI 10.1145/3728373. arXiv:2302.11370 [cs.IR]. Cited 17 times. Cited 3 times.

Jacy Reese Anthis, Kristian Lum, Michael Ekstrand, Avi Feller, and Chenhao Tan. 2025. The Impossibility of Fair LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL 2025), Jul 27–Aug 1, 2025. Association for Computational Linguistics, pp. 105–120. DOI 10.18653/v1/2025.acl-long.5. arXiv:2406.03198 [cs.CL]. Acceptance rate: 20.3%. Cited 76 times (shared with HEAL24). Cited 15 times.

Michael D. Ekstrand, Lex Beattie, Maria Soledad Pera, and Henriette Cramer. 2024. Not Just Algorithms: Strategically Addressing Consumer Impacts in Information Retrieval. In Proceedings of the 46th European Conference on Information Retrieval (ECIR ’24, IR for Good track), Mar 24–28, 2024. Lecture Notes in Computer Science 14611:314–335. DOI 10.1007/978-3-031-56066-8_25. NSF PAR 10497110. Acceptance rate: 35.9%. Cited 22 times. Cited 6 times.

Michael D. Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2022. Fairness in Information Access Systems. Foundations and Trends® in Information Retrieval 16(1–2) (July 2022), 1–177. DOI 10.1561/1500000079. arXiv:2105.05779 [cs.IR]. NSF PAR 10347630. Impact factor: 8. Cited 311 times. Cited 108 times.

Michael D. Ekstrand, Ben Carterette, and Fernando Diaz. 2024. Distributionally-Informed Recommender System Evaluation. Transactions on Recommender Systems 2(1) (March 2024; online Aug 4, 2023), 6:1–27. DOI 10.1145/3613455. arXiv:2309.05892 [cs.IR]. NSF PAR 10461937. Cited 40 times. Cited 11 times.

Amifa Raj and Michael D. Ekstrand. 2022. Fire Dragon and Unicorn Princess: Gender Stereotypes and Children’s Products in Search Engine Responses. In SIGIR eCom ’22, Jul 15, 2022. 9 pp.  DOI 10.48550/arXiv.2206.13747. arXiv:2206.13747 [cs.IR]. Cited 17 times. Cited 6 times.

Unresolved citation facctrec-grids

Christine Pinney, Amifa Raj, Alex Hanna, and Michael D. Ekstrand. 2023. Much Ado About Gender: Current Practices and Future Recommendations for Appropriate Gender-Aware Information Access. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (CHIIR ’23), Mar 19, 2023. pp. 269–279. DOI 10.1145/3576840.3578316. arXiv:2301.04780 [cs.IR]. NSF PAR 10423693. Acceptance rate: 39.4%. Cited 39 times. Cited 15 times.

Michael D. Ekstrand and Maria Soledad Pera. 2022. Matching Consumer Fairness Objectives & Strategies for RecSys. Presented at the 5th FAccTrec Workshop on Responsible Recommendation at RecSys 2022 (peer-reviewed but not archived). arXiv:2209.02662 [cs.IR]. Cited 11 times. Cited 3 times.

Michael D. Ekstrand and Daniel Kluver. 2021. Exploring Author Gender in Book Rating and Recommendation. User Modeling and User-Adapted Interaction 31(3) (February 2021), 377–420. DOI 10.1007/s11257-020-09284-2. arXiv:1808.07586v2 [cs.IR]. NSF PAR 10218853. Impact factor: 4.412. Cited 246 times (shared with RecSys18). Cited 117 times (shared with RecSys18).

Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018), Feb 23, 2018. PMLR, Proceedings of Machine Learning Research 81:172–186. proceedings.mlr.press/v81/ekstrand18b.html. Acceptance rate: 24%. Cited 429 times. Cited 229 times.

Works Cited

  • Anderson, C. 2009. The long tail: why the future of business is selling less of more. Random House.
  • Belkin, Nicholas J, and Stephen E Robertson. 1976. “Some Ethical and Political Implications of Theoretical Research in Information Science.” In Proceedings of the ASIS Annual Meeting. https://www.researchgate.net/publication/255563562.
  • Burke, Robin, and Morgan Sylvester. 2024. “Post-Userist Recommender Systems: A Manifesto.” <arXiv:2410.11870>.
  • Dwork, Cynthia, and Christina Ilvento. 2019. “Fairness under Composition.” In 10th Innovations in Theoretical Computer Science Conference (ITCS 2019). doi:10.4230/LIPICS.ITCS.2019.33.
  • Green, Ben, and Salomé Viljoen. 2020. “Algorithmic Realism: Expanding the Boundaries of Algorithmic Thought.” FAT* ’20. doi:10.1145/3351095.3372840.
  • Green, Ben, and Yiling Chen. 2019. “Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments.” FAT* ’19. doi:10.1145/3287560.3287563.
  • Hill, William, Larry Stead, Mark Rosenstein, and George Furnas. 1995. “Recommending and Evaluating Choices in a Virtual Community of Use.” In CHI ’95: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 194–201. doi:10.1145/223904.223929.
  • Mehrotra, Rishabh, Ashton Anderson, Fernando Diaz, Amit Sharma, Hanna Wallach, and Emine Yilmaz. 2017. “Auditing Search Engines for Differential Satisfaction Across Demographics.” In Proceedings of the 26th International Conference on World Wide Web Companion, 626–33. doi:10.1145/3041021.3054197.
  • Sapiezynski, Piotr, Wesley Zeng, Ronald E Robertson, Alan Mislove, and Christo Wilson. 2019. “Quantifying the Impact of User Attention on Fair Group Representation in Ranked Lists.” Companion Proceedings of The 2019 World Wide Web Conference, WWW ’19 Companion, May 13, 553–62. https://doi.org/10.1145/3308560.3317595.
  • Selbst, Andrew D., Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. “Fairness and Abstraction in Sociotechnical Systems.” In Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. FAT* ’19. doi:10.1145/3287560.3287598.
  • Smith, Jessie J., Lex Beattie, and Henriette Cramer. 2023. “Scoping Fairness Objectives and Identifying Fairness Metrics for Recommender Systems: The Practitioners’ Perspective.” In Proceedings of the ACM Web Conference 2023, 3648–59. doi:10.1145/3543507.3583204.
  • Hanna Wallach, Meera Desai, A. Feder Cooper, et al. 2025. “Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge.” Proceedings of the 42nd International Conference on Machine Learning. https://proceedings.mlr.press/v267/wallach25a.html.
  • World Day of Peace message from Pope Francis.