Education
- Ph.D (2014)
- Computer Science, University of Minnesota.
- Advisers: John T. Riedl and Joseph A. Konstan
- B.S. (2007)
- Computer Engineering, Iowa State University.
Employment History
- 2023โpresent
- Assistant Professor, Dept. of Information Science, Drexel University
- PI/Lead, Impact, Novation, Effectiveness, and Responsibility of Technology for Information Access Lab (INERTIAL)
- 2022โ2023
- Associate Professor, Dept. of Computer Science, Boise State University
- Co-director, People and Information Research Team (PIReT)
- 2016โ2022
- Assistant Professor, Dept. of Computer Science, Boise State University
- Co-director, People and Information Research Team (PIReT)
- 2014โ2016
- Assistant Professor, Dept. of Computer Science, Texas State University
Students
Current Ph.D.ย Students
- Samira Vaez Barenji (expected 2029)
- Sushobhan Parajuli (expected 2029)
Ph.D.ย Graduates
- Ngozi Ihemelandu (Ph.D.ย 2024, Boise State University; dissertation: Best Practices for Offline Evaluation for Top-N Recommendation: Candidate Set Sampling and Statistical Inference)
- Amifa Raj (Ph.D.ย 2023, Boise State University; dissertation: Fair Layouts in Information Access Systems: Provider-Side Group Fairness in Ranking Beyond Ranked Lists)
M.S. Graduates
- Srabanti Guha (M.S. 2023, Boise State University; project: Explaining Misallocated Exposure across Multiple Rankings)
- Carlos Segura Cerna (M.S. 2020, Boise State University; project: Recommendation Server for LensKit)
- Mucun Tian (M.S. 2019, Boise State University; thesis: Estimating Error and Bias of Offline Recommender System Evaluation Results)
- Vaibhav Mahant (M.S. 2016, Texas State University; thesis: Improving Top-N Evaluation of Recommender Systems)
- Sushma Channamsetty (M.S. 2016, Texas State University; thesis: Recommender Response to User Profile Diversity and Popularity Bias)
- Mohammed Imran R Kazi (M.S. 2016, Texas State University; thesis: Exploring Potentially Discriminatory Biases in Book Recommendation)
- Shuvabrata Saha (M.S. 2016, Texas State University; co-advised with Dr.ย Apan Qasem; thesis: A Multi-objective Autotuning Framework For The Java Virtual Machine)
Undergraduate Student Research
I have supported and mentored the following undergraduate research students: Christine Pinney (BSU, UGRA + REU), Liana Shiroma (Colby Coll., REU 2021), Stephen Randall (U. Pitt, REU 2021), Connor Wood (BSU, REU 2020 + UGRA), Ananda Montoly (Smith Coll., REU 2020), Sandra Ambriz (BSU, HERC + UGRA).
Funding key:
- UGRA: undergraduate research assistant hired from research funds
- REU: Research Experience for Undergraduates
- HERC: Higher Education Research Consortium
Research Funding
External Grants /โ/โ2 /โ/โ$664K
- 2023โ2025: NSF 22-32553: Collaborative Research: CCRI: New: A Research News Recommender Infrastructure with Live Users for Algorithm and Interface Experimentation ($1.4M; Drexel PI, my share $150K; Lead PI Joseph A. Konstan, UMN).
- 2018โ2025: NSF 17-51278: CAREER: User-Based Simulation Methods for Quantifying Sources of Error and Bias in Recommender Systems ($514K incl.ย REU supplements; PI; $43,598 brought to Drexel).
Internal Grants /โ/โ2 /โ/โ$27K
- 2017: Boise State College of Education Civility Grant LITERATE: Locating Informational Texts for Engaging Readers And Teaching Equitably ($19K; co-PI; with PI Katherine Wright & co-PI Sole Pera).
- 2014: Texas State University Research Enhancement Program (competitive internal research grant) Temporal Analysis of Recommender Systems ($8K; PI).
Publications
Author formatting key: myself, advised student, other student; โ presenter, ยงundergraduate student.
Citation counts are from Google Scholar (total 5482, h-index 31).
โ These publications have citations merged in Google Scholar; count is reported on the most most final version, such as the journal expansion of a conference article.
Journal Articles /โ/โ10
Referreed articles published in journals.
Fernando Diaz, Michael D. Ekstrand, and Bhaskar Mitra. 2025. Recall, Robustness, and Lexicographic Evaluation. Transactions on Recommender Systems (to appear). arXiv:2302.11370. Cited 2 times.
Jonathan Stray, Alon Halevy, Parisa Assar, Dylan Hadfield-Menell, Chloe Bakalar, Craig Boutilier, Amar Ashar, Lex Beattie, Michael Ekstrand, Claire Leibowicz, Connie Moon Sehat, Sara Johansen, Lianne Kerlin, David Vickrey, Spandana Singh, Sanne Vrijenhoek, Amy Zhang, McKane Andrus, Natali Helberger, Polina Proutskova, Tanushree Mitra, and Nina Vasan. 2024. Building Human Values into Recommender Systems: An Interdisciplinary Synthesis. Transactions on Recommender Systems 2(3) (June 5th, 2024; online November 12th, 2023), 20:1โ57. DOI 10.1145/3632297. arXiv:2207.10192 [cs.IR]. Cited 70 times. Cited 46 times.
Michael D. Ekstrand, Ben Carterette, and Fernando Diaz. 2024. Distributionally-Informed Recommender System Evaluation. Transactions on Recommender Systems 2(1) (March 7th, 2024; online August 4th, 2023), 6:1โ27. DOI 10.1145/3613455. arXiv:2309.05892 [cs.IR]. NSF PAR 10461937. Cited 16 times. Cited 9 times.
Michael D. Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2022. Fairness in Information Access Systems. Foundations and Trendsยฎ in Information Retrieval 16(1โ2) (July 11th, 2022), 1โ177. DOI 10.1561/1500000079. arXiv:2105.05779 [cs.IR]. NSF PAR 10347630. Impact factor: 8. Cited 191 times. Cited 85 times.
Michael D. Ekstrand and Daniel Kluver. 2021. Exploring Author Gender in Book Rating and Recommendation. User Modeling and User-Adapted Interaction 31(3) (February 4th, 2021), 377โ420. DOI 10.1007/s11257-020-09284-2. arXiv:1808.07586v2. NSF PAR 10218853. Impact factor: 4.412. Cited 205 times (shared with RecSys18โ). Cited 110 times (shared with RecSys18โ).
Michael D. Ekstrand, Katherine Landau Wright, and Maria Soledad Pera. 2020. Enhancing Classroom Instruction with Online News. Aslib Journal of Information Management 72(5) (November 17th, 2020; online June 14th, 2020), 725โ744. DOI 10.1108/AJIM-11-2019-0309. Impact factor: 1.903. Cited 19 times. Cited 12 times.
Michael D. Ekstrand and Michael Ludwig. 2016. Dependency Injection with Static Analysis and Context-Aware Policy. Journal of Object Technology 15(1) (February 1st, 2016), 1:1โ31. DOI 10.5381/jot.2016.15.1.a1. Cited 16 times.
Joseph A. Konstan, J.D. Walker, D. Christopher Brooks, Keith Brown, and Michael D. Ekstrand. 2015. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. Transactions on Computer-Human Interaction 22(2) (April 1st, 2015), 10:1โ23. DOI 10.1145/2728171. Impact factor: 1.293. Cited 119 times (shared with L@S14โ). Cited 30 times.
Justin J. Levandoski, Michael D. Ekstrand, Michael J. Ludwig, Ahmad Eldawy, Mohamed F. Mokbel, and John T. Riedl. 2011. RecBench: Benchmarks for Evaluating Performance of Recommender System Architectures. Proceedings of the VLDB Endowment 4(11) (August 1st, 2011), 911โ920. Acceptance rate: 18%. Cited 22 times. Cited 9 times.
Michael D. Ekstrand, John T. Riedl, and Joseph A. Konstan. 2011. Collaborative Filtering Recommender Systems. Foundations and Trendsยฎ in Human-Computer Interaction 4(2) (February 1st, 2011), 81โ173. DOI 10.1561/1100000009. Cited 1748 times. Cited 664 times.
Peer-Reviewed Conference Papers /โ/โ32
Peer-reviewed full and short papers published in conference proceedings.
Mohammad Namvarpour, Elham Aghakhani, Michael D. Ekstrand, Rezvaneh Rezapour, and Afsaneh Razi. 2025. The Evolving Landscape of Online Child Safety: Insights from Media Analysis. To appear in Proceedings of the 17th ACM Web Science Conference (WebSci โ25), May 20โ24, 2025. Acceptance rate: 39.6%.
Andrรฉs Ferraro, Michael D. Ekstrand, and Christine Bauer. 2024. Itโs Not You, Itโs Me: The Impact of Choice Models and Ranking Strategies on Gender Imbalance in Music Recommendation. Short paper in Proceedings of the 18th ACM Conference on Recommender Systems (RecSys โ24), Oct 14, 2024. ACM, pp.ย 884โ889. DOI 10.1145/3640457.3688163. arXiv:2409.03781 [cs.IR]. NSF PAR 10568004. Acceptance rate: 22%.
Ngozi Ihemelandu and Michael D. Ekstrand. 2024. Multiple Testing for IR and Recommendation System Experiments. Short paper in Proceedings of the 46th European Conference on Information Retrieval (ECIR โ24), Mar 24โ28, 2024. Lecture Notes in Computer Science 14610:449โ457. DOI 10.1007/978-3-031-56063-7_37. NSF PAR 10497108. Acceptance rate: 24.3%. Cited 3 times.
Michael D. Ekstrand, Lex Beattie, Maria Soledad Pera, and Henriette Cramer. 2024. Not Just Algorithms: Strategically Addressing Consumer Impacts in Information Retrieval. In Proceedings of the 46th European Conference on Information Retrieval (ECIR โ24, IR for Good track), Mar 24โ28, 2024. Lecture Notes in Computer Science 14611:314โ335. DOI 10.1007/978-3-031-56066-8_25. NSF PAR 10497110. Acceptance rate: 35.9%. Cited 9 times. Cited 3 times.
Amifa Raj and Michael D. Ekstrand. 2024. Towards Optimizing Ranking in Grid-Layout for Provider-side Fairness. In Proceedings of the 46th European Conference on Information Retrieval (ECIR โ24, IR for Good track), Mar 24โ28, 2024. Lecture Notes in Computer Science 14612:90โ105. DOI 10.1007/978-3-031-56069-9_7. NSF PAR 10497109. Acceptance rate: 35.9%. Cited 1 time. Cited 1 time.
Ngozi Ihemelandu and Michael D. Ekstrand. 2023. Candidate Set Sampling for Evaluating Top-N Recommendation. In Proceedings of the 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT โ23), Oct 26โ29, 2023. pp.ย 88-94. DOI 10.1109/WI-IAT59888.2023.00018. arXiv:2309.11723 [cs.IR]. NSF PAR 10487293. Acceptance rate: 28%. Cited 5 times. Cited 1 time.
Amifa Raj, Bhaskar Mitra, Michael D. Ekstrand, and Nick Craswell. 2023. Patterns of Gender-Specializing Query Reformulation. Short paper in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR โ23), Jul 23, 2023. pp.ย 2241โ2245. DOI 10.1145/3539618.3592034. arXiv:2304.13129. NSF PAR 10423689. Acceptance rate: 25.1%. Cited 3 times. Cited 1 time.
Ngozi Ihemelandu and Michael D. Ekstrand. 2023. Inference at Scale: Significance Testing for Large Search and Recommendation Experiments. Short paper in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR โ23), Jul 23, 2023. pp.ย 2087โ2091. DOI 10.1145/3539618.3592004. arXiv:2305.02461. NSF PAR 10423691. Acceptance rate: 25.1%. Cited 3 times.
Christine Pinney, Amifa Raj, Alex Hanna, and Michael D. Ekstrand. 2023. Much Ado About Gender: Current Practices and Future Recommendations for Appropriate Gender-Aware Information Access. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (CHIIR โ23), Mar 19, 2023. pp.ย 269โ279. DOI 10.1145/3576840.3578316. arXiv:2301.04780. NSF PAR 10423693. Acceptance rate: 39.4%. Cited 21 times. Cited 12 times.
Amifa Raj and Michael D. Ekstrand. 2022. Measuring Fairness in Ranked Results: An Analytical and Empirical Comparison. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR โ22), Jul 11, 2022. pp.ย 726โ736. DOI 10.1145/3477495.3532018. NSF PAR 10329880. Acceptance rate: 20%. Cited 66 times. Cited 45 times.
A. K. M. Nuhil Mehdy, Michael D. Ekstrand, Bart Knijnenburg, and Hoda Mehrpouyan. 2021. Privacy as a Planned Behavior: Effects of Situational Factors on Privacy Perceptions and Plans. In Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization (UMAP โ21), Jul 1, 2021. ACM, pp.ย 169โ178. DOI 10.1145/3450613.3456829. arXiv:2104.11847 [cs.SI]. NSF PAR 10223377. Acceptance rate: 23%. Cited 25 times. Cited 16 times.
รmer Kฤฑrnap, Fernando Diaz, Asia J. Biega, Michael D. Ekstrand, Ben Carterette, and Emine Yฤฑlmaz. 2021. Estimation of Fair Ranking Metrics with Incomplete Judgments. In Proceedings of The Web Conference 2021 (TheWebConf 2021), Apr 19, 2021. ACM, pp.ย 1065โ1075. DOI 10.1145/3442381.3450080. arXiv:2108.05152. NSF PAR 10237411. Acceptance rate: 21%. Cited 49 times. Cited 36 times.
Michael D. Ekstrand. 2020. LensKit for Python: Next-Generation Software for Recommender Systems Experiments. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM โ20, Resource track), Oct 21, 2020. ACM, pp.ย 2999โ3006. DOI 10.1145/3340531.3412778. arXiv:1809.03125 [cs.IR]. NSF PAR 10199450. No acceptance rate reported. Cited 102 times. Cited 72 times.
Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, and Ben Carterette. 2020. Evaluating Stochastic Rankings with Expected Exposure. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM โ20), Oct 21, 2020. ACM, pp.ย 275โ284. DOI 10.1145/3340531.3411962. arXiv:2004.13157 [cs.IR]. NSF PAR 10199451. Acceptance rate: 20%. Nominated for Best Long Paper. Cited 194 times. Cited 170 times.
Mucun Tian and Michael D. Ekstrand. 2020. Estimating Error and Bias in Offline Evaluation Results. Short paper in Proceedings of the 2020 Conference on Human Information Interaction and Retrieval (CHIIR โ20), Mar 14, 2020. ACM, 5 pp.ย DOI 10.1145/3343413.3378004. arXiv:2001.09455 [cs.IR]. NSF PAR 10146883. Acceptance rate: 47%. Cited 13 times. Cited 10 times.
Michael D. Ekstrand, Mucun Tian, Mohammed R. Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. 2018. Exploring Author Gender in Book Rating and Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys โ18), Oct 3, 2018. ACM, pp.ย 242โ250. DOI 10.1145/3240323.3240373. arXiv:1808.07586v1 [cs.IR]. Acceptance rate: 17.5%. Citations reported under UMUAI21โ. Citations reported under UMUAI21โ.
Michael D. Ekstrand, Rezvan Joshaghani, and Hoda Mehrpouyan. 2018. Privacy for All: Ensuring Fair and Equitable Privacy Protections. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018), Feb 23, 2018. PMLR, Proceedings of Machine Learning Research 81:35โ47. Acceptance rate: 24%. Cited 106 times. Cited 78 times.
Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018), Feb 23, 2018. PMLR, Proceedings of Machine Learning Research 81:172โ186. Acceptance rate: 24%. Cited 300 times. Cited 213 times.
Michael D. Ekstrand and Vaibhav Mahant. 2017. Sturgeon and the Cool Kids: Problems with Random Decoys for Top-N Recommender Evaluation. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference (Recommender Systems track), May 29, 2017. AAAI, pp.ย 639โ644. No acceptance rate reported. Cited 17 times. Cited 11 times.
Sushma Channamsetty and Michael D. Ekstrand. 2017. Recommender Response to Diversity and Popularity Bias in User Profiles. Short paper in Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference (Recommender Systems track), May 29, 2017. AAAI, pp.ย 657โ660. No acceptance rate reported. Cited 22 times. Cited 19 times.
Michael D. Ekstrand and Martijn C. Willemsen. 2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys โ16, Past, Present, and Future track), Sep 17, 2016. ACM, pp.ย 221โ224. DOI 10.1145/2959100.2959179. Acceptance rate: 36%. Cited 144 times. Cited 96 times.
Michael D. Ekstrand, Daniel Kluver, F. Maxwell Harper, and Joseph A. Konstan. 2015. Letting Users Choose Recommender Algorithms: An Experimental Study. In Proceedings of the 9th ACM Conference on Recommender Systems (RecSys โ15), Sep 16, 2015. ACM, pp.ย 11โ18. DOI 10.1145/2792838.2800195. Acceptance rate: 21%. Cited 140 times. Cited 100 times.
Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, and Joseph A. Konstan. 2014. User Perception of Differences in Recommender Algorithms. In Proceedings of the 8th ACM Conference on Recommender Systems (RecSys โ14), Oct 6, 2014. ACM, pp.ย 161โ168. DOI 10.1145/2645710.2645737. Acceptance rate: 23%. Cited 287 times. Cited 188 times.
Joseph A. Konstan, J.D. Walker, D. Christopher Brooks, Keith Brown, and Michael D. Ekstrand. 2014. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. In Proceedings of the First ACM Conference on Learning @ Scale (S โ14), Mar 4, 2014. ACM, pp.ย 61โ70. DOI 10.1145/2556325.2566244. Acceptance rate: 37%. Citations reported under TOCHI15โ. Cited 77 times.
Tien T. Nguyen, Daniel Kluver, Ting-Yu Wang, Pik-Mai Hui, Michael D. Ekstrand, Martijn C. Willemsen, and John Riedl. 2013. Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys โ13), Oct 14, 2013. ACM, pp.ย 149โ156. DOI 10.1145/2507157.2507188. Acceptance rate: 24%. Cited 60 times. Cited 42 times.
Michael Ekstrand and John Riedl. 2012. When Recommenders Fail: Predicting Recommender Failure for Algorithm Selection and Combination. Short paper in Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys โ12), Sep 10, 2012. ACM, pp.ย 233โ236. DOI 10.1145/2365952.2366002. Acceptance rate: 32%. Cited 88 times. Cited 73 times.
Daniel Kluver, Tien T. Nguyen, Michael Ekstrand, Shilad Sen, and John Riedl. 2012. How Many Bits per Rating?. In Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys โ12), Sep 10, 2012. ACM, pp.ย 99โ106. DOI 10.1145/2365952.2365974. Acceptance rate: 20%. Cited 48 times. Cited 38 times.
Justin J. Levandoski, Mohamed Sarwat, Mohamed F. Mokbel, and Michael D. Ekstrand. 2012. RecStore: An Extensible And Adaptive Framework for Online Recommender Queries Inside the Database Engine. In Proceedings of the 15th International Conference on Extending Database Technology (EDBT โ12), Mar 26, 2012. ACM, pp.ย 86โ96. DOI 10.1145/2247596.2247608. Acceptance rate: 23%. Cited 19 times. Cited 16 times.
Michael D. Ekstrand, Michael Ludwig, Joseph A. Konstan, and John T. Riedl. 2011. Rethinking The Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys โ11), Oct 24, 2011. ACM, pp.ย 133โ140. DOI 10.1145/2043932.2043958. Acceptance rate: 27% (20% for oral presentation, which this received). Cited 258 times. Cited 195 times.
Michael Ekstrand, Wei Li, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2011. Searching for Software Learning Resources Using Application Context. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST โ11), Oct 17, 2011. ACM, pp.ย 195โ204. DOI 10.1145/2047196.2047220. Acceptance rate: 25%. Cited 56 times. Cited 48 times.
Michael D. Ekstrand, Praveen Kannan, James A. Stempter, John T. Butler, Joseph A. Konstan, and John T. Riedl. 2010. Automatically Building Research Reading Lists. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys โ10), Sep 27, 2010. ACM, pp.ย 159โ166. DOI 10.1145/1864708.1864740. Acceptance rate: 19%. Cited 124 times. Cited 101 times.
Michael D. Ekstrand and John T. Riedl. 2009. rv youโre dumb: Identifying Discarded Work in Wiki Article History. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym โ09), Oct 25, 2009. ACM, 10 pp.ย DOI 10.1145/1641309.1641317. Acceptance rate: 36%. Selected as Best Paper. Cited 37 times. Cited 28 times.
Book Chapters /โ/โ2
Michael D. Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2022. Fairness in Recommender Systems. In Recommender Systems Handbook (3rd edition). Francesco Ricci, Lior Roach, and Bracha Shapira, eds.ย Springer-Verlagpp.ย 679โ707. DOI 10.1007/978-1-0716-2197-4_18. ISBN 978-1-0716-2196-7. Cited 38 times. Cited 21 times.
Daniel Kluver, Michael D. Ekstrand, and Joseph A. Konstan. 2018. Rating-Based Collaborative Filtering: Algorithms and Evaluation. In Social Information Access. Peter Brusilovsky and Daqing He, eds.ย Springer-Verlag, Lecture Notes in Computer Science vol.ย 10100, pp.ย 344โ390. DOI 10.1007/978-3-319-90092-6_10. ISBN 978-3-319-90091-9. Cited 152 times. Cited 100 times.
Invited Talks /โ/โ42
- Feb 2025
- Invited talk at Post-SWIRL Talks & Posters Session (meeting for attendees at the Strategic Workshop on Information Retrieval, Melbourne, Australia)
โInformation is Made of Peopleโ - Oct 2024
- Keynote at ROEGEN (workshop at RecSys 2024, Bari, Italy)
โResponsible Recommendation in the Age of Generative AIโ - Jul 2024
- Panelist at Workshop on Large Language Models (LLMs) for Evaluation in Information Retrieval (at SIGIR 2024)
- May 2024
- Overview talk at Dagstuhl Seminar 24211
- May 2024
- Mar 2024
- Feb 2024
- Oct 2023
- May 2023
- Invited talk at Beyond Nudging, Towards Diversity: Understanding Transparent Algorithmic Recommendation Practices for Media and Communications (post-conference panel at ICA 2023, virtual)
โBeyond Diversity and Transparency: Normative Recommendation Goals in Human Contextโ - Mar 2023
- Feb 2023
- Seminar at Drexel University
โMaps and Lenses on Fairness in Information Access Systemsโ - Jan 2023
- Seminar at University of Washington RAISE group
โEquity and Discrimination in Information Accessโ - Nov 2022
- Keynote at IBIS2022 (Information-Based Inductive Systems and Machine Learning) (Japanese machine learning conference, Tsukuba, JP)
โThe Complexity of Fairness in Information Accessโ - Nov 2022
- Oct 2022
- Keynote at EvalRS workshop at CIKM 2022
โDo You Want To Hunt A Kraken? Mapping and Expanding Recommendation Fairnessโ - Aug 2022
- Guest lecture at University of Maine IR course (virtual)
โFair IR and Test Collectionsโ - Mar 2022
- Seminar at University of Michigan School of Information (virtual)
โYou Might Also Think This Is Unfairโ - Nov 2021
- Oct 2020
- Guest lecture at Carnegie Mellon University Human-AI Interaction course
โRecommender Systems and Fairnessโ - Apr 2020
- Guest lecture at Emory University recommender systems course
โRecommender Systems and Fairnessโ - Mar 2020
- Seminar at Boise State University Ph.D in Computing Colloquium
โUser, Agent, Subject, Spyโ - Nov 2019
- Seminar at University of Texas at Austin
โUser, Agent, Subject, Spyโ - Oct 2019
- Session at Idaho Library Association 2019 Conference
โOnline Recommendation: What? Where? Why? How?โ - Aug 2019
- Lecture at IVADO Summer School (Montrรฉal, QC)
โFairness and Discrimination in Recommendation and Retrievalโ - Aug 2019
- Seminar at Microsoft Research Montrรฉal
โUser, Agent, Subject, Spyโ - Jul 2019
- Seminar at Criteo AI Labs (Paris, France)
โUser, Agent, Subject, Spy โ - May 2019
- Invited talk at CRA CCC Visioning Workshop on Economics and Fairness
โRecommendations, Decisions, Feedback Loops, and Maybe Saving the Planetโ - Dec 2018
- Seminar at Clemson University
โUser, Agent, Subject, Spyโ - Nov 2018
- Seminar at Carnegie Mellon University Human-Computer Interaction Institute
โUser, Agent, Subject, Spyโ - Nov 2018
- Guest lecture at Carnegie Mellon University Human-AI Interaction course
โRecommender Systemsโ - Nov 2017
- Seminar at Whitman College (Walla Walla, WA)
โMaking Information Systems Good for Peopleโ - Oct 2017
- Overview talk at Dagstuhl Seminar 17442
- Jun 2017
- Seminar at RecSysNL at TU Delft (Delft, NL)
โRecommending for Peopleโ - Jun 2017
- Seminar at Jheronimus Academy of Data Science (โs-Hertogenbosch, NL)
โRecommending for Peopleโ - Jun 2017
- Seminar at UCL Mons (Mons, BE)
โRecommending for Peopleโ - Jun 2017
- Keynote at Brussels Big Data and Ethics Meetup (inaugural event of the DigitYser Big Data community, Brussels, BE)
โResponsible Recommendationโ - Nov 2016
- Seminar at University at Albany
โRecommending for Peopleโ - Oct 2016
- Lecture at Clearwater Developer Conference (Boise, ID)
โIntroduction to Recommender Systems โ - Sep 2015
- Invited talk at Large-Scale Recommender Systems (workshop at RecSys โ15)
โChallenges in Scaling Recommender Systems Researchโ - Sep 2015
- Invited talk at RecSys Doctoral Symposium
โLevelling Up your Academic Careerโ - Sep 2012
- Invited talk at RecSys Challenge (workshop at RecSys โ12)
โFlexible Recommender Experiments with LensKitโ - Sep 2012
- Invited talk at RecSys Challenge (workshop at RecSys โ12)
โThe MovieLens Data Setโ
Teaching
Drexel University
- DSCI 641 (Recommender Systems for Data Science)
- INFO 659 (Intro to Data Analytics)
Boise State University
- CS 410/510 (Databases)
- CS 533 (Intro to Data Science)
- CS 538 (Recommender Systems)
- CS 697 (Special Topics: Equity and Discrimination in Computing Systems)
Texas State University
- CS 4332 (Intro to Database Systems)
- CS 3320 (Internet Software Development)
- CS 5369Q/4379Q (Recommender Systems)
- CS 4350 (Unix Systems Programming)
Coursera
I co-created the Recommender Systems specialization on Coursera, along with its two previous single-class versions, with Joseph A. Konstan. This course has reached over 95,000 learners across its 3 iterations.
University of Minnesota
- Instructor for CS 5980-1 (Intro to Recommender Systems)
- Summer instructor for CS 1902 (Structure of Computer Programming II)
- TA for CSCI 5125 (Collaborative and Social Computing) and CSCI 1902
Teaching Professional Development
- Boise State University teaching portfolio faculty learning community.
- Boise State University Ten for Teaching program.
- Boise State University Center for Teaching and Learning Course Design Institute, a one-week intensive session in Summer 2017.
- CTL workshops on service learning, mastery-based grading, and other topics.
- Texas State Universityโs Program for Excellence in Teaching and Learning (2014โ2015).
- Preparing Future Faculty at the University of Minnesota.
Service
Ongoing Professional Service, Memberships, and Honors
- Associate editor, ACM Transactions on Recommender Systems (2024โ)
- Editorial board, Foundations and Trends in Information Retrieval (2023โ)
- Co-chair, FAccT Network, 2019โ
- Steering committee, ACM Conference on Recommender Systems (RecSys), 2017โ
- Senior Member, Association for Computing Machinery (since 2019)
- Distinguished Reviewer, ACM Transactions on Interactive Intelligent Systems (TiiS) (2017โpresent)
Past Service Highlights
- Executive committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2020โ2023
- Steering committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2017โ2023 (inaugural member)
- Program co-chair, 16th ACM Conference on Recommender Systems (RecSys 2022)
- General co-chair, 12th ACM Conference on Recommender Systems (RecSys 2018)
Program Committee and Editorial Service
- ACM SIGIR main program (AC 2024โ2025; PC 2020โ2021, 2023), Perspectives (PC 2021), short papers (PC 2021), resource track (PC 2021)
- ACM FAccT (AC 2023โ2025; PC 2021)
- ECIR main program (PC 2024โ2025), short papers (PC 2024โ2025), IR for Good (PC 2024), tutorials (PC 2024)
- ACM CIKM main program (PC 2024), resource track (PC 2020โ2021)
- ACM RecSys main program (SPC 2019โ2021, 2023โ2024; PC 2014โ2017), Reproducibility (PC 2021, 2023), LBR (PC 2019โ2020), Posters (PC 2016โ2017)
- Best paper committee, ACM SIGIR 2023
- SIGIR Asia-Pacific (SPC 2023)
- Best paper committee, TheWebConf 2023
- Track chair, UMAP 2023 (Responsibility, Compliance, and Ethics)
- Guest editor, 2021 special issue of User Modeling and User-Adapted Interaction (UMUAI) on fairness in user modeling.
- TheWebConf User Modeling, Behavior, & Personalization (SPC 2021; PC 2016, 2018โ2020), Behavior Analysis and Recommendation (PC 2016)
- Track Chair, UMAP 2021
- ACM WSDM (PC 2020โ2021)
- Ethics reviewer, NeurIPS 2021
- UMAP (PC 2018โ2020)
- CHI Posters (PC 2019)
- FLAIRS Special Track on Recommender Systems (PC 2015โ2017)
- ACM SAC Recommender Systems (PC 2013, 2016)
- NeurIPS
- Additional conference reviews for CHI (2012, 2015โ2017, 2019โ2020), CSCW (2014, 2017, 2019โ2020), FAT (2017โ2019), ICSOC (2016), IUI (2016), and UIST (2012, 2016โ2017, 2020).
- Journal reviews for Advances in AI, Artificial Intelligence Review, CACM, CSUR, IBM Journal of Research and Development, INRT, Information Retrieval Journal, Interacting with Computers, International Journal of Artificial Intelligence Tools, JMLR Open Source, JRC, Journal of Librarianship & Information Science, PLOS ONE, PeerJ Computer Science, TDS, TDSC, TIST, TKDE, TOCHI, TOIS, TORS, TSC, TWEB, TiiS, and UMUAI.
- Reviewer for numerous workshops at RecSys, UMAP, and elsewhere.
Other Professional Service
- Track co-oragnizer, Product Search and Recommendation track at TREC 2025
- Doctoral symposium co-chair, ACM RecSys 2025
- Founder and co-organizer, FATREC Workshop on Responsible Recommendation at RecSys 2017โ2018, 2020โ2021, 2023โ2024
- Co-organizer, AltRecSys Workshop on Alternative, Unexpected, and Critical Ideas in Recommendation at RecSys 2024
- Participant, Dagstuhl Seminar 24211: Evaluation Perspectives of Recommender Systems: Driving Research and Education (2024)
- Co-author and signatory, FAccT Statement on AI Harms and Policy (2023); covered by VentureBeat and The Hill (op-ed)
- Co-organizer, CRAFT panel โTheories of Change in Responsible AIโ at FAccT 2023
- Ph.D.ย symposium mentor, CIKM 2023
- Co-organizer, TREC Track on Fairness in Information Retrieval (2019โ2022)
- Co-organizer, SimuRec Workshop on Simulation and Synthetic Data for Recommender Systems at RecSys 2021
- Sponsorship co-chair, ACM FAccT 2021โ2022
- Doctoral symposium co-chair, ACM RecSys 2022
- Co-organizer, FairUMAP workshop at UMAP 2018โ2020
- Organized and moderated panel at RecSys 2019 on responsible recommendation
- PR & Publicity co-chair, 2nd Conference on Fairness, Accountability, and Transparency (ACM FAT* 2019)
- Co-organizer, Workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval (FACTS-IR) at SIGIR 2019
- Publications working group, FAccT steering committee (2017)
- Participant, Dagstuhl Perspectives Workshop 17442: Towards Cross-Domain Performance Modeling and Prediction: IR/RecSys/NLP (2017)
- Publicity co-chair, ACM RecSys 2016
- External advisor, CrowdRec (EU Framework Programme collaborative research project, 2014โ2016)
- Proceedings co-chair, ACM CHI 2012โ2013
- Demos co-chair, ACM RecSys 2012
Department and University Service
- Drexel IS Ph.D.ย committee (2023-2025)
- Drexel MSDS Working Group (2024โ2025)
- Drexel IS 2023-2024 Faculty Search Committee
- Boise State 2020โ2021 CS Faculty Search Committee
- Boise State COEN SAGE Scholars Program Mentor (2019โ2021)
- Boise State College of Engineering Curriculum Committee (2019โ2022)
- Boise State Ph.D.ย in Computing Steering Committee (2017โ2022)
- Boise State CS Dept. Curriculum Committee (2017โ2022)
- Boise State CS Dept. Graduate Recruiting Committee (2017)
- Texas State CS Dept. Undergraduate Committee (2014โ2016)
- Texas State CS Dept. Written Comp Exam Grading (2014โ2016)
- UMN CS Graduate Student Association secretary (2009โ2010)
Community and Civic Service
- January 2023 โ joined amicus brief before SCOTUS on Gonzalez v. Google.
- July 2020 โ taught continuing education session for Idaho Council for Libraries.
- October 2019 โ presented at Idaho Library Association Annual Conference.
- February 2019 โ addressed Idaho State House Judiciary Committee on H.B. 118, regulating pretrial risk assessment algorithms; through subsequent engagement, I contributed language that is in the final enacted legislation.
- December 2017 โ Boise Public Library panel on preparing for a career in computer science.
- Spring 2015 โ Judge for Travis Elementary School Science Fair.