Interests

Improving peoples’ lives through useful and usable technology. My research background is in human-computer interaction, with specific focuses on interactive machine learning and end-user software engineering. I enjoy studying how (and why) people perform specific tasks, pairing these observations with predictive theories, and using the results to inform and refine designs.

Affiliation

Design Researcher at Microsoft

Publications

Get notified of new publications by following my work on ResearchGate.

Journal articles

• Power to the people: The role of humans in interactive machine learning. Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. AI Magazine, 35, 4 (Winter 2014), pp. 105–120.
• You are the only possible oracle: Effective test selection for end users of interactive machine learning systems. Alex Groce, Todd Kulesza, Chaoqiang Zhang, Shalini Shamasunder, Margaret Burnett, Weng-Keen Wong, Simone Stumpf, Shubhomoy Das, Amber Shinsel, Forrest Bice, and Kevin McIntosh. IEEE Transactions on Software Engineering, 40, 3 (Mar. 2014), pp. 307–323.
• Why-oriented debugging of naive bayes text classification. Todd Kulesza, Simone Stumpf, Weng-Keen Wong, Margaret Burnett, Stephen Perona, Andrew Ko, and Ian Oberst. ACM Transactions on Interactive Intelligent Systems, 1, 1 (Oct. 2011).

Refereed conference papers

• Principles of Explanatory Debugging to personalize interactive machine learning. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. ACM Conference on Intelligent User Interfaces, Atlanta, GA, Mar. 2015, pp. 126–137.
• Structured labeling to facilitate concept evolution in machine learning. Todd Kulesza, Saleema Amershi, Rich Caruana, Danyel Fisher, and Denis Charles. ACM SIGCHI Conference on Human Factors in Computing Systems, Toronto, Canada, Apr. 2014, pp. 3075–3084. (Best Paper award)
• Too much, too little, or just right? Ways explanations impact mental models. Todd Kulesza, Simone Stumpf, Margaret Burnett, Sherry Yang, Irwin Kwan, Weng-Keen Wong. IEEE Symposium on Visual Languages and Human-Centric Computing, San Jose, CA, Sept. 2013, pp. 3–10.
  
• Tell me more? The effects of mental model soundness on personalizing an intelligent agent. Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. ACM SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, May 2012, pp. 1–10. (Honorable mention for Best Paper award)
• Mini-crowdsourcing end-user assessment of intelligent assistants: a cost-benefit study. Amber Shinsel, Todd Kulesza, Margaret Burnett, William Curran, Alex Groce, Simone Stumpf, and Weng-Keen Wong. IEEE Symposium on Visual Languages and Human-Centric Computing, Pittsburg, PA, Sept. 2011, pp. 47–54.
• Where are my intelligent assistant’s mistakes? A systematic testing approach. Todd Kulesza, Margaret Burnett, Simone Stumpf, Weng-Keen Wong, Shubhomoy Das, Alex Groce, Amber Shinsel, Forrest Bice, and Kevin McIntosh. International Symposium on End-User Development, Torre Canne, Italy, June 2011, pp. 171–186.
• Explanatory debugging: supporting end-user debugging of machine-learned programs. Todd Kulesza, Simone Stumpf, Margaret Burnett, Weng-Keen Wong, Yann Riche, Travis Moore, Ian Oberst, Amber Shinsel, and Kevin McIntosh. IEEE Symposium on Visual Languages and Human- Centric Computing, Madrid, Spain, Sept. 2010, pp. 41–48.
• Fixing the program my computer learned: barriers for end users, challenges for the machine. Todd Kulesza, Weng-Keen Wong, Simone Stumpf, Stephen Perona, Rachel White, Margaret Burnett, Ian Oberst, and Andrew J. Ko. ACM Conference on Intelligent User Interfaces, Sanibel Island, FL, Feb. 2009, pp. 187–196.
• Can feature design reduce the gender gap in end-user software development environments? Valentina Grigoreanu, Jill Cao, Todd Kulesza, Christopher Bogart, Kyle Rector, Margaret Burnett, and Susan Wiedenbeck. IEEE Symposium on Visual Languages and Human-Centric Computing, Herrsching am Ammersee, Germany, Sept. 2008, pp. 149–156.

Grad consortia, workshops, and Notes

• End-user development in Internet of Things: we the people. Margaret Burnett and Todd Kulesza. CHI 2015 Workshop on End-User Development in the Internet of Things Era, Seoul, Korea, Apr. 2015.
• IUI workshop on interactive machine learning. Saleema Amershi, Maya Cakmak, W. Bradley Knox, Todd Kulesza, and Tessa Lau. IUI 2013 Workshop on Interactive Machine Learning, Santa Monica, CA, Mar. 2013, pp. 121–123.
• Making intelligent systems understandable and controllable by end users. Simone Stumpf, Weng-Keen Wong, Margaret Burnett, and Todd Kulesza. Second Workshop on Intelligibility and Control in Pervasive Computing, Newcastle, UK, June 2012.
• The role of explanations in assessing and correcting personalized intelligent agents. Todd Kulesza, Margaret Burnett, Simone Stumpf, and Weng-Keen Wong. CHI 2012 Workshop on End-user Interactions with Intelligent and Autonomous Systems, Austin, TX, May 2012.
• Towards recognizing “cool”: can end users help computer vision recognize subjective attributes of objects in images? William Curran, Travis Moore, Todd Kulesza, Weng-Keen Wong, Sinisa Todorovic, Simone Stumpf, Rachel White, and Margaret Burnett. ACM Conference on Intelligent User Interfaces, Lisbon, Portugal, Feb. 2012, pp. 285–288.
• An explanation-centric approach for personalizing intelligent agents. Todd Kulesza. Doctoral consortium at the ACM Conference on Intelligent User Interfaces, Lisbon, Portugal, Feb. 2012, pp. 375–378.
• Toward end-user debugging of machine-learned classifiers. Todd Kulesza. Graduate consortium at the IEEE Symposium on Visual Languages and Human- Centric Computing, Madrid, Spain, Sept. 2010, pp. 253–254.
• End-user software engineering and distributed cognition. Margaret Burnett, Christopher Bogart, Jill Cao, Valentina Grigoreanu, Todd Kulesza, Joseph Lawrance. Proceedings of the 2009 ICSE Workshop on Software Engineering Foundations for End User Programming, Washington, DC, 2009, pp. 1–7.

Invited Talks

• Invited presentation, DUB Group at University of Washington. Seattle, WA. Aug. 2013.

Experience

• Research intern, Microsoft Research. Redmond, WA. June 2013 – Sept. 2013.

Open-Source Projects

• Hola, a dead-simple, pure-Java implementation of mDNS Service Discovery.
• Archivo, a Java app for archiving recordings from a TiVo to your computer (video demo).
• TivoLibre, a Java app and library for decoding TiVo files to standard MPEGs.
• IML Playground, source code (C#/.NET 4.5) for exploring interactive machine learning systems (video demo).
• QuickTuring, an efficient Java implementation of Qualcomm’s Turing stream cipher.

Education

• Oregon State University, 2010–2014 PhD in Computer Science
• Oregon State University, 2007–2010, MS in Computer Science
• Oakland University, 2001–2005, BS in Computer Science with a minor in History