Harsh Kumar

PhD Student · Artificial Intelligence · Human Computer Interaction · Computational Social Science · harsh@cs.toronto.edu

I am a third-year PhD student in the Department of Computer Science at the University of Toronto, working under the supervision of Dr. Ashton Anderson. My research intersects cognition, mental health, and education, with a focus on utilizing technology, like Large Language Models (LLMs), for social good. I explore disciplines such as Human-Computer Interaction and Psychology to enhance well-being and education through technology. My work includes studies on how LLMs can promote mindfulness and the design of LLM-based educational tools that facilitate genuine learning. I use Reinforcement Learning algorithms, like contextual bandits, to create personalized systems that adapt and improve over time, emphasizing inclusivity and tailored experiences for each user.

Updates

[August 24] Transparency Disclosures and Reliability Disclaimers for Learner-LLM Interactions got accepted to HCOMP'24!
[July 24] Presented 3 ongoing work at IC2S2 in Philadelphia! Chaired the session on "HCI & AI."
[May 24] Paper on guiding students to use LLMs got accepted to CSCW'24!
[April 24] Supporting Self-Reflection at Scale with Large Language Models got accepted to Learning@Scale'24!
[March 24] Accepted an offer to return to Microsoft Research in New York City for a summer internship.
[February 24] Presented Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health during AAAI'24.
[Jan 24] Math Education with Large Language Models: Peril or Promise? was featured in the Microsoft "New Future of Work" report for 2023 (Slide 25) [aka.ms/nfw]

Experience

Research Intern

Microsoft Research

At the Computational Social Science group at Microsoft Research, I investigated the potential of Large Language Models to augment human cognition, focusing on how they can be utilized to teach Mathematics.

Summer 2023 & 2024

Graduate Research Assistant

University of Toronto

My current research involves designing LLM-based educational tools for enhancing student comprehension, focusing on ensuring learners grasp underlying concepts. I am also researching the efficacy of Large Language Models in fostering mindfulness and mental well-being. My work includes using advanced reinforcement learning methods, like contextual bandits, to personalize LLM experiences for diverse users in educational and mental well-being contexts.

September 2021 - Present

Graduate Teaching Assistant

University of Toronto

I have been a teaching assistant for courses including CSC428: Human-Computer Interaction, CSC2552: Computational Social Science, CSC2558: Multidisciplinary HCI, CSC318: Design of Interactive Media, and CSC108: Introduction to Computer Programming, contributing to both educational content and student guidance.

September 2021 - Present

Software Engineer

J.P.Morgan Chase & Co.

Worked on a Long Short-term Memory (LSTM) network for Natural Language Understanding (NLU) module which could be improved using users' feedback. The project involved working on an end-to-end system involving the design of a user interface, the building and deployment of scalable models, and databases with continuous integration and delivery (CI/CD) pipelines.
Skills: Python - Django, Flask, Tensorflow, PyTorch. Tech stacks - LAMP, MERN. CI/CD.

January 2020 - August 2021

Research Analyst Intern

McKinsey & Company

Involved in the development of a semi-autonomous unmanned aerial vehicle (UAV) to optimize warehouse operations. Responsible for developing the vision system for object detection and avoidance using visual simultaneous localization and mapping technology.
Skills: Pytorch, OpenCV, NumPy, Microcontrollers.

May 2019 - January 2020

Programming Co-head

Students for the Exploration and Development of Space (SEDS-VIT)

Led the software teams of SEDS-VIT Projects, which took part in a number of international robotics competitions like AUVSI-SUAS, European Rover Challenge etc.

July 2018 - September 2019

Education

University of Toronto

Ph.D. in Computer Science

Courses: Introduction to Machine Learning, Design of Self-improving Systems, Ubiquitous Computing, Computational Social Science

September 2021 - Present

Vellore Institute of Technology

Bachelor of Technology, Computer Science and Engineering

July 2016 - July 2020

Competitions

DARPA AI Tools for Adult Learning

September 2023

Awarded US$250,000 for our project "QuickTA," an LLM-based tool for education, blending educational content, Reinforcement Learning adaptive algorithms, and Intelligent Tutoring Systems. This tool has already supported over 1,500 students in learning to write code.

XPRIZE Digital Learning Challenge

April 2023

As part of the Adaptive Experimentation Accelerator team, won the US$500,00 grand prize in the XPRIZE Digital Learning Challenge. Developed the Personalization System (MOOClet), incorporating contextual bandits, to conduct large-scale adaptive experiments in classrooms.

Publications

Accepted Publications

Harsh Kumar, Ilya Musabirov, Mohi Reza, Jiakai Shi, Joseph J. Williams, Anastasia Kuzminykh, Michael Liut. Guiding Students in Using LLMs in Supported Learning Environments: Effects on Interaction Dynamics, Learner Performance, Confidence, and Trust. (Just Accepted) ACM CSCW (2024).

Harsh Kumar, Ruiwei Xiao, Benjamin Lawson, Ilya Musabirov, Jiakai Shi, Xinyuan Wang, Huayin Luo, Joseph Jay Williams, Anna Rafferty, John Stamper, Michael Liut. Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms.(In Press) Proceedings of the Eleventh Annual Conference on Learning at Scale (L@S'24).

Harsh Kumar, Tong Li, Jiakai Shi, Ilya Musabirov, Rachel Kornfield, Jonah Meyerhoff, Ananya Bhattacharjee, Chris Karr, David Mohr, Anna Rafferty, Sofia Villar, Nina Deliu, Joseph Jay Williams. Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Digital Mental Health. (In Press) Proceedings of the Thirty-Sixth Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-24). AAAI 2024.

Ananya Bhattacharjee, Joseph J. Williams, Jonah Meyerhoff, Harsh Kumar, Alex Mariakakis, Rachel Kornfield. Investigating the Role of Context in the Delivery of Text Messages for Supporting Psychological Wellbeing. (Best Paper Award) ACM Conference on Human Factors in Computing Systems (CHI 2023).

Rachel Kornfield, Caitlin A Stamatis, Ananya Bhattacharjee, Bei Pang, Theresa Nguyen, Joseph J Williams, Harsh Kumar, Sarah Popowski, Miranda Beltzer, Christopher J Karr, Madhu Reddy, David C Mohr, Jonah Meyerhoff. A text messaging intervention to support the mental health of young adults: User engagement and feedback from a field trial of an intervention prototype. Internet Interventions (2023).

Jonah Meyerhoff, Miranda Beltzer, Sarah Popowski, Chris J Karr, Theresa Nguyen, Joseph J Williams, Charles J Krause, Harsh Kumar, Ananya Bhattacharjee, David C Mohr, Rachel Kornfield. Small Steps over time: A longitudinal usability test of an automated interactive text messaging intervention to support self-management of depression and anxiety symptoms. Journal of Affective Disorders (2023).

Working Papers

Harsh Kumar, David M. Rothschild, Daniel G. Goldstein, Jake M. Hofman. "Math Education With Large Language Models: Peril or Promise?" (Work done at Microsoft Research. Presented at CODE@MIT 2023).

Harsh Kumar, Ilya Musabirov, Jiakai Shi, Adele Lauzon, Kwan Kiu Choy, Ofek Gross, Dana Kulzhabayeva, Joseph J. Williams. "Exploring The Design of Prompts For Applying GPT-3 based Chatbots: A Mental Wellbeing Case Study on Mechanical Turk." arXiv preprint arXiv:2209.11344 (2022).

Tong Li, Jacob Nogas, Haochen Song, Harsh Kumar, Audrey Durand, Anna Rafferty, Nina Deliu, Sofia S. Villar, Joseph J. Williams. "Algorithms for Adaptive Experiments that Trade-off Statistical Analysis with Reward: Combining Uniform Random Assignment and Reward Maximization." arXiv preprint arXiv:2112.08507 (Under review) ACM Web Conference (2024).

Non-archival Publications

Harsh Kumar, Yiyi Wang, Jiakai Shi, Norman Farb, Joseph J. Williams. "Exploring the Use of Large Language Models for Improving the Awareness of Mindfulness." (Extended Abstract) CHI Conference on Human Factors in Computing Systems (CHI 2023).

Harsh Kumar, Jiakai Shi, Kwan Kiu Choy, Tong Li, Rachel Kornfield, Jonah Meyerhoff, Adele Lauzon, Ananya Bhattacharjee, Chris Karr, David Mohr, Anna Rafferty, Sofia Villar, Nina Deliu, Joseph Jay Williams. "Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Digital Mental Health." (Session Presentation & Poster) AI for Behaviour Change (AI4BC) workshop at AAAI-23.

Harsh Kumar, Jiakai Shi, Ilya Musabirov, Adele Lauzon, Meagan Peters, Ofek Gross, Joseph J. Williams. "Prompt Engineering to Improve Experiences with Large Language Model-Administered Support: A Digital Experiment with Online Crowdworkers." (Parallel Talk) Conference on Digital Experimentation (CODE 2022).

Harsh Kumar, Kunzhi Yu, Andrew Chung, Jiakai Shi, Joseph J. Williams. "Exploring The Potential of Chatbots to Provide Mental Well-being Support for Computer Science Students." (Poster) The ACM Technical Symposium on Computer Science Education (SIGCSE TS 2023).

Harsh Kumar, Taneea S Agrawaal, Kwan Kiu Choy, Jiakai Shi, Joseph J. Williams. "“Sounds like a Cheesy Radio Ad”: Using User Perspectives for Enhancing Digital COVID Vaccine Communication Strategies for Public Health Agencies." (Extended Abstract) CHI Conference on Human Factors in Computing Systems (CHI 2022).

Harsh Kumar, Dana Kulzhabayeva, Joseph J. Williams. "Using Adaptive Experiments for Personalizing Mental-Health Related Interventions to Account for Heterogeneity between Participants." (Poster) Intervention Science Preconference at Society For Personality and Social Psychology (SPSP 2023).

Aayush Kapur, Himanshu Thakur, Harsh Kumar. "Safeguarding People against Social Media Frauds during the COVID-19 Oxygen Supply Crisis in India." (Poster) NeurIPS Workshop on Machine Learning for the Developing World (ML4D, NeurIPS 2021).