Research

My research lies at the intersection of human cognition and artificial intelligence. I use computational and experimental methods to investigate how people perceive, learn, represent, and remember information, and what these processes can teach us about intelligence more broadly.

My work so far has examined human memory and attention across visual and linguistic domains, combining behavioral experiments, eye-tracking, neural network models, and semantic network analyses. These investigations have also shaped my interest in broader questions: what constitutes understanding or intentionality in artificial systems, how human and machine reasoning differ, and whether principles of human cognition can inform the development of AI systems that learn and generalize more effectively.

A toy semantic network
waterriverboatfishraincloudskystarmoonnightlampbook

Hover or select a word. Then switch languages and watch the same words regroup.

Edges here are illustrative, not study data. In the word memorability study, networks like this were built from word embeddings for each language, and measures such as degree were compared with how memorable each word was.

Themes

Memorability as a property of items

People differ in how good their memory is, yet they tend to agree on which items are memorable. I study which properties of words and videos, and of their representations, predict that agreement.

Methods: Online memory experiments, eye-tracking, reliability analysis

Comparing human and model behavior

A model can predict a behavioral measure without processing information the way people do. I compare where models and people attend, and how model-derived structure relates to human memory.

Methods: CNN–Transformer models, saliency metrics, panoptic segmentation, word embeddings

What transfers across languages

If memorability reflects something general about items, it should survive translation. If it depends on how a language organizes meaning, it may not. I test this with English and Hindi.

Methods: Cross-language correlation, semantic networks, graph measures, frequency controls


Projects

Peer-reviewed, WACV 2025

Seeing Eye to AI: Comparing Human Gaze and Model Attention in Video Memorability

When a model is trained only to predict how memorable a video is, does it end up attending to the same things people look at? We built a CNN–Transformer memorability model whose attention can be read out over space and time, collected eye-tracking data from people doing a video memory task, and compared the two using saliency metrics, temporal analyses, and panoptic segmentation.

Model diagram: video frames pass through a backbone encoder; per-frame features plus a CLS token enter a Transformer encoder; an MLP head on the CLS token outputs predicted memorability.
Model overview. From Kumar, Khandelwal et al., WACV 2025.

Manuscript in preparation

Does Word Memorability Generalize Across Languages?

Some words are remembered better than others, consistently across people. I ran a paired-associate memory experiment with 763 eligible native Hindi speakers on 221 Hindi words derived from an English memorability study, and compared item-level memorability across the two languages. In this dataset the English and Hindi scores were not significantly correlated. I am now examining how memorability relates to the structure of language-specific semantic networks, and how much of that relationship is carried by word frequency.


Observatory

Research notes, reading trails, and ideas worth examining more closely. The notes below are in draft.

  • Can an AI have intentions, or only behave as though it does?Draft
  • Is memorability a property of words, or of their representations?Draft
  • Why can humans learn from one example when machines often need thousands?Draft

Resources

Code
mandalab/SeeingEyeToAI on GitHub
Project page
Seeing Eye to AI, from Katha-AI and MANDALab, IIIT Hyderabad
Talk
WACV 2025 presentation video
Publications
Full list with citations
Google Scholar
Profile