I am a first year CS PhD student at the University of Michigan, advised by Justin Johnson. I am broadly interested in deep learning, with applications in computer vision and natural language processing.

Prior to joining UMich, I was a visiting scholar at the Georgia Institute of Technology, collaborating with awesome folks in the labs of Devi Parikh and Dhruv Batra. My work was focused on image captioning (nocaps), probabilistic neural-symbolic models for VQA, and visual dialog.

I graduated from Indian Institute of Technology Roorkee in 2018, with a major in Electrical Engineering and minor in Computer Science. I was selected twice as a Google Summer of Code student, with TARDIS Foundation (2016) and OpenCV (2017). Parallel to my second GSoC, I interned at Goldman Sachs, Bangalore. I was also an active member of the Mobile Development Group IITR, and started a reading group, now named Vision and Language Group.

Feel free to say hi: kdexd at umich dot edu


What's New

[Jun 2020] Paper out on arxiv: VirTex: Learning Visual Representations from Textual Annotations.
[May 2020] Serving as a reviewer for ECCV 2020.
[Oct 2019] Presented my work nocaps at ICCV 2019, Seoul.
[Sep 2019] Started as a PhD student at the University of Michigan!
[Aug 2019] nocaps: novel object captioning at scale accepted to ICCV 2019!
[Apr 2019] Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering accepted to ICML 2019!
[Feb 2019] Paper out on arxiv: Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering.


Research

VirTex: Learning Visual Representations from Textual Annotations
Karan Desai and Justin Johnson
virtex model
nocaps: novel object captioning at scale
Harsh Agrawal*, Karan Desai*, Yufei Wang, Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, Stefan Lee, Peter Anderson
nocaps dataset

Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering
Ramakrishna Vedantam, Karan Desai, Stefan Lee, Marcus Rohrbach, Dhruv Batra, Devi Parikh
probnmn model

Open Source

visdial-challenge-starter-pytorch
Starter code for the Visual Dialog Challenge. Built using PyTorch v1.0 and provides out of the box support with CUDA 9 and CuDNN 7. Provides a simple implementation of Late Fusion encoder and Discriminative decoder. Complete with efficient scripts for data preprocessing, image feature extraction, training and evaluation, along with support to generate a submission file for the challenge.
lang-emerge-parlai
Implementation of the paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" by Kottur et al (EMNLP 2017), using PyTorch and ParlAI.
lang-emerge-parlai
ntm-pytorch
Implementation of Neural Turing Machines introduced in Graves et al (2014), using PyTorch. Supports training and evaluation on four out of six tasks described in the paper.
ntm-pytorch
trianglify
Trianglify is a highly customizable library to generate beautiful triangle art views for android. Uses the Delaunay Triangulation algorithm behind the curtains.
trianglify
yolog
Yolog wraps over vanilla git log and displays commit history complete with graph, timestamp, author and refs. Colors are configurable, and all standard git log commands work.
yolog

First Projects

While I keep marching forward as years pass by, my first projects would always remind me of my humble beginnings: it has been five years on this roller-coaster, which I call my professional life.

digit-classifier
First neural network built using numpy (2015), a multi layer perceptron classifier for MNIST. This repository stayed on Github trending charts for almost a week, with 600+ stars.
digit-classifier
snake
First github repository (2015), browser based game of snake. During my first year of undergrad studies. The game still works on gh-pages!
snake