From Object Interactions to Fine-grained Video Understanding

Video understanding tasks such as action recognition and caption generation are crucial for various real-world applications in surveillance, video retrieval, human behavior understanding, etc. In this work, we present a generic recurrent module to detect relationships and interactions between arbitrary object groups for fine-grained video understanding. Our work is applicable to various open domain video … Continue reading From Object Interactions to Fine-grained Video Understanding

The Minds of the New Machines | Research Horizons | Georgia Tech’s Research News

Georgia Tech's Research Horizons Magazine has done a very nice write-up of the ML@GT center, featuring many of our research projects. Machine learning has been around for decades, but the advent of big data and more powerful computers has increased its impact significantly — ­moving machine learning beyond pattern recognition and natural language processing into a … Continue reading The Minds of the New Machines | Research Horizons | Georgia Tech’s Research News

Robust Skill Generalization Using Probabilistic Inference

Everyday skills, such as making your bed or even pressing a doorbell, might seem trivial to us, but are actually quite complicated for today’s robots. Think about your performance the first time you tried a sport.  Did you seek help from a peer or coach? Did you perform better after that? Most probably you answered yes. It … Continue reading Robust Skill Generalization Using Probabilistic Inference

Embodied Question Answering

Embodied Question Answering is a new AI task where an agent is spawned at a random location in a 3D environment and asked a question ("What color is the car?"). In order to answer, the agent must first intelligently navigate to explore the environment, gather information through first-person (egocentric) vision, and then answer the question ("orange").

Visualizing Deep Learning Models at Facebook

This post summarizes the latest joint research between researchers at Georgia Tech and  Facebook on using visualization to make sense of deep learning models, published at IEEE VIS’17, a top visualization conference. While powerful deep learning models have significantly improved prediction accuracy, understanding these models remains a big challenge. Deep learning models are more difficult … Continue reading Visualizing Deep Learning Models at Facebook

Syntax-Directed Variational Autoencoder for Structured Data

Advances in deep learning of representation have resulted in powerful generative approaches on modeling continuous data like time series and images, but it is still challenging to correctly deal with discrete structured data, such as chemical molecules and computer programs. To tackle these challenges, there has been many improvements in formalization of structure generation that … Continue reading Syntax-Directed Variational Autoencoder for Structured Data