GSF Sarah Adel Bargal Gives Thesis Proposal Defense
Monday, February 12, 2018
1:30pm – 3:00pm
MCS-B08
111 Cummington Mall
Sarah Adel Bargal
PhD Candidate, Computer Science
Hariri Institute Graduate Student Fellow
Join the Computer Science Department for Sarah Adel Bargal’s thesis proposal defense. Sarah Adel is a PhD candidate in the Computer Science Department and a Hariri Institute Graduate Student Fellow.
Grounding Deep Models of Visual Data
Abstract: Deep models are state-of-the-art for many vision tasks including video action recognition and video captioning. Models are trained to caption or classify activity in videos, but little is known about the evidence used to make such decisions. Grounding decisions made by deep networks gives more insight into model predictions. The main theme of this thesis is to formulate, implement, and evaluate new algorithms for grounding deep model outputs. Grounding can take a number of forms, including identifying evidence within the image or video input. We devise a formulation that simultaneously grounds evidence within a video in space and time, in a single pass, using top-down saliency. We visualize the spatiotemporal cues that contribute to a deep model’s output using the model’s internal representation. Based on these spatiotemporal cues, we are able to localize segments within a video that correspond with a specific action, or phrase from a caption, without explicitly optimizing/training for these tasks. We propose to develop algorithms to tackle other forms of grounding: identifying evidence within the training corpus for the model, or evidence within the trained model itself.
Committee Members:
Stan Sclaroff
Associate Dean of the Faculty, Mathematical and Computational Sciences
Professor, Department of Computer Science
College of Arts and Sciences
Boston University
Margrit Betke
Professor, Department of Computer Science
College of Arts and Sciences
Boston University
Kate Saenko
Associate Professor, Department of Computer Science
College of Arts and Sciences