jameslittle.me
blog portfolio reading list guestbook

← Back to Portfolio

Isolation-Based Scene Generation

My senior thesis describing a synthetic data generation method for training machine-learning object detectors.

Tensorflow 2018-19

My senior thesis built on some of the RoboCup work I had done as a Freshman and Sophomore. One previous team captain had tried (with limited success) to write an algorithm to detect the league's new soccer ball, which was black and white instead of orange, and could not be detected using image color segmentation alone.

We had tried using machine learning but quickly ran into the issue of getting a training dataset. My thesis worked to formalize a method for generating training images and tried to argue that, in this scenario, synthetic training data would be nearly as effective as real training data.

This project aimed to solve the problem of not having enough training data by defining and exploring the use cases for a synthetic data generation methodology. This methodology would let researchers with limited resources build a dataset that could be used to train object detectors for audio-visual purposes. The methodology, called Isolation-Based Scene Generation (IBSG), involved identifying and extracting key “ingredients” in the dataset and computationally compositing these ingredients to create artificial, automatically classified scenes.

We trained Tensorflow-based neural networks with hand-labeled, real-world-collected data and trained similar networks with IBSG data that has been designed to mimic that real data. We then compared the accuracy of those neural nets using a real data test set on both the IBSG-trained models and the real-trained models. Finally, we explored if and when ISBG is an appropriate replacement for real data, and make suggestions on how ISBG datasets should be created and designed.

A paper based on the thesis was submitted to the 2019 Conference on Neural Information Processing Systems (NIPS).

For more information, see the project's website at https://ibsg.jameslittle.me.

Made by James Little between 2016 and 2020.