Publications (including pre-prints)

Underwater Multi-Robot Convoying using Visual Tracking by Detection

Abstract:

We present a robust multi-robot convoying approach relying on visual detection of the leading agent, thus enabling target following in unstructured 3D environments. Our method is based on the idea of tracking by detection, which interleaves image-based position estimation via temporal filtering with efficient model-based object detection. This approach has the important advantage of mitigating tracking drift (i.e. drifting out of the view of the target), which is a common symptom of model-free trackers and is detrimental to sustaining convoying in practice. To illustrate our solution, we collected extensive footage of an underwater swimming robot in ocean settings, and hand-annotated its location in each frame. Based on this dataset, we present an empirical comparison of multiple tracker variants, including the use of several Convolutional Neural Networks both with and without recurrent connections, as well as frequency-based model-free trackers. We also demonstrate the practicality of this tracking-by-detection strategy in real-world scenarios, by successfully controlling a legged underwater robot in five degrees of freedom to follow another robot's arbitrary motion.

Download: Paper [In Review]

View: Project Website

A survey of available corpora for building data-driven dialogue systems

Abstract:

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective.

Download: Paper

View: Project Website

Autonomous Swarm Behaviour in Mesh Networked Agents

Abstract:

In recent years, unmanned aerial vehicles (UAVs) have become commonplace. They have been used most notably for military purposes, film making, and more recently, parcel delivery. However, current UAV technologies focus on piloted systems. While some focus has been shifted toward fully autonomous systems with large companies like Google and Amazon investing heavily in such tech- nologies, there is still much technology to be developed for these autonomous systems to become commonplace. Most important and lacking is the flight time of these systems. Current battery technologies cannot sustain motor function for extensive flight times, particularly in multi-copter systems, as such these systems still cannot accomplish the tasks they need without manual super- vision to travel safely and recharge. This restriction comes into play heavily with mapping and exploration tasks, which can be beneficial for many industries, particularly agriculture. However, recent developments in coordinated systems of multiple autonomous agents have shown that co- ordinated tasks can be accomplished at a much faster pace and in the available amount of flight time. As such, we pursue a multi agent system interconnected with mesh networking technologies, which can pursue a mapping and exploration task in a fully decentralized and self-coordinated autonomous manner. While a complete system including hardware may not be feasible during the year, we plan to design a software system and choose appropriate technologies to accomplish this task. Additionally, our goal is to run our designed software system and algorithms in simulation to show its feasibility and the agents’ ability to autonomously make decisions in a decentralized fashion. During this semester, we designed such a system and its requirements, researched the appropriate technologies, and began work on testing basic communication amongst hardware nodes. Finally, we began the development of a simulator which we could test our algorithms in.

Download: Technical Report I

Download: Technical Report II

Download: Poster

Download: Code

Watch: Brief 3D Demo

Implanted Intracortical Electrodes as Chronic Neural Interfaces to the Central Nervous System

Abstract:

Recent developments in neural interfaces show that it is possible to have fine control of a robotic prosthetic by interfacing with the motor cortex of the human brain. Development of long term systems for this purpose is a challenging task with many different possibilities. Intracortical implants have shown the most promise in providing enough signal selectivity and throughput for complex control systems with many degrees of freedom. Intracortical systems generally fall into two categories: MEMS devices and bundle of wire systems. While both show promise, MEMS systems have been greatly popularized due to their reproducibility. In particular, the Michigan probe and Utah microarray are often used as a base for construction of more complex intracortical systems. However, these systems still carry many downsides. Their long-term viability is questionable, with mixed results. The effects of damage from implantation are still inconclusive and immune responses remain a problem for long-term use. While there is some promising research in the use of bioactive molecules and biocompatible materials to prevent immune responses, more controlled study is needed before intracortical systems become widespread.

Download: Full Review Paper

An Analysis of Parallelized Motion Masking Using Dual-Mode Single Gaussian Models

Abstract:

Motion detection in video is important for a number of applications and fields. In video surveillance, motion detection is an essential accompaniment to activity recognition for early warning systems. Robotics also has much to gain from motion detection and segmentation, particularly in high speed motion tracking for tactile systems. There are a myriad of techniques for detecting and masking motion in an image. Successful systems have used Gaussian Models to discern background from foreground in an image (motion from static imagery). However, particularly in the case of a moving camera or frame of reference, it is necessary to compensate for the motion of the camera when attempting to discern objects moving in the foreground. For example, it is possible to estimate motion of the camera through optical flow methods or temporal differencing and then compensate for this motion in a background subtraction model. We selection a method by Yi et al. using Dual-Mode Single Gaussian Models which does just this. We implement the technique in Intel's Thread Building Blocks (TBB) and NVIDIA's CUDA libraries. We then compare parallelization improvements with a theoretical analysis of speedups based on the characteristics of our selected model and attributes of both TBB and CUDA. We make our implementation available to the public.

Download: Full Paper

Download: Code

Chaotic Memory Randomization for Securing Embedded Systems

Abstract:

Embedded systems permeate through nearly all aspects of modern society. From cars to refrigerators to nuclear refineries, securing these systems has never been more important. Intrusions, such as the Stuxnet malware which broke the centrifuges in Iran’s Natanz refinery, can be catastrophic to not only the infected systems, but even to the wellbeing of the surrounding population. Modern day protection mechanisms for these embedded systems generally look only at protecting the network layer, and those that try to discover malware already existing on a system typically aren’t efficient enough to run on a standalone embedded system. As such, we present a novel way to ensure that no malware has been inserted into an embedded system. We chaotically randomize the entire memory space of the application, interspersing watchdog-monitor programs throughout, to monitor that the core application hasn’t been infiltrated. By validating the original program through conventional methods and creating a clean reset, we can ensure that any inserted malware is purged from the system with minimal effect on the given system. We also present a software prototype to validate the possibility of this approach, but given the limitations and vulnerabilities of the prototype, we also suggest a hardware alternative to the system.

Download: Paper

Extending H-MAX with End-Stopped Simulations for Biologically Inspired Object Recognition

Abstract:

Here we modify the HMAX object recognition system of Serre et al. with an end-stopped filter to try and improve the accuracy of the model and to further the parallels in the algorithm to the actual processing of the visual cortex for learning object representations.

Download: Full Paper

Multilingual Sentiment Analysis of Twitter Posts

Abstract:

Using modified Deep Learning techniques presented by Socher et al., show the accuracy of a language-independent sentiment analysis algorithm. Training on a combined and separated sets, the learner determines a grammatical and sentiment model for each, then its performance is determined using k-fold validation.

Download: Poster

Russia’s Transfer From Communism to Capitalism: A Poor Economic Model and Its Aftermath

Description:

This was me delving into the world of anthropology freshman year. On rereading this, it could definitely use some clean-up linguistically, but there's quite a bit of research poured into this paper, so I wanted to share it.

Download: Full Paper

Learning Algorithms to Simulate Human Problem-Solving and Decision-Making

Description:

Before I actually knew in depth what Machine Learning algorithms really did and how they worked I took a course called CCOM 206. In it they wanted us to write a research paper. I did it on AI/ML, and here it is. It'll probably be more entertaining than useful, but I thought I'd throw it up just in case someone might get some use out of it (even if it is just a laugh).

Download: Full Paper