Incorporating Rich User Feedback Into Interactive Machine Learning Applications
Successful interactive machine learning systems need to generalize robustly from a very small number of examples. This poses challenges for most machine learning algorithms, which typically only solicit labels from the users while ignoring any additional explanations users might be willing to provide to explain their choices. Several projects have shown that incorporating richer feedback---that captures the user's rationale---leads to faster and more generalizable learning. So far, this feedback has been limited to feature relevance. Is this the best or the only type of rich feedback we can elicit from users?
The results of our preliminary study shows that people naturally provide several other types of feedback to explain their decisions and that those other types of feedback have an even stronger positive impact on the predictive accuracy of machine learning algorithms than feature relevance. These results will impact both the algorithm and the interaction design for interactive machine learning systems.
Controlling Complex Applications with a Brain-Computer Interface
Brain-Computer Interfaces (BCIs) have the potential to enable paralyzed people to continue to communicate and control their environment even if they lack voluntary control over any muscle in their bodies. Last few decades saw a lot of progress in how quickly and how robustly people can transmit information through such interfaces. Relatively little effort, however, went into designing BCI-controlled applications. Because BCIs have very different properties from any of our current input devices, efficient BCI-mediated control of complex applications will require rethinking of all the basic user interface building blocks.
We have implemented our own p300 speller (see another group's explanation) -- a very successful EEG-based text entry application. We are now about to start exploring approaches to p300-based control of more complex interactions. This is a fascinating upcoming project that requires a combination of signal processing, machine learning, and interaction design.
Ability-Based User Interfaces
Most of today's GUIs are designed for the typical, able-bodied user;
atypical users are, for the most part, left to adapt as best they can,
perhaps using specialized assistive technologies as an aid. We have
developed an alternative
approach: our ABILITY MODELER uses a one-time motor performance test to build a personalized model of a person's motor abilities and SUPPLE
automatically generates interfaces which are tailored to an
individual's motor capabilities and which can be easily adjusted to
accommodate varying vision capabilities.
In a study comparing this approach to baseline interfaces, our results
show that users with motor impairments were much faster and strongly
preferred SUPPLE ability-based interfaces. Specifically,
motor-impared participants were 26.4% faster using interfaces
generated by SUPPLE. They made 73% fewer errors, strongly preferred
those interfaces to the manufacturers' defaults, and found them more
efficient, easier to use, and much less physically tiring. These
findings indicate that rather than requiring some users with motor
impairments to adapt themselves to software using separate assistive
technologies, software can now adapt itself to the capabilities of its
users.
[Related papers][SUPPLE Project web site]
Exploring The Design Space Of Adaptive User Interfaces
For decades, researchers have presented different adaptive user
interfaces and discussed the pros and cons of adaptation on task
performance and satisfaction. Little research, however, has been
directed at isolating and understanding those aspects of adaptive
interfaces which make some of them successful and others not. We have conducted several laboratory studies to systematically isolate some of the design and contextual factors that affect the impact of adaptation on users' performance and satisfaction.
[Related papers]
Prior work has highlighted the challenges faced by people with
motor impairments when trying to acquire on-screen targets using
a mouse or trackball. Two reasons for this are the difficulty of
positioning the mouse cursor within a confined area, and the
challenge of accurately executing a click. We hypothesize that
both of these difficulties with area pointing may be alleviated in a
different target acquisition paradigm called "goal crossing." In
goal crossing, users do not acquire a confined area, but instead
pass over a target line. Although goal crossing has been studied
for able-bodied users, its suitability for people with motor
impairments is unknown.
In our study, participants with motor impairments were faster with and preferred goal-crossing to area pointing.
This work provides the empirical
foundation from which to pursue the design of crossing-based
interfaces as accessible alternatives to pointing-based interfaces.
Recent years have revealed a trend towards
increasing use of optimization as a method for automatically designing
aspects of an interface's interaction with the user. In most cases,
this optimization may be thought of as decision-theoretic --
the objective is to minimize the expected cost of a user's
interactions or (equivalently) to maximize the user's expected
utility. While decision-theoretic optimization provides a powerful,
flexible, and principled approach for these systems, the quality of
the resulting solution is completely dependent on the accuracy of the
underlying utility or cost function. Unfortunately, determining the
correct utility function is a complex, time-consuming, and error-prone
task. While domainspecific learning techniques have been used
occasionally, most practitioners parameterize the utility function and
then engage in a laborious and unreliable process of hand-tuning.
SUPPLE is an application- and device-independent system that automatically
generates user interfaces for a wide variety of display devices.
SUPPLE uses decision-theoretic optimization to render an interface
from an abstract functional specification and an interchangeable
device model. SUPPLE can use information from the user
model to automatically adapt user interfaces to different tasks
and work styles while also prividing extensive customization
mechanisms that allow for modifications to the appearance,
organization and navigational structure of the user interface.
Child mortality is one of the most pressing health concerns almost 10
million children die worldwide each year before reaching their fifth
birthday, mostly in low-income countries. To aid overburdened and
undertrained health workers the World Health Organization (WHO) and
United Nations Children's Fund (UNICEF) have developed clinical
guidelines, such as the Integrated Management of Childhood Illness
(IMCI) to help with the classification and treatment of common
childhood illness. To help with deployment, we have developed an
electronic version (e-IMCI) that runs on a PDA. From July to September
2007, we ran a pilot of e-IMCI in southern Tanzania. The system guides
health workers step-by-step through the treatment algorithms and
automatically calculates drug doses. Our results suggest that
electronic implementations of protocols such as IMCI can reduce
training time and improve adherence to the protocol. They also
highlight several important challenges including varying levels of
education, language and expertise, which could be most adequately
addressed by implementing novel intelligent user interfaces and
systems.
Opportunity Knocks (OK) is an automated transportation routing system,
whose goal is to improve the efficiency, safety and independence of
individuals with mild cognitive disabilities. OK is
implemented on a combination of a Bluetooth sensor beacon that
broadcasts GPS data, a GPRS-enabled cell-phone, and remote activity
inference software. The system uses a novel inference engine that does
not require users to explicitly provide information about the start or
ending points of their journeys; instead this information is learned
from users' past behavior.
Alfred is an electronic butler for Intelligent Environments.
Alfred allows an end user to "program" the system by telling it the name of a new
goal, demonstrating one or more plans for achieving that goal, and finally telling
the system the conditions under which it would prefer one plan to another.
Similarly, the user can name events that arise in the environment and tell the
system what goals should be posted when those events arise. Each of these steps
can be done by simple verbal commands or other natural forms of interaction.
End users, in effect, record "macros" which, are executed adaptively and reactively.
"Look-to-talk" is a gaze-aware interface for directing a spoken
utterance to a software agent in a multiuser collaborative
environment. Through a prototype and a Wizard-of-Oz (WOz) experiment,
we showed that "look-totalk" is indeed a natural alternative to speech
and other paradigms.
FIRE is a multimodal interface for
information retrieval deployed in the Intelligent Room at the MIT AI
Lab. FIRE extracts all the category terms related to the search query
and uses entropy to generate questions that would quickly allow the
user to disambiguate her query and arrive at a small set of relevant
documents. FIRE presents information over several large displays in
the Intelligent Room and supports both speech and gesture input for
more natural interaction.
Rascal is a high-level resource management
system for the Intelligent Room Project, that addresses the problem of
the numerous applications competing for limited physical resources.
Rascal performs the service mapping and and uses constrained search
for arbitration among different requesters.