Idea 1: Customisable virtual keyboard
Idea 2: Smart Pen
Idea 3: Ring based interaction
Customisable virtual keyboard
Year + Title + Link + Initial
|
Summary/ Interesting Findings/ Implications for our project
|
KITTY glove using Bluetooth, that is similar to what we want to do: and removes the need to type on a specific pressure sensitive surface. To detect letters (QWERTY layout) they use specific contact point Presents the use case of virtual/ augmented reality (e.g. typing while wearing Oculus Rift).
Updated version is a little less intrusive, but we could potentially make finger socks instead of gloves which could be less intrusive? Users commented on haptic feedback- perhaps we could use vibrations/sound when a letter is registered? Unpractised users took 2-7 secs per letter, perhaps we could improve on this with sensors or parallelisation: users wanted a tutorial
| |
[2009] Fast Finger Tracking System for In-air
|
Markerless tracking of finger movements from a camera, recognising typing on air. To achieve typing in real time, hardware that parallelizes image processing (throughput 138 fps)
(Ideally we’d want markered tracking so that the system could be used on a users lap)
|
A universal input device for both text (Korean) and Braille input was developed in a Glove-typed interface using all the joints of the four fingers and thumbs of both hands. Korean characters showed comparable performance with cellular phone input keypads, but inferior to conventional keyboard. Letters are typed by the touching of certain finger combinations (not QWERTY).
| |
PointGrab
|
This is not really a paper, but I wanted to investigate if someone had already tried to develop a system for home automation using gesture tracking. It turns out that there is a company called PointGrab that has been doing it for a few years. Here is the website: http://www.pointgrab.com/
|
Dextype
|
Dextype is a product that lets you type in the air. Because typing in air is so inaccurate, dextype helps by trying to figure out the key that you pressed. Also, it includes functions that make it very easy to complete words, draw symbols in air and correct previous typed input. Here is the website: http://www.cnet.com/news/type-in-the-air-with-dextype-for-leap-motion/
|
Eye Gaze Tracking for
Human Computer Interaction
|
This is a (very large) study on the advantages/disadvantages of using eye tracking as pointing mechanism as someone proposed last Tuesday. http://edoc.ub.uni-muenchen.de/11591/1/Drewes_Heiko.pdf
|
Hand Gesture Recognition Using
|
This paper investigates the detection of hand gestures using computer vision. Recognition of one-handed sign language is then used to implement a method of typing (see sections five and six).
|
[CHI 2003] Typing in Thin Air
The Canesta Projection Keyboard –
|
This device was envisioned as a solution for typing with mobile devices, similar to our use cases. A keyboard is projected on to a surface, then the user types on the projected keyboard. Infrared light is projected in a plane slightly above the surface. The intersection of fingers with the infrared plane is used to work out which key the user pressed, and an audible click noise is made. User studies show that users of this keyboard perform worse than they would with a standard mechanical keyboard but better than they do with ‘thumb keyboards’ - however, this is before the revolution in smartphones and the associated improvements in the touchscreen keyboards, so this may no longer be a relevant comparison.
|
TiltType: Accelerometer-Supported Text Entry
for Very Small Devices
|
TiltType is a novel text entry technique for mobile devices. To enter a character, the user tilts the device and presses one or more buttons. The character chosen depends on the button pressed, the direction of tilt, and the angle of tilt. TiltType consumes minimal power and requires little board space, making it appropriate for wristwatch-sized devices. But because controlled tilting of one's forearm is fatiguing, a wristwatch using this technique must be easily removable from its wriststrap. Applications include two-way paging, text entry for watch computers, web browsing, numeric entry for calculator watches, and existing applications for PDAs.
|
WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry
|
The lack of tactile feedback on touch screens makes typing difficult, a challenge exacerbated when situational impairments like walking vibration and divided attention arise in mobile settings. We introduce WalkType, an adaptive text entry system that leverages the mobile device's built-in tri-axis accelerometer to compensate for extraneous movement while walking. WalkType's classification model uses the displacement and acceleration of the device, and inference about the user's footsteps. Additionally, WalkType models finger-touch location and finger distance traveled on the screen, features that increase overall accuracy regardless of movement. The final model was built on typing data collected from 16 participants. In a study comparing WalkType to a control condition, WalkType reduced uncorrected errors by 45.2% and increased typing speed by 12.9% for walking participants.
|
Smart Pen
Year + Title + Link + Initial
|
Summary/ Interesting Findings/ Implications for our project
|
[2010] Increasing Viscosity and Inertia Using a Robotically Controlled Pen Improves Handwriting in Children, LINK, DB
|
The paper aims to determine the effect of mechanical properties of the pen on quality of handwriting in children.
They used the device the child holds to try and improve handwriting.
“We predict that children may have either improved or worsened handwriting using the robot, but writing will not be slower.”
They use the robot to increase the apparent inertia and viscosity of the pen.
 |
[2009] Poster: Teaching Letter Writing using a Programmable Haptic Device Interface for Children with Handwriting Difficulties, LINK, DB
|
The aim was to use a haptic device (robotic arm) to improve the handwriting of children who had difficulty writing. This was done in a virtual environment to so no ink was written to any paper. It was all virtual.
The results showed that handwriting improved by using the haptic device. There was an advantage of 3D force feedback over just 2D force feedback. Further work is needed to show 3D force feedback is superior though.
|
This paper again used a virtual environment but with a real haptic device to teach people handwriting - in this case Chinese.
| |
“The Reactive Robot technology is capable of interpreting the human actions and exerting a more intelligent force feedback strategy.”
The Reactive robots are inspired to emulate the presence of a human tutor which feedbacks forces to the student.
The system not only reproduces the task, but also should be able to identify the action to be performed by the user.
This sounds similar to what we wanted to do but they only did it for a few letters of the Japanese alphabet. It was also still virtual.
|
Ring based interaction
Year + Title + Link + Initial
|
Summary/ Interesting Findings/ Implications for our project
|
An energy harvesting wearable ring platform for gestureinput on surfaces
|
This paper presents a remote gesture input solution for interacting indirectly with user interfaces on mobile and wearable devices. The proposed solution uses a wearable ring platform worn on users index finger. The ring detects and interprets various gestures performed on any available surface, and wirelessly transmits the gestures to the remote device. The ring opportunistically harvests energy from an NFC-enabled phone for perpetual operation without explicit charging. We use a finger-tendon pressure-based solution to detect touch, and a light-weight audio based solution for detecting finger motion on a surface. The two level energy efficient classification algorithms identify 23 unique gestures that include tapping, swipes, scrolling, and strokes for hand written text entry. The classification algorithms have an average accuracy of 73% with no explicit user training. Our implementation supports 10 hours of interactions on a surface at 2 Hz gesture frequency. The prototype was built with off-the-shelf components has a size similar to a large ring.
|
finger-worn textile sensor for eyes-free mobile interaction during daily activities. Although existing products like a data glove possess multiple sensing capabilities, they are not designed for environments where body and finger motion are dynamic. Usually an interaction with fingers couples bending and pressing. In Plex, we separate bending and pressing by placing each sensing element in discrete faces of a finger. Our proposed simple and low-cost fabrication process using conductive elastomers and threads transforms an elastic fabric into a finger-worn interaction tool. Plex preserves an inter-finger natural tactile feedback and proprioception. We also explore the interaction design and implement applications allowing users to interact with existing mobile and wearable devices using Plex.
| |
More than touch: understanding how people use skin as an input surface for mobile computing
|
This paper contributes results from an empirical study of on-skin input, an emerging technique for controlling mobile devices.
Look at section “On-Skin Sensors” |
The Sound of Touch: On-body Touch and Gesture Sensing
Based on Transdermal Ultrasound Propagation
|
Can detect pressure/distance of touch on skin. Requires two ultrasound transducers - one transmits, one receives. Paper focuses on forearm but notes: “our signal propagation experiments suggest
that the sensing method could be extended to various body
parts”
|
Others:
Hand-writing Rehabilitation in the Haptic Virtual Environment - LINK
Motor Skill Training Assistance Using Haptic Attributes - LINK
Using haptics to improve motor skills but not specific to handwriting.
Human Motion Prediction in a Human-Robot Joint Task - LINK
Optimal Kinematic Design of a Haptic Pen - LINK
0 comments:
Post a Comment