Pain from Carpel Tunnel will make you want AI speech and gesture recognition
Overview
Here are the various options I’m exploring to using voice and gestures to control my Mac.
There are two aspects to operating a computer without hands:
-
Dictation (speech recognition).
-
Gesture recognition via a camera.
Dictation (speech recognition)
-
Open the Apple System Preferences
-
Click “Accessibility” or type that until the icon lights up or it appears on the list, then select it.
-
Scroll to select Dictation.
-
PROTIP: I once checked “Play sound when command is recognized” but realized that the action being carried out is enough feedback.
-
Click “Dictation Commands”.
These are the commands you speak.
-
Check “Enable advanced commands” for the full list to appear.
-
Memorize the various commands in this (more logical) order of categories:
- System
- Application
- Navigation
- Document
- Selection
- Editing
- Formatting
-
Check “Show Accessibility status in menu bar”, then you would be allowed to check “Enable the dictation keyword phrase”.
The default is “Computer”, like what people on Star Trek do.
-
Click Shortcut to select how to go into (and out of Dictation mode).
The default is press Fn key twice in rapid succession. But this requires having to look for it since it’s far from the home row.
BLAH: I would prefer to Customize and select pressing the Tab key twice.
-
Click on the microphone icon now visible at the top of the screen.
-
Take the time to try these in various applications.
Hand gestures
ControlAir from EyeSight-Tech.com is a free Mac OS X app that uses the FaceTime camera on your Mac Pro laptop to recognize your index finger and its movements.
With it you can control apps using hand recognition, similar to how Tom Cruise controlled 3D space using hand gestures within the sci-fi movie Minority Report. But in a more crude way.
Right now, it’s able to control iTunes and Spotify.
-
Click here to open iTunes.
Looks like it’s recently updated within the last few months?
-
Click Get, then Install.
-
Click the Launchpad icon and type “ControAir” until you see the its blue and white icon. Click it to open.
For some reason, the app’s icon doesn’t show up when I press Command+Tab to switch among apps.
But its icon is at the top of the screen.
Calibration Setup
-
Click Next.
PROTIP: Remember that you’ll need to always keep the Mac’s monitor at the same angle and be the same distance away from the monitor all the time.
-
Open Spotify to use during the Tutorial or iTunes will be opened automatically.
-
Click Start Tutorial.
-
I raised my finger but nothing happened.
BLAH: I wish the camera would still show so I see what the program sees.
BLAH: We can’t go forward with this until I hear back from support, but the link to support comes back with “Not Found”.
Configuration
- Click the icon at the top of the screen.
- Select Preferences.
- Click “Launch at startup”.
-
Select the apps which you have installed.
Operating
-
To mute, raise your finger to your lips.
-
To control volume, raise your index finger in the air for the menu, then move slightly to the left or right to select the button associated with the action you desire.
BLAH: I wish that the program would recognize gestures without requiring much hand-eye coordination. For example, recognize that I have a finger up, and that I move it upward for louder sound, or left to skip back. This would be useful not just for people who have difficulty seeing and managing intricate movement, but so I can concentrate on whatever other app I’m working on, and just need to control the background music without losing my concentration.
-
To pause, ???
BLAH: How about recognizing an open palm to pause?
This does use up power.
Better than the Leap Motion for Windows?
http://www.macworld.co.uk/how-to/mac-software/control-you-mac-with-hand-gestures-3598714/