Facebook programs computers to describe photos for the blind

ap.orghttp://www.ap.org

April 5, 2016

FILE - This Feb. 19, 2014, file photo shows the Facebook app icon on an iPhone in New York. A feature rolling out Tuesday, April 5, 2016, on Facebook's iPhone app interprets what's in a photo using a form of artificial intelligence to recognize faces and objects for blind and visually impaired people as they scroll through the pictures posted on the world's largest online social network (AP Photo/Karly Domb Sadof, File)

MENLO PARK, Calif. (AP) — Facebook is training its computers to become seeing-eye guides for blind and visually impaired people as they scroll through the pictures posted on the world's largest online social network.

The feature rolling out Tuesday on Facebook's iPhone and iPad apps interprets what's in a picture using a form of artificial intelligence that recognizes faces and objects. VoiceOver, a screen reader built into the software powering the iPhone and iPad, must be turned on for Facebook's photo descriptions to be read. For now, the feature will only be available in English.

Until now, people relying on screen readers on Facebook would only hear that a person had shared a photo without any elaboration.

The photo descriptions initially will be confined to a vocabulary of 100 words in a restriction that will prevent the computer from providing a lot of details. For instance, the automated voice may only tell a user that a photo features three people smiling outdoors without adding that the trio also has drinks in their hands. Or it may say the photo is of pizza without adding that there's pepperoni and olives on top of it.

Facebook is being careful with the technology, called "automatic alternative text," in an attempt to avoid making a mistake that offends its audience. Google learned the risks of automation last year when an image recognition feature in its Photos app labeled a black couple as gorillas, prompting the company to issue an apology.

Eventually, though, Facebook hopes to refine the technology so it provides more precise descriptions and even answers questions that a user might pose about a picture.

The vocabulary of Facebook's photo-recognition program includes "car," ''sky," ''dessert," ''baby," ''shoes," and, of course, "selfie."

Facebook also plans to turn on the technology for its Android app and make it available through Web browsers visiting its site.

The Menlo Park, California, company is trying to ensure the world's nearly 300 million blind and visually impaired people remain interested in its social network as a steadily increasing number of photos appear on its service. On an average day, Facebook says more than 2 billion photos are posted on its social network and other apps that it owns, a list that includes Messenger, Instagram and WhatsApp.

In a Tuesday post, Facebook CEO Mark Zuckerberg hailed the photo description tool as "an important step towards making sure everyone has equal access to information and is included in the conversation."

___

Online:

Facebook's post: https://newsroom.fb.com/news/2016/04/using-artificial-intelligence-to-help-blind-people-see-facebook/

iPad & iOS9 Accessibility Features Part2

Guided Access

Guided Access is a way to set certain parts of your screen off limits. Are you using a free app with ads and your child keeps clicking the ads? Just draw a square around those ads and suddenly they aren't clickable anymore! Guided Access has not changed since the iOS8 updates, but is definitely a feature worth mentioning for kids with motor challenges or just learning to use the iPad!

Here's how it works: Go to Settings > General > Accessibility and turn Guided Access on. You can set a passcode at this point if you want to (make it something easy, like 1-2-3-4) or you can do it later when you use the feature. You'll need the passcode to turn Guided Access off.

Now when you're in your app (I'm going to use Talking Tom as my example), triple click the home button to open up your Guided Access options. Draw a box around the areas you don't want your child to access (for Talking Tom, I drew a box around the ad at the top of the screen and the buttons that ask you to buy things on the right of the screen). Click start and you're back in the app, but now those areas are not accessible. Those areas are also grayed out a bit so they are less likely to draw your child's attention, too. Another hint is to make sure that you do not draw a box in the middle of the screen where you will need to enter your passcode to turn Guided Access off later!

You can also use Guided Access to set a time limit on apps. Are you using iPad play as a reward, but want your child to get back to another task in 10 minutes? Set Guided Access to 10 minutes and when the time is up, the session will end and you'll have to use a passcode to get the iPad up and running again.

When you're done using Guided Access, triple click the home button again and enter your passcode or place your finger on the Touch ID. Easy!

Switch Control

Switch access for the iPad allows users to interact with one or two large buttons rather than have to maneuver through the complexities of the touch screen. The iPad has always been switch accessible, particularly with apps designed specifically for switches, but starting in iOS7 the iPad now allows for complete system-wide switch control.

Go to Settings > General > Accessibility > Switch Control to turn Switch Control on. You can then tell the iPad how many switches you wish to use and you can fully customize how the switches interact with the iPad, from changing the cursor speed, size and color to adding sound effects and speech options (which is perfect for children who are visually impaired). A feature new to iOS9 is Recipes. A Recipe is a set of specialized actions you can temporarily assign to switches in place of their normal actions. Need to record a series of gestures to perform a complex action in an app? You can make it happen with a Recipe. You can also create a customized Launch Recipe that will be applied automatically whenever Switch Control starts.

For more information on Switch Control, check out this post from Assistive Technology Blog or this very in-depth piece (which includes screenshots and videos) from Jane Farrall's Blog. To learn more about the specific iOS9 updates, read this guide from AbleNet, "iOS9 Accessibility Switch Control - The Missing User Guide."

If you're looking to purchase switches, AbleNet sells a Bluetooth iPad switch that allows you to use either one or two built-in switches or even attach your own favorite switch. There is also the Bluetooth Switch Interface from RJ Cooper. Also, Inclusive Technology creates a series of fun and educational iPad apps that are designed specifically to work with a switch.

Head Gestures & Other Switch Options

Another really interesting feature of the iPad is the ability to do everything a switch allows you to do... but without a switch! Go toSettings > General > Accessibility > Switch Control > Switches > Add New Switch. You have three options here: "External" allows you to hook up a traditional external switch with your iPad; "Screen" lets you use the whole iPad screen as your switch; and "Camera" lets you use head gestures (a move of the head to the left or the right) as your switch. Isn't that neat?

So with "Screen" you can use your iPad just like you would when using a switch, but the touch screen itself acts as a large interactive switch.

And with "Camera" you control the cursor on the screen with a move of your head from left to right. You get to choose which movements control which features in the settings. For more information and a walk through on using head gestures to control the iPad, watch this video below:

Low Vision Options

Go to Settings > General > Accessibility and you have a few options to change the way your screen looks. You can Zoom to magnify the entire screen. It takes a somewhat complicated move (once turned on in your settings, you need to double-tap with three fingers in order to zoom in), but if you can help you child learn to control the zoom it can be a really powerful tool (unfortunately AssistiveTouch doesn't seem to be able to pull off the three-fingered double tap for you).

In iBooks it's a terrific way of showing a child with low vision what the illustrations are and because of how sweetly it functions, you or your child can move through the pictures smoothly, enlarged as required, to find out things like how sad Eyore's face looks or how big Pooh's tummy is.

The Zoom feature underwent a pretty major upgrade in iOS8. Once turned on, you can choose to magnify the entire screen (which is the default setting) or create a Window Zoom. Go to Settings > General > Accessibility > Zoom > Zoom Region and choose Window Zoom. Now, instead of enlarging the entire screen with your three-fingered double tap, you'll just see a zoom lens (sort of like using a magnifying glass on your iPad) so you can zoom in on specific areas of the screen. If you also turn on the Zoom controller (Settings > General > Accessibility > Zoom > Show Controller), you can control the power of the zoom, resize your lens or even add filters. So now, you could create a lens that zooms and gives you a higher contrast! Wow!

In the same area in your settings, you can also Invert Colors (for more contrast) or turn on Larger Text (to increase just the text font size). You can also turn on Bold Text under Accessibility options. This creates bold text in apps, the keyboard and the calculator (not just in eBooks). There's also an option to turn on Button Shapes, which highlights buttons and links throughout the iPad or iPhone making them much easier to see, or you can choose the Grayscale option, giving you another way to make the screen clearer for users with low vision.

How Apple Saved My Life By James Rath

Born legally blind, the 20-year-old Rath says he was severely bullied as a child and had difficulty learning. He attempted suicide when he was 11 years old, and was diagnosed with depression at the same age.

But when he got his first MacBook Pro on his 14th birthday, he found that the computer's accessibility features allowed him to see things he otherwise wouldn't have been able to see. He could read his schoolbooks and zoom in on software that allowed him to edit video.

"My whole outlook on life changed after I got my first Mac," Rath told CNNMoney. "I stopped doubting myself."

9.7-inch iPad Pro unboxing!

9.7-inch iPad Pro Unboxing

In a separate unboxing we take the 9.7-inch iPad Pro - you guessed it - out of the box. We compare it to the iPad Air 2 and 12.9-inch iPad Air.

iPad & iOS9 Accessibility Features Part1

The iPad and other Apple devices that use the new iOS9 operating system (like the iPod Touch and iPhone) received another round of updates in accessibility after the major changes implemented in iOS8. There continue to be some really big changes with the advancement of iOS9 for blind users, however users with low vision have commented that little has changed. One major change has been in the layout of the Accessibility Settings menu and how to customize the available options. So, if you haven't updated your devices yet, now's the time to do it so you may take advantage of the newest features and apps available!

iOS8 built upon many accessibility features with an improved zoom function, a much easier to access text-to-talk feature, a system-wide virtual six-dot Braille keyboard and much more! Now, iOS9 gives you increased customization for your unique needs on your iPad. It's now more accessible than ever!

In this article I'll outline some of my favorite features and link to other articles where you can read about particular features in more detail. Every person has different needs, many of which can be tailored for your child to give her the greatest access possible. Find which features make the most sense for your child so you can create an iPad designed specifically for her!

(Although I'll be writing specifically about the iPad in this article, virtually everything I write here is also true for the iPhone and iPod Touch.)

To read more on Apple's commitment to accessibility, visit Apple's pages on Accessibility or Vision.

A note on commands: In this article I will use the > to denote a set of commands you'll need to use to access features in your iPad. For example, Settings > General > Accessibility means that you would tap the Settings app on your homescreen, then tap Generalfrom the list of options on the left, then scroll down until you find Accessibility.

Topics covered in this article:

1. Simplify the iPad for Hands with Less Motor Control
2. AssistiveTouch
3. Guided Access
4. Switch Control
5. Head Gestures & Other Switch Options
6. Low Vision Options
7. VoiceOver
8. Speak Selection
9. Camera
10. Hearing Impairments

• and a bonus... Braille Displays and Other Equipment!

Simplify the iPad for Hands with Less Motor Control

One of the great things about the iPad is that it can help develop fine motor skills. How? Well, if lightly tapping the screen creates a desired response (like a cow mooing in Peekaboo Barn), then your child will do their best to learn how to control their fingers and get that cow to moo!

However, some kids need assistance in getting to that point, and for them the iPad may seem like a frustrating device that is constantly turning off, changing apps or stalling for no good reason.

The problem is that the iPad comes preset with all sorts of fancy features that allow you to switch between apps or toggle things on and off simply by performing special hand swipes on the screen or by rotating the iPad itself. And these hand swipes or rotations can be performed unintentionally by a child who is just trying to get a cow to moo!

First, go into Settings > General > Gestures and turn these off. Now a tap is a tap and won't activate anything else.

While in General Settings, you can also set your Side Switch to Lock Rotation if you do not want to use it as the Mute switch. Now, when you slide the switch along the right-hand side of the iPad down, the screen will be locked in either a horizontal or vertical scape and won't move around. If you would rather keep this setting for the Mute sound control, you may also access the Lock Rotation setting in the Control Center at any time by swiping up from the bottom of the screen and tapping the Lock Rotation button (looks like a padlock circled with an arrow) in the menu.

Next, go to Settings > General > Accessibility > AssistiveTouch and turn Assistive Touch on. Now your child can use all their fingers or just one finger to do complicated moves on the screen. More on this below.

Finally, go to Settings > Notifications and turn notifications off for all of the apps or customize which apps do or do not allow notifications by going through each app and turning Allow Notifications on or off to your preference. Yes if you turn them all to off, this means you won't be receiving a notification when someone mentions you on Twitter or your friend comments on your Facebook status, but it also means your child won't be interrupted by these notifications when they are concentrating on an app and learning new skills.

Now your iPad is ready for some fun stuff!

AssistiveTouch

AssistiveTouch itself has not changed tremendously from iOS8 but the basics have been expanded upon through two additional settings found in the Interaction submenu of Accessibility: Switch Control and Touch Accommodations. For example, now in Touch Accommodations you can turn off Shake to Undo. AssistiveTouch is still one of my favorite features. Go to Settings > General > Accessibility > AssistiveTouch and turn AssistiveTouch on. You'll see a small square show up in the lower right hand corner of your screen which will remain there wherever you go on the iPad (you can move it into different corners, but it won't disappear). Tap this square to bring up the AssistiveTouch menu (as seen on the right).

The AssistiveTouch menu allows you to do some simple actions all from one place with a single tap, like rotate the screen or "shake" the iPad (a necessary movement for some apps). This menu has also been updated in iOS9 to include quick links to the Notification Center and Control Center. It also gives you access to the Gestures menu which allows you to tell the iPad to read two, three, four or five fingers on the screen no matter how many fingers you actually have on the screen. Why would you want to do this? Well, what if you have an app that requires your child to do a two-finger swipe, but they don't have the fine motor control to do this? You can set your Gestures to two fingers and now no matter how many fingers your child has on the iPad (one or all five, which is usually the case with my son), the iPad will read it as a two-finger swipe. How cool is that? You can now customize it even further by going to Customize Top Level Menu... to add additional icons from a special menu for tasks relating to Volume, Screen Rotation, Siri and many more.

You can also create custom gestures by going to Settings > General > Accessibility > AssistiveTouch > Create New Gesture. The screen allows you to do any movement you want and will record it, from pinching, sliding and swiping to circles, taps, or whatever you can think of. If you're working with an app that requires a very specific gesture, you can do that here and record it. Once it's been recorded, you simply name your gesture and it will now appear in your AssistiveTouch menu under Favorites. Choose the gesture, touch the screen and little blue finger points will flawlessly reenact the gesture you recorded.

To see AssistiveTouch in action along with a tutorial on how to make your own custom gestures, check out this video:

For more information about iPad features for users with motor disabilities, check out Apple's article on Physical and Motor Skills.

Should Apple comply with the order to unlock San Bernardino iPhone?

News broke yesterday that a United States magistrate judge in California ordered Apple to comply with the FBI’s request for assistance in bypassing the passcode lock of the San Bernardino gunman’s iPhone. Hours later Apple published an open letter by Tim Cook explaining that creating a tool to bypass this specific iPhone would jeopardize the security of all iPhones.The battle between personal privacy and information gathering as it relates to Apple and security has been building up for years now, and the government narrowing it down to one specific iPhone used by a terrorist in the U.S. has caused the debate to reach new levels. This may be Apple’s battle to lose, but it will be a very public one nonetheless.Since Apple’s response to the FBI and court order, the White House has stood by the Department of Justice and argued that it’s not about a backdoor for all devices but just a single device, which Tim Cook’s argument already addressed.Tim Cook’s open letter is on Apple’s homepage and headlines about the government’s demands are all over the news.  Tim Cook’s open letter is on Apple’s homepage and headlines about the government’s demands are all over the news. From my view, Apple customers seem to be overwhelmingly in favor of Tim Cook’s position, while presidential candidates are unsurprisingly siding with the FBI. Where do you weigh in? Here’s what we know so far.
iPhones with passcode locks are currently disabled after multiple failed attempts to guess passwords. Try too many incorrect passcodes on an iPhone and you’re temporarily only allowed to place emergency calls for 1 minute. Try again after that and it extends to 5 minutes, then 15 minutes, and so on. Optionally, iPhones can be set to erase all data after just 10 failed attempts.
In Cook’s words, this is how he describes the government’s request:The government would have us remove security features and add new capabilities to the operating system, allowing a passcode to be input electronically. This would make it easier to unlock an iPhone by “brute force,” trying thousands or millions of combinations with the speed of a modern computer.

Voiceover for iOS let voiceover be your guide

Braille Displays

iOS devices are fully compatible with many refreshable braille displays. You can connect a Bluetooth wireless braille display to read VoiceOver output, including contracted and uncontracted braille and equations using Nemeth Code. And braille displays with input keys can be used to control your iOS device when VoiceOver is turned on.

Learn more about braille displays

VoiceOver is a gesture-based screen reader that lets you enjoy the fun and simplicity of iOS even if you can’t see the screen. With VoiceOver enabled, just triple-click the Home button to access it wherever you are in iOS. Hear a description of everything happening on your screen, from battery level to who’s calling to which app your finger’s on. You can adjust the speaking rate and pitch to suit you.

Navigation

With VoiceOver, you’ll use a simple set of gestures to control your iOS device. For example, touch or drag your finger around the screen and VoiceOver tells you what’s there. Tap a button to hear a description; double-tap to activate. Swipe up or down to adjust a slider. Flick left and right to move from one app to the next. When you interact with an element on the screen, a black rectangle appears around it, so sighted users can follow along. When you prefer privacy, you can activate a screen curtain to turn off the display so no one can see it, even as you’re controlling it. And when VoiceOver is talking, the volume of any background audio from music or video automatically lowers, “ducking” under the voice, so you can hear VoiceOver clearly.

Text Input

Whether you’re composing an email or jotting a note, VoiceOver echoes each character on the keyboard as you touch it, and again when you enter it. Have VoiceOver speak each completed word. A flick up or down moves the cursor so you can edit precisely. To help you type more quickly and accurately, iOS supports multiple character inputmethods — including handwriting — and corrects misspelled words. Enable Speak Auto-text and you hear a sound effect and the suggested word spoken. Keep typing to ignore it, or tap the space bar to have your iOS device type it for you. Pair up an Apple Bluetooth keyboard, and VoiceOver talks you through inputting text in the same way.

The Rotor

VoiceOver features a virtual control called the rotor. Turning the rotor — by rotating two fingers on the screen as if you were turning an actual dial — changes the way VoiceOver moves through a web page or document based on the setting you choose. When you’re on a web page, turn the rotor to hear settings like “headings,” “links,” and “images.” Then flick to choose how you navigate the page — for example, you can skip from one heading to the next. Move through documents with similar ease. The rotor has settings like “word” or “character” that let you choose how to navigate your text, which comes in handy for checking spelling and grammar. You can customize the rotor elements from a variety of options, including language.

Apps

Because VoiceOver is integrated in iOS, it works with all the built-in apps, including Safari, Mail, App Store, iTunes, Music, Calendar, Reminders, and Notes. You can also create custom labels for buttons in any app — including third-party apps. So it’s even easier to get to your news, word games, or anything else you love on your iOS device. And Apple is working with the iOS developer community to make even more apps compatible with VoiceOver. 

Learn more about how third-party apps can support VoiceOver

International Voices

VoiceOver includes built-in voices that speak over 30 languages: English (U.S.), English (UK), English (Australia), English (Ireland), English (South Africa), Spanish (Mexico), Spanish (Spain), French (France), French (Canada), German, Italian, Japanese, Korean, Mandarin (Mainland China), Mandarin (Taiwan), Cantonese (Hong Kong), Arabic, Czech, Danish, Dutch, Finnish, Flemish (Belgium), Greek, Hindi, Hungarian, Indonesian, Norwegian, Polish, Portuguese, Portuguese (Brazil), Romanian, Russian, Slovak, Swedish, Thai, and Turkish.

VoiceOver for OS X

Once you learn how to use VoiceOver on your iOS device, you can easily use the same gestures on a Mac. In OS X, VoiceOver allows for gesture-based navigation using the trackpad and supports multiple braille displays. And it features Alex, the voice of Mac, which closely matches how people naturally talk. Learn more about VoiceOver for OS X