Facebook programs computers to describe photos for the blind

ap.orghttp://www.ap.org

April 5, 2016

FILE - This Feb. 19, 2014, file photo shows the Facebook app icon on an iPhone in New York. A feature rolling out Tuesday, April 5, 2016, on Facebook's iPhone app interprets what's in a photo using a form of artificial intelligence to recognize faces and objects for blind and visually impaired people as they scroll through the pictures posted on the world's largest online social network (AP Photo/Karly Domb Sadof, File)

MENLO PARK, Calif. (AP) — Facebook is training its computers to become seeing-eye guides for blind and visually impaired people as they scroll through the pictures posted on the world's largest online social network.

The feature rolling out Tuesday on Facebook's iPhone and iPad apps interprets what's in a picture using a form of artificial intelligence that recognizes faces and objects. VoiceOver, a screen reader built into the software powering the iPhone and iPad, must be turned on for Facebook's photo descriptions to be read. For now, the feature will only be available in English.

Until now, people relying on screen readers on Facebook would only hear that a person had shared a photo without any elaboration.

The photo descriptions initially will be confined to a vocabulary of 100 words in a restriction that will prevent the computer from providing a lot of details. For instance, the automated voice may only tell a user that a photo features three people smiling outdoors without adding that the trio also has drinks in their hands. Or it may say the photo is of pizza without adding that there's pepperoni and olives on top of it.

Facebook is being careful with the technology, called "automatic alternative text," in an attempt to avoid making a mistake that offends its audience. Google learned the risks of automation last year when an image recognition feature in its Photos app labeled a black couple as gorillas, prompting the company to issue an apology.

Eventually, though, Facebook hopes to refine the technology so it provides more precise descriptions and even answers questions that a user might pose about a picture.

The vocabulary of Facebook's photo-recognition program includes "car," ''sky," ''dessert," ''baby," ''shoes," and, of course, "selfie."

Facebook also plans to turn on the technology for its Android app and make it available through Web browsers visiting its site.

The Menlo Park, California, company is trying to ensure the world's nearly 300 million blind and visually impaired people remain interested in its social network as a steadily increasing number of photos appear on its service. On an average day, Facebook says more than 2 billion photos are posted on its social network and other apps that it owns, a list that includes Messenger, Instagram and WhatsApp.

In a Tuesday post, Facebook CEO Mark Zuckerberg hailed the photo description tool as "an important step towards making sure everyone has equal access to information and is included in the conversation."

___

Online:

Facebook's post: https://newsroom.fb.com/news/2016/04/using-artificial-intelligence-to-help-blind-people-see-facebook/

iPad & iOS9 Accessibility Features Part2

Guided Access

Guided Access is a way to set certain parts of your screen off limits. Are you using a free app with ads and your child keeps clicking the ads? Just draw a square around those ads and suddenly they aren't clickable anymore! Guided Access has not changed since the iOS8 updates, but is definitely a feature worth mentioning for kids with motor challenges or just learning to use the iPad!

Here's how it works: Go to Settings > General > Accessibility and turn Guided Access on. You can set a passcode at this point if you want to (make it something easy, like 1-2-3-4) or you can do it later when you use the feature. You'll need the passcode to turn Guided Access off.

Now when you're in your app (I'm going to use Talking Tom as my example), triple click the home button to open up your Guided Access options. Draw a box around the areas you don't want your child to access (for Talking Tom, I drew a box around the ad at the top of the screen and the buttons that ask you to buy things on the right of the screen). Click start and you're back in the app, but now those areas are not accessible. Those areas are also grayed out a bit so they are less likely to draw your child's attention, too. Another hint is to make sure that you do not draw a box in the middle of the screen where you will need to enter your passcode to turn Guided Access off later!

You can also use Guided Access to set a time limit on apps. Are you using iPad play as a reward, but want your child to get back to another task in 10 minutes? Set Guided Access to 10 minutes and when the time is up, the session will end and you'll have to use a passcode to get the iPad up and running again.

When you're done using Guided Access, triple click the home button again and enter your passcode or place your finger on the Touch ID. Easy!

Switch Control

Switch access for the iPad allows users to interact with one or two large buttons rather than have to maneuver through the complexities of the touch screen. The iPad has always been switch accessible, particularly with apps designed specifically for switches, but starting in iOS7 the iPad now allows for complete system-wide switch control.

Go to Settings > General > Accessibility > Switch Control to turn Switch Control on. You can then tell the iPad how many switches you wish to use and you can fully customize how the switches interact with the iPad, from changing the cursor speed, size and color to adding sound effects and speech options (which is perfect for children who are visually impaired). A feature new to iOS9 is Recipes. A Recipe is a set of specialized actions you can temporarily assign to switches in place of their normal actions. Need to record a series of gestures to perform a complex action in an app? You can make it happen with a Recipe. You can also create a customized Launch Recipe that will be applied automatically whenever Switch Control starts.

For more information on Switch Control, check out this post from Assistive Technology Blog or this very in-depth piece (which includes screenshots and videos) from Jane Farrall's Blog. To learn more about the specific iOS9 updates, read this guide from AbleNet, "iOS9 Accessibility Switch Control - The Missing User Guide."

If you're looking to purchase switches, AbleNet sells a Bluetooth iPad switch that allows you to use either one or two built-in switches or even attach your own favorite switch. There is also the Bluetooth Switch Interface from RJ Cooper. Also, Inclusive Technology creates a series of fun and educational iPad apps that are designed specifically to work with a switch.

Head Gestures & Other Switch Options

Another really interesting feature of the iPad is the ability to do everything a switch allows you to do... but without a switch! Go toSettings > General > Accessibility > Switch Control > Switches > Add New Switch. You have three options here: "External" allows you to hook up a traditional external switch with your iPad; "Screen" lets you use the whole iPad screen as your switch; and "Camera" lets you use head gestures (a move of the head to the left or the right) as your switch. Isn't that neat?

So with "Screen" you can use your iPad just like you would when using a switch, but the touch screen itself acts as a large interactive switch.

And with "Camera" you control the cursor on the screen with a move of your head from left to right. You get to choose which movements control which features in the settings. For more information and a walk through on using head gestures to control the iPad, watch this video below:

Low Vision Options

Go to Settings > General > Accessibility and you have a few options to change the way your screen looks. You can Zoom to magnify the entire screen. It takes a somewhat complicated move (once turned on in your settings, you need to double-tap with three fingers in order to zoom in), but if you can help you child learn to control the zoom it can be a really powerful tool (unfortunately AssistiveTouch doesn't seem to be able to pull off the three-fingered double tap for you).

In iBooks it's a terrific way of showing a child with low vision what the illustrations are and because of how sweetly it functions, you or your child can move through the pictures smoothly, enlarged as required, to find out things like how sad Eyore's face looks or how big Pooh's tummy is.

The Zoom feature underwent a pretty major upgrade in iOS8. Once turned on, you can choose to magnify the entire screen (which is the default setting) or create a Window Zoom. Go to Settings > General > Accessibility > Zoom > Zoom Region and choose Window Zoom. Now, instead of enlarging the entire screen with your three-fingered double tap, you'll just see a zoom lens (sort of like using a magnifying glass on your iPad) so you can zoom in on specific areas of the screen. If you also turn on the Zoom controller (Settings > General > Accessibility > Zoom > Show Controller), you can control the power of the zoom, resize your lens or even add filters. So now, you could create a lens that zooms and gives you a higher contrast! Wow!

In the same area in your settings, you can also Invert Colors (for more contrast) or turn on Larger Text (to increase just the text font size). You can also turn on Bold Text under Accessibility options. This creates bold text in apps, the keyboard and the calculator (not just in eBooks). There's also an option to turn on Button Shapes, which highlights buttons and links throughout the iPad or iPhone making them much easier to see, or you can choose the Grayscale option, giving you another way to make the screen clearer for users with low vision.

How Apple Saved My Life By James Rath

Born legally blind, the 20-year-old Rath says he was severely bullied as a child and had difficulty learning. He attempted suicide when he was 11 years old, and was diagnosed with depression at the same age.

But when he got his first MacBook Pro on his 14th birthday, he found that the computer's accessibility features allowed him to see things he otherwise wouldn't have been able to see. He could read his schoolbooks and zoom in on software that allowed him to edit video.

"My whole outlook on life changed after I got my first Mac," Rath told CNNMoney. "I stopped doubting myself."