Facebook's new Portal smart displays: Who's listening and what's happening to your data?
The tech giant compromised your privacy and broke your trust. Now, it has a new set of "Smart Cameras" and always-listening microphones for your living room.
- 10 years product testing experience with the CNET Home team
Facebook introduced three new versions of its Portal smart display on Wednesday -- a "Mini" version with an 8-inch screen for $129, a $179 version with a 10-inch screen (the same size as the Amazon Echo Show and the Google Nest Hub Max), and a standalone, Kinect-like camera accessory that costs $149 and lets you use your entire TV as a Portal device.
Read more: No one should buy the Facebook Portal TV
Each comes with an AI-powered "Smart Camera" that can track you as you move about in the frame during a call, and each comes with the same microphones for voice-activated controls as the originals. You can say, "Hey, Portal," to wake it up and make a video call or any other number of functions, and you can say, "Alexa," to access the full capabilities of Amazon's digital assistant, too.
Read: The best smart displays of 2019
After numerous privacy fails and outright scandals, Facebook is trying to reassure consumers that you can trust it with your data.
That means that each Portal also comes with privacy concerns. After all, this is Facebook, a company that recently received a record-setting $5 billion fine from the Federal Trade Commission for its privacy failures, as well as a $100 million fine from the US Securities Exchange Commission. Each of those stemmed from Facebook's March 2018 disclosure that the political consultancy Cambridge Analytica had, years before, improperly accessed the personal data of up to 87 million Facebook users. Cambridge Analytica now stands accused of using that stolen data to influence consequential elections around the globe, including the 2016 presidential campaign and the UK's Brexit vote.
Perhaps worst of all, Facebook broke your trust when it failed to notify its users of the colossal breach until long after it learned about it, and only after the New York Times and the Guardian were set to run stories on the scandal.
"For more than two years, Facebook's public disclosures presented the risk of misuse of user data as merely hypothetical when Facebook knew that a third-party developer had actually misused Facebook user data," the SEC said.
In other words, anyone considering bringing into their home a Facebook-branded device with always-listening microphones and an AI camera really ought to consider Facebook's privacy practices, first. So, let's do that.
Is Facebook eavesdropping? (Is anybody not eavesdropping?)
Facebook's Portal devices use the same sort of microphones as other smart displays, which means they're always listening for the wake words (in this case, "Hey, Portal"). When you say the wake words, the device activates and begins recording the audio of whatever you say next. From there, the device sends that audio snippet to Facebook's cloud in order to figure out how to respond.
That's the same way other voice-activated smart assistant gadgets work, including Amazon's Alexa devices, Google Home smart speakers and Google Nest smart displays, and assistants like Apple's Siri and Microsoft's Cortana, too. It's also how those voice assistants work when you use them on your phone.
The logical follow-up: What do these companies do with those audio snippets, some of which may contain background conversations not intended to be shared? Is anybody listening to them?
As it turns out, the answer is yes. Both Amazon and Google admitted earlier this year that they had hired contractors to listen to recordings like that in order to help refine their respective assistants' capabilities. Apple and Microsoft soon followed suit. Human review of user audio was a standard practice across the board for all of them.
The Facebook Portal and Portal Mini smart displays send audio of your voice queries to Facebook's servers in order to figure out how to respond. Facebook stores that audio and may listen to it in order to refine the capabilities of the device. You can opt out of having your audio stored and potentially listened to in the device settings.
From there, Apple announced that it would only allow for human review of Siri voice recordings after users opted in. Meanwhile, Amazon now allows Alexa users to opt out of human review in the Alexa app. Google put the practice on pause after German regulators banned it outright. Microsoft says that it's stopped listening to Xbox recordings, though it seems that Skype translations are still on the table.
And Facebook? The company admitted that it, too, paid contractors to transcribe Messenger calls for the purpose of improving the service. After a brief pause, that practice is back on as these new Portal devices launch, though now you have the option of saying, "no thanks."
"If you have storage enabled, which is the default, then those [audio snippets] may be reviewed by humans, by a team of trained reviewers to improve voice services overall," a Facebook spokesperson said. "Those, of course, can be deleted individually, or you can also just disable storage entirely, and then they can't be reviewed or stored."
That's a good option -- but many might prefer to have human review off by default, and available only for those that opt in. So far, Apple is the only company that claims to take that approach.
OK, so who's listening?
In most cases, companies like Facebook that have acknowledged human review of user audio say that they hire outside contractors for the job.
"There are vendors who are really specialized at this and who are really good at this," a Facebook spokesperson explained. "Getting the right people to help review voice transcripts makes the service a lot better. It makes it more inclusive."
But Facebook adds that company employees may listen to user audio, too.
"There are specialists out there who are vendors," the spokesperson added. "We're happy to use them. They're under very strict protocols in terms of how we engage with them. But we also use a mix of employees when it's appropriate."
And what constitutes "appropriate?" I asked Facebook, but haven't received an answer yet. I'll update this space if that changes.
Each Portal device includes a physical shutter that can cover the camera lens when you aren't using it.
Can I cover up that camera?
Yes. You can cover each Portal's camera with a physical shutter when it isn't in use.
Good for Facebook -- that's better than the Google Nest Hub Max and the Amazon Echo Show, each of which skipped the shutter in favor of a digital kill switch that disables the camera electronically without covering it. Amazon at least seemed to realize that consumers were more comfortable with a shutter when it added one to its most recent smart display, the mini Echo Show 5.
As for the Portal, Facebook adds that the camera's movement-tracking features and the microphone's audio enhancement capabilities are all processed locally on each Portal device, and never sent to Facebook's cloud. Meanwhile, Messenger calls are encrypted in transit, and WhatsApp calls are encrypted end-to-end. Facebook stresses that it does not listen to, view or keep the contents of your Portal video calls.
What about when I'm not using it?
All of that is good -- but unless you've switched them off or closed the shutter, the camera and microphone are always on. Are they gathering any other data on you?
"When Portal's camera and microphone are on, which you can control, we collect camera and audio information," reads Facebook's supplemental data policy for its Portal devices. It goes on to describe the data that's collected whenever you make a call, which Facebook says is the same data collected from other Messenger-enabled devices during calls.
"This information can include volume level, number of bytes received, or frame resolution," reads the current policy. A new Facebook privacy policy document, which takes effect on Oct. 15 (the launch day for the new Portal and Portal Mini) revises the language as follows:
"When using Portal's camera or microphone to make a call, we collect technical information about your call, such as volume level, number of bytes received, or frame resolution."
Smart displays let Amazon, Facebook, Google show you answers to your questions
See all photos
That sounds innocuous enough, and in the latter version Facebook specifies a bit more to characterize the data as purely technical information, but I still default to skepticism when a company uses terms like "can include" or "such as" to detail the various types of data it collects. After all, both phrases connote a nonexhaustive list, which raises an obvious follow-up question: Can that list include anything else?
And what about when you're not making a call? If the device is just sitting there plugged in on your kitchen counter, is it quietly taking notes on your day-to-day routine?
I asked Facebook these questions and several others about the data policy, but have not received answers. I'll update this space if that changes.
In the interest of transparency, I've pasted the text from Facebook's data policy for Portal devices into a Google Doc and annotated it with the exact questions I asked the company with respect to each section. You can see that document, along with what Facebook did and did not answer, by clicking here. As of writing this, the company hasn't answered any of my questions, but I'll add those answers to each annotation as I receive them.
Facebook currently faces a class-action lawsuit over its use of facial recognition on photos uploaded to its social network. It says that the cameras in Portal devices don't recognize faces at all.
Is Facebook tracking my face?
Facebook has a long history with facial recognition. Since 2010, the company has used the technology to identify the faces in photos uploaded to its social network, which you can opt out of. Last year, in Illinois, where the collection of face data is regulated under the Biometric Information Privacy Act, plaintiffs brought a class-action lawsuit against Facebook alleging that the company violated user privacy by using facial recognition without explicit consent. Last month, a federal appeals court rejected Facebook's bid to have the case tossed by a 3-0 vote, exposing the company to billions in potential damages.
But the cameras in Facebook's Portal devices don't use facial recognition technology at all, the company says.
"Right now we understand faces, but we don't understand your face," a Facebook spokesperson explained. "So, we can look at the features like eyes and mouth and things like that. It helps us place things like when you put on an AR mask, knowing how to overlay it on top of your face."
Facebook's Portal devices track facial features like noses and eyes in order to position augmented reality masks over your face during video chats. But they don't track or recognize individual faces, Facebook says.
That distinguishes the Portal devices from the Google Nest Hub Max, which uses facial recognition in order to show individual users personalized bits of information from their Google accounts. Google calls the feature Face Match, and initially told us that all of the processing for it happens locally, on each user's device. That's true in real time, when you're using the feature, but the product's fine print makes it clear that your face data may occasionally get sent to Google's cloud so it can ensure the feature works in multiuser households, and so it can test new features before pushing them to users' devices.
"We occasionally use the images you provide during setup to generate a face model in the cloud for a couple of reasons, all related to improving your product experience specifically on Nest Hub Max, and motivated by the fact that we have more computing power available in the cloud," Google said.
"This only uses the images originally taken on the phone during the setup process," a Google spokesperson emphasized. "Any processing on our servers is temporary, and all face models processed in the cloud are permanently discarded. The only feature that has access to the enrollment images and subsequent face data is Face Match -- this data is not used for anything apart from this feature on the device.
Is this thing going to affect what ads I see?
Yes. Whenever you use Facebook or a Facebook device like the Portal, part of the fine print is that you're agreeing to share data with the company, and that the company may use that data to target ads at you that may be relevant to your interests.
"We collect the same information that other Facebook properties collect," a Facebook spokesperson said. "So, when you're using Facebook Watch, for example, we'll know what you're watching and we'll use that to prioritize what things to show you. And there are things that will be used for our ad system there."
The Portal's primary use case is video calls -- do the calls you make impact the ads that make it into your feed?
A video call using Portal TV.
"The best way to think of this honestly is it's like a mobile phone," a Facebook spokesperson said. "And it's running Messenger, it's running Facebook Watch, and it's running WhatsApp. And so it's using the same services in the same way that your phone uses those services. And so when you're using Messenger, we have an awareness of, you know, who the call was initiated by and who received the call and how long the call was. And you can imagine it informing advertising on Facebook. Like, for example, 'this is a person who makes video calls.'
"That's relatively unlikely," the spokesperson added. "The amount of data we're generating here is very, very small. It's certainly not material. It's certainly not the point of the product. But it's also something that could happen."
Will Facebook share my data with the police?
Something else that could happen -- law enforcement showing an interest in people's Portal data. It's happened before, when the FBI wanted to force Facebook to wiretap the Messenger voice conversations of MS-13 gang members. Police have shown a strong interest in data collected from other devices, too, including Alexa speakers and wearable fitness trackers.
In the case of the gang members' voice calls, Facebook argued that it couldn't carry out the FBI's demands because Messenger has end-to-end encryption, meaning that it'd have to completely rewrite the app to do so. In the end, the Justice Department wasn't able to make Facebook break its own encryption, even after reportedly trying to hold the company in contempt of court.
The current version of Facebook's data policy for Portal devices makes no mention of law enforcement -- but the new version, effective Oct. 15, does.
"We may also share voice interactions with third parties where we have a good faith belief that the law requires us to do so," it reads.
Originally published Sept. 18, 2019.
Update: Oct. 4, 2019: Added additional comment from Google explaining its Face Match privacy practices.