NASA astronaut Jessica Meir on how VR can help space missions

Sitting in the window-filled cupola of the International Space Station, looking out at the Earth's curved horizon and then down at the impossible carpet of the planet scrolling by, makes me feel like a bit like I'm in space. But I'm not. I'm watching the 360-degree VR documentary, Space Explorers, on the Oculus Quest. Passing the headset on to my kid, he's equally fascinated.

VR isn't a perfect simulation of being in space. Not by a long shot. But it's also a far more valuable tool than maybe you might think.

Space Explorers is a multipart VR documentary made by Felix & Paul Studios, led by Félix Lajeunesse and Paul Raphaël. The VR production company has already made documentary VR films featuring Cirque du Soleil, President Barack Obama, and created the award-winning Traveling While Black. Their collaboration with NASA has been ongoing, starting with training programs and continuing with a documentary being shot in VR on the ISS.

Felix & Paul plan to take VR outside the ISS on a space walk this summer using a specially modified camera. And, after that, all the footage will be used to create an immersive walk-through experience in a touring exhibition that will allow visitors to explore a 3D re-creation of the ISS and see 3D 360-degree videos projected all around them. A TV documentary using the same footage is also in the works. 

But in many ways, the ISS VR experience feels like just the beginning of a larger relationship between space and VR. VR could eventually be used to document even more distant missions: the moon, or Mars. Or be used for telepresence. Or to help astronauts feel more at home while in space.

NASA astronaut Jessica Meir, featured in the documentary, spoke with me over Zoom about what it was like on the ISS and what filming in VR felt like. Felix Lajeunesse and Paul Raphael also shared thoughts on where things are heading.

One more camera in a sea of cameras

"We're kind of used to that as astronauts already: We're always under the microscope, and we have other cameras on the space station throughout our workday, pretty much always recording what we're doing up there," Meir said of being filmed in VR in space. "It really gives us a way of sharing our experiences, which are so difficult to put into words for other people that haven't had the experience themselves."

The 8K VR cameras Felix and Paul used for the ISS Experience are modified Federal Communications Commission and space-certified versions of the Z-Cam V1 Pro, ensuring thermal and electromagnetic properties would be space-compliant, and that lenses wouldn't shatter. The production has two VR cameras on the ISS and uses one for filming at any time. 

Meir felt that being in the documentary wasn't all that strange, especially since the ISS is already full of cameras, but the possibilities for VR as a documentary memory are unique: "Other astronauts in our office, when they see the ISS experience now, it feels like we're right back on the space station. The first time I put it on, I felt like I was right back there, kind of like being in a place you've lived before. You recognize everything, and you're transported there. It's really incredible."

The VR camera on the ISS was pretty big and was used to continuously record in certain parts of the space station.

"I was up there for 205 days and because of when I arrived on the space station, it pretty much was completely chronicled on the ISS Experience," Meir says. Lower-res footage was seen by Felix & Paul via streaming to get a sense of what was shot, but the complete files were brought down to Earth between missions.

"We would just be there talking to the camera, and we would have some suggested material to talk about, but we could talk about whatever we wanted, really, "Meir says." For her, it will also function as a memory of space even though all the footage isn't being initially used for the VR documentary. "It's really nice now that we have these narratives, even if most of the content won't end up in the actual production. It's a wealth of our own kind of journaling to the camera."

The footage could very well be used to improve training for future missions. "I think almost every astronaut that has had the experience so far, watching it has noted that instantly, saying this would be such a powerful tool for training -- because we just can't really replicate and train for all the aspects of spaceflight. Without having microgravity, without having that truly three-dimensional volumetric space," says Meir. 

"There are things everywhere on the actual space station, " she continues. "There are cables coming out every which way, and it is just so different than what our training facility looks like. Having that sense of realism in terms of being able to look around you and behind you, I think there's so much that this could offer in terms of astronaut training."

Next up: VR space walk

The VR documentary has been collecting footage for more than two years, recording over 200 hours of Expeditions 58-62 and SpaceX Crew 1 aboard the ISS. 

A modified version of the camera will work outside the space station for what should be a five-day shoot later this year, when the camera will be mounted on the ISS' external Canadarm2 robot arm. "That's going to be the first time that Earth gets filmed in ultra high-definition video in a full 360 environment unhindered by anything" Lajeunesse says. "And that will culminate with a six-and-a-half-hour space walk that we will film with two astronauts from the moment they come out of the station until the moment they get back in."

VR can't capture the oddness of zero gravity

Despite VR's benefits as a documenting and training tool, there's one thing it can't simulate: what weightlessness does to the brain. Despite playing a number of "zero g" type VR games and experiences, I'm still feeling gravity outside my headset. Despite on-Earth training equipment to simulate weightlessness, and VR simulations, "It is just impossible for us to put into words how it feels to be weightless all the time," Meir says of her time on the ISS.

"That takes your brain a long time. I mean, I can tell you, if you're eating some soup with a spoon, it is very difficult for your brain and for your hand to be trained to realize that I can just hold that spoon upside down in space. And then it's not going to fall off.

"We say when we arrive in space, we're kind of like newborns, and we have to figure out how to drink and how to feed ourselves and how to go to the bathroom," Meir says. "You can't just put something down, you have to always remember to Velcro it. People in the beginning often misplace things, or things float away. There's so many different surfaces where you could have left things, combined with the fact that every square inch is covered with stuff."

I'm amazed at the calm, almost ballet-like way that astronauts on the ISS can toss food to each other when eating. Meir says it's definitely not as effortless as it looks: "Well, I can tell you that skill is acquired. You have to train your brain because, if you think about it, whenever you throw something here [on Earth], like when you throw a baseball, you are adapting for gravity." Meir describes movement in space as being more like subtle, direct pushes. And sometimes, when you get where you're going, spatial relationships can shift, creating a different sensation of where "ground" and "ceiling" are. 

"On Earth, our brains use this directionality all the time for navigation and spatial awareness," says Meir. "I think that my brain had kind of remapped to use different cues for navigation."

VR as a possible future of space telepresence

My experience visiting the ISS via a VR documentary made me wonder whether this type of experience could ever happen in real time, and if that could enable astronauts to be there via telepresence. "That's something that NASA has thought a lot about as well in the past, particularly in regard to telemedicine," Meir says. "We have a lot of experiments that use ultrasound, for example. I was involved in one of those where I had a team of medical doctors and scientists that were all around the planet, and they were actually guiding me, they had the real-time image of my ultrasound as I was tuning it and could help guide me to get the probe that they needed. With virtual reality, they could have even more of a presence and, perhaps, could make it even easier."

VR as a psychological tool

One idea Meir suggests makes me reconsider the role of VR: While I'm watching astronauts in space at home on a headset, astronauts may use VR to feel like they're at home. "I think that virtual reality will be really valuable in the future for psychological support as well," Meir says. "There were a couple of experiments here at NASA evaluating that, in terms of long-duration missions. There was some VR-type content where you could kind of go on a little trip or nature excursion or pick the environment you were going in. And they had actually taken recordings of my mother's voice, and there were messages as I walked around the environment that I would find from my mother. Something in this 3D, immersive virtual reality could be even more powerful if we're talking about going all the way to Mars, and having some kind of a three year mission, something like that could be a really powerful tool for psychological support for astronauts."

Where VR goes next

Felix & Paul Studios started with this multiyear project shooting VR on the ISS, but the goals afterward are farther beyond Earth. "The next step is the return of humanity to the Moon," says Lajeunesse. "We want to leverage the immersive power of augmented reality and virtual reality and immersive media as a whole, to take hundreds of millions of people to the Moon alongside the astronauts at the forefront of the mission. And then we want to go to Mars."