Niantic is testing Pokemon Go on HoloLens 2 to let you play with friends remotely
CEO John Hanke talks about the leap to AR glasses, and figuring out telepresence.
- Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
Niantic is test-driving playing Pokemon Go with the HoloLens 2 to explore how playing with friends in future smart glasses may work.
Someday, maybe, you'll be playing Pokemon Go in a park, and a friend will find you and ask to join. Except they'll be somewhere else, and they'll join you as a hologram. You'll play side by side. Will they be in your space, or will you be in their space? How exactly will that work?
Pokemon Go defined the idea of real-world mobile location-based gaming back in 2016, and became identified as augmented reality's "killer app" because of its global popularity. Its developer, Niantic, has been extending into more immersive experiments: a partnership with immersive theater company Punchdrunk, and a collaboration with Qualcomm on concepts for future smart glasses. Niantic also announced it's an early partner with Microsoft's new collaborative AR/VR telepresence technology, Microsoft Mesh, and will be using the HoloLens 2 to test out playing the game in AR between people wearing headsets.
AR headsets are different than phones, both in terms of the tech and the metaphysics. Niantic founder and CEO John Hanke, in a conversation over Zoom, admits that the exploration is an experiment right now.
"The experiences that we build, we anticipate a world where there are multiple headsets in the market, there are going to be multiple platforms. And we want to get our experiences out broadly," Hanke says of the upcoming future of AR and mixed reality. "I don't think we're at the point where there's one ecosystem to rule them all."
Hanke sees the HoloLens 2 as one of the best ways to test out experiences that may eventually run on other smart glasses. "It's a great opportunity for us to get in really early and work on this stuff."
Niantic already has AR and world-mapped shared experiences on its phone apps, but Hanke acknowledges that doing the same thing on a headset involves a different type of perception and interaction. "What Microsoft has not done much of with HoloLens, owing to limitations in the hardware itself, is outdoor stuff," Hanke says, and Niantic is testing the HoloLens 2 by pairing it with phones to get mobile connection and GPS and using them in places similar to where phones are used. "Holding up your phone and seeing through the screen is one thing, but when you have the headset on and the Pokemon are all around you, and they're 12 feet tall, it's just magical."
Hanke's on-stage Microsoft Ignite appearance via VR showed Microsoft's mixed-reality head Alex Kipman and Hanke appearing together, Hanke beaming in from a park in San Francisco. Kipman beams over to Hanke, and then a concept of what Pokemon Go in AR could look like is briefly shown.
Hanke says the goal is to work out key concepts in how people would interact: "What happens when somebody reports in from somewhere else? And what do they see? What do you share? It's very much a tool for us to experiment with anticipating future hardware that would be, you know, more suitable for outdoor consumer use."
Microsoft's Alex Kipman appears as a cartoon avatar next to Niantic CEO John Hanke. This is how players might appear to each other.
AR smart glasses seem like they're being developed from multiple corners of the tech world, from Facebook, to Qualcomm, to possibly Apple, Samsung and Google (and, maybe, at some point, Microsoft). But the social dynamics of AR glasses haven't been figured out yet. Privacy issues haven't been overcome. Those considerations, in a game or elsewhere, will need to be resolved.
Microsoft's vision of blended realities and telepresence carries over from AR to VR and even flat-screen phones and PCs, but Niantic is testing its gameplay only on the HoloLens. "Our focus was peer-to-peer on HoloLens," Hanke says. "It's easier from a conceptual point of view, because everyone has the same set of assumptions and interactions. But theoretically, you could imagine a world where it spans phones and AR and potentially even VR, but we have not crossed that Rubicon yet."
Hanke sees the idea of playing with a holographic friend as natural, because in a sense we're with virtual companions all the time -- via audio. "In our neighborhood, everyone's talking on the phone, or listening to a podcast. This idea of having companionship while you're out in the world is very, very natural. Taking that to the next level, having that be a sort of holographic companionship while you're out ... in a sense, it's novel and sci-fi, but you know, in other ways, we're kind of already doing it."
Niantic's partnership with immersive theater company Punchdrunk points to more theatrical, social experiences that could be on the horizon, too.
"That area of live entertainment, thinking about exploding that out into a mixed reality world where you could have people physically present and at a performance, you could map the real show to your house and have real performances happening with holographic players," he says. "People are really interested in trying to figure it out. And I think it's super exciting. Things have been kind of stale in the tech world for a while. This is like a whole new set of toys, particularly when people and co-presence are involved, it just sort of immediately gets exponentially more interesting."