Connecting the dots: VR, AR, BCI and space

These past few weeks were remarkable: Facebook-people talking about Full Augmented Reality Glasses and Elon Musk about his Neuralink company, which wants to develop implantable brain–computer interfaces (BCIs). The goal is not just to treat brain diseases but to enhance the human being so that it can compete with AI-powered robots. Even the direct transmission of thoughts – rather than speak or type – would become possible. More about this can be read in the extensive article on Wait But Why.

It somehow makes sense. If you live in an augmented reality (virtual reality being just an option of the Full AR Glasses), and you want constant interaction with the Machines (your virtual assistant, all the electronic affordances you can imagine) it would be very convenient to be able to do so by just thinking. Once you can do that, why not transfer thoughts from one person to the other?

Longevity

Once we get near perfect telepresence – summoning people to “be” here right next to me (as a kind of holograms) and to be able to look around as if they were actually here (which they would be, in a sense) – we get used to the Other in a spectral form, hence why not dream about beating death itself? Longevity is yet another ambition of Silicon Valley – Alphabet and others investing heavily in the struggle against disease and decay.

Space

But wait, there are about 7.5 billion people on this planet, when we eradicate diseases, avoid using doomsday weapons, lengthen life expectancy dramatically, the population growth will increase dramatically which could become very uncomfortable – so it really makes sense to explore space and establish colonies in space or on neighboring worlds such as Mars.

Politics and ethics

We could throw in easily lots of other new technologies – everything related to smart cities, food production in extreme environments, identity management (blockchain…) and so on. So let’s not be surprised that some of the most passionate debates about politics, philosophy and ethics emerge from this constellation of disruptions. A few examples: transhumanists split in a left- and a right-wing, researchers want to expand human rights to protect us from the abuse of neurotechnology. These debates did not yet go mainstream, but eventually they will. One can only hope that an informed debate will be possible – even though the current state of political discourse makes me feel pessimistic.

Full AR Glasses will augment the homo sapiens

This is a fascinating presentation by Michael Abrash, Chief Scientist at Oculus/Facebook, during the F8 conference. It’s about nothing less than augmenting the capabilities of the homo sapiens. He advocates full AR glasses and boy, they go far beyond Pokemon Go on your smartphone. Abrash refers to J.C.R. Licklider’s famous paper Man-Computer Symbiosis (1960) to underline the importance of the new developments.

The full AR glasses will give us better vision and hearing, will make us more intelligent, productive and connected. Yet they will be stylish, power efficient and socially acceptable. That will be necessary as they will be a constant part of our lives.

The glasses will know about our surroundings, history and needs. They will blend the physical and the virtual world according to our needs and desires.

Unfortunately, they do not yet exist. It will take breakthroughs in materials science, perceptual science, graphics, AI. So give it five or ten years, maybe longer – but imperfect versions will be available sooner and then develop just like happened with the personal computer.

It implies stuff like new brain-computer interfaces allowing us to think our instructions for our tiny but powerful artificial assistent. What Abrash did not say, is that there will be a divide between those owning and using the glasses efficiently and those who don’t have the glasses or don’t use them in a productive way.

Facebook Spaces – in search of applications

Facebook launches the social virtual reality thing Facebook Spaces.

It seems to include Messenger (friends who don’t have an expensive Oculus Rift) can join through “windows”), 360 degrees video, real life pictures and Snapchat-like filters. Very mixed reality, but of course there are some challenges here:
– It’s based on Oculus Rift which is not exactly attracting a broad user group
– It’s based on the use of avatars while in this day and age we’re using all sorts of video services to communicate showing ourselves in real life.
– There is not much to do (Wagner James Au on New World Notes).

That being said, Snapchat has its Spectacles, Facebook and Apple probably work diligently on AR/VR glasses, some see the end of the smartphone, so I can imagine a VR/AR fusion in a few years time.

In such a scenario, what application can we imagine which makes a difference to humanity – beyond birthday parties, funny faces and people vomiting rainbows?

Being surrounded by people who love drawing mindmaps and even developing mindmap-tools (Metamaps) I’m wondering about the possibilities of creating 3D-mindmaps, eventually with live updating nodes, and navigating through and around those structures in an immersive environment.

That’s just one, admittedly rather nerdy, possible future development – there must be many more out there.

 

Don’t say “augmented reality”, say “paint the world”

Snapchat introduces World Lenses, cute 3D objects which can fit in about everywhere.

In the meantime Facebook announced during its F8 Developer Conference that augmented reality is ready to go mainstream.

One obvious point to note is that Facebook is just following Snapchat here. The other: the clever guys at Snapchat, who are masters in detecting trends, carefully avoid to use the words “augmented reality” in their communication. They talk about “painting the world” or something like that.

Could it be they realize the words “augmented reality” are cursed already – not to mention “virtual reality”?

Yuval Noah Harari asks tough ethical questions

Screenshot of the Time Well Spent site

Screenshot of the Time Well Spent site


I’m reading the book Homo Deus these days, written by Yuval Noah Harari, the author of another bestselling book,Sapiens: A Brief History of Humankind. Homo Deus is about “a brief history of tomorrow”, it goes back to the hunter-gatherers and leaps into a largely unknown future of a possible successor of the homo sapiens.I won’t try to review the book here, as others did so very well.

The book made a big impression on me, so I was delighted to discover a long article by Harari in The Financial Times, in which he challenges the “the future according to Facebook” – reacting on Mark Zuckerberg’s Manifesto about the Global Community Facebook wants to build.

I think this discussion is very crucial for those of us who are interested in VR, AR and MR, because Facebook is a big player in these fledgling industries and Zuckerberg has a very articulated vision on the importance of all these realities. In “Building Global Community” Mark Zuckerberg explains how much value the users of his network get from participating in specific groups/communities (parenting, patients… ) and how important this can be also for their offline lives. Harari is critical:

(…) he never acknowledges that in some cases online comes at the expense of offline, and that there is a fundamental difference between the two. Physical communities have a depth that virtual communities cannot hope to match, at least not in the near future. If I lay sick at my home in Israel, my online friends from California can talk to me, but they cannot bring me soup or a nice cup of tea.

People have bodies, so Harari reminds us, and we don’t have (yet?) the means to virtually recreate the depth of physical presence. The problem with Facebook and its business model is that it needs its users to use its services as much, as long and as intensely as possible – even when that’s not in the best interest of the users.

He refers to the designer/philosopher Tristan Harris, an ex-Google person who nowadays promotes the Time Well Spent movement. He wants to make people aware of the design of tools such as smartphones (or headsets I guess) and he asks designers some inconvenient questions: in whose interest do they work? Will they stimulate people to connect to the physical world around them, or will they serve the interests of the shareholders of Facebook, Netflix and other similar companies who have a vested interest in being as sticky as possible?

I think this should not be read as a damning condemnation of all things virtual or augmented. One can imagine virtual experiences which open our eyes to the realities around us, and even more so augmented and mixed reality inciting us to explore the physical world (remember Pokemon Go?). What I like about Harari and Harris is that they confront us with ethical choices in our exploration of VR and AR.

Mixed Realities Musings on Facebook

I launched a page on Facebook about the same subjects I cover here on this blog: virtual worlds, virtual reality, augmented and mixed reality.

On MixedRealitiesMusings I’ll post links to this blog but also to posts and videos on other blogs and sites.

The very first visitor to post on the new page was Howard Rheingold:

MixedRealities lives also on Twitter.

Big tech in love with VR and AR, so we update our mindmap

Crazy times for virtual reality, or so it seems. ‘It’s the future’, people at Facebook say. Users will not only be able to experience immersive Facebook-experiences, but will also create them, Eric David posts on SiliconAngle. ‘Users’ means not just developer-types, but the average user. However, it is totally unclear when virtual Facebook-apps would hit the market.

While Facebook executives dream about immersing all of us in their product, some people at Apple share that dream for their own products. Apple was awarded a patent titled “Head-Mounted Display Apparatus for Retaining a Portable Electronic Device with Display,” which details a virtual reality headset powered by an iPhone or iPod, according to Apple Insider. The device does look not fundamentally different from other systems strapping your smartphones on your face such as Google’s Cardboard or Samsung Gear VR (powered by Oculus).

But we may still have to wait a long time before Apple presents a consumer-ready device, and that device could also be something rather different from Oculus and be more along the line of Microsoft’s HoloLens – as one could think looking at patents filed two years ago, BusinessInsider says.

Since I started a wiki mind map about Oculus Rift developments, quite some people added useful stuff to it. I now added these latest developments (use the icon with the arrow under the map to maximize):



Create your own mind maps at MindMeister

Oculus/Facebook wants to track your fingers and environment

In case you still have any doubts: virtual reality is hot. News from the Digits blog (The Wall Street Journal) by Timothy Hay: Oculus/Facebook bought two start-upsNimble VR, which was formerly known as 3Gear Systems (San Francisco), they are specialized in skeletal hand-tracking using tiny cameras, and Swedish start-up 13th Lab which uses cameras making 3-D recreations of various physical environments.

Here you see some magic from Nimble VR (video from the Kickstarter Campaign). Notice the Minecraft game and the school examples but of course there is much more in this:

13th Lab shows some nice videos about SLAM or Simultaneous localization and mapping. This is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent’s location within it. This is one example where the technology is used for the recreation of a stairway:

This is a game application by 13th Lab:

The post on Digits mentions also various companies in the VR industry getting venture investment such as Survios (resulting from Project Holodeck at the University of Southern California), Jaunt (toolset for cinematic VR) and Virtuix (a company founded by the Belgian Jan Goetgeluk and specializing in a VR treadmill, Omni).

I’m pretty sure that the way we interact with the digital world and how we integrate the digital into the physical is about to change dramatically.

How one gets tired of virtual worlds

Reacting on my previous post about Second Life veterans uniting, draxfiles asked on this site and on Facebook:

Interesting how some folks get “tired” of a world unlimited in its creative possibilities, a world that is continuously mind-boggling in the sense that folks make awesome things and yes: a LOT of them are a lot better than what was done in the hype days. The creative forces that are marveling at the tools that Linden churns out [xp tools, ALM model, etc] and use them = I am in awe of them and the monthly video show is not nearly enough to cover this bustling life. I simply do not understand how you get tired of it: it is like saying “I am tired of this pencil and this piece of paper!” :) Come on the show Roland and help those of us [in the audience of the show] who are pushing the envelope every day understand what this means!

Well here we go. Of course I agree that new tools are making new things possible (such as using Second Life on a mobile device or on old machines), and more is to come when Oculus Rift will release its VR-headset on the broader market.
However, people get tired because well, that’s life: people evolve, move on to new jobs, expand their families and end up with less time and energy. But there is more: Second Life and virtual worlds in 2007/2008 were technologically speaking less sophisticated, but there was this feeling that ‘this is the new thing’: the online world was evolving from 2D to 3D, and websites or social media feeds would become just a part of this encompassing 3D-metaverse.

Embassies, corporations, news wires and other media, religious institutions entered Second Life because it was the future. Then we all know what happened: the end of the hype, mass traction not materializing, corporations and institutions did not find out how to use this new space. The new headlines now claimed that Second Life was dying or that is was just “a niche of a niche”.

While it’s not correct to say Second Life is dying (I hope), I think it’s true to say that it’s a niche-activity. Will it finally gain traction during the Oculus (or similar devices) era? I’m not sure. People are tired and stressed out: this is an era of fast dipping into media feeds (Twitter, Facebook, Whatsapp, SnapChat… ) not of long immersive media-experiences (at least when you’re not a young student). This argument has nothing to do with technological breakthroughs but with how we structure time in this first part of the 21st century.

So, when we speak about people getting tired of immersive worlds, this is what I mean: we have such limited time and right now and quite possibly for a long time these worlds will remain a niche industry. While you can probably find a job quite easily as a Facebook-marketing expert, I’m afraid it will be difficult to market your skills as a Second Life guru.

This being said, yes I’m impressed by the new energy and developments which are to be found in virtual reality and virtual worlds (both concepts being rather different). So, can one be impressed and tired at the same time? I think so.

Second Life veterans unite

I remember, in the good old days of the Metanomics show in Second Life, that we discussed a study about what happens when people have to leave a virtual world. You get a virtual diaspora, groups of people settling in new virtual worlds, and eventually you get tensions between those groups and people who already ‘live’ there.

Second Life is not closing down or forcing huge groups out, but there is this other phenomenon that people get tired after a few years. They look for something else and lose interest, or at least they reduce the time spent in Second Life. However, for many of those ‘old’ residents, Second Life changed their lives in some way, and they feel a need to meet up with their old friends and former fellow-residents.

That’s what happens on Facebook, where veterans launched the group Second Lifers for Life. People compile lists of names, remember those who passed away, discuss what they’re doing these days. Maybe it’s not really like a diaspora but more like former classmates meeting again. There is even a little bit of drama, with people complaining about the initiative and telling others they should look at the future, not the past. But then again, maybe it’s by meeting again that new projects will be born.