Avatars as digital personal assistants

Imagine that you don’t have to search for each individual app, select it, then use it according to its own little system. Instead you would speak to your device (smart phone, watch, headset, whatever) using natural language, and the thing would do whatever you instructed it to do. You could ask for a high quality French restaurant in a certain neighborhood of your city at a certain time and specify that parking should be easy and safe. Your device, or rather the “digital personal assistant” in it, would mobilize a number of apps and databases and book a table. That would be the era of the post-app, so Richard Waters explains in the Financial Times, writing about Artificial intelligence: Digital designs for life.

I wonder what form that personal assistant would take. A cute robot? Or rather something very small, almost invisible? In a virtual world setting such as Second Life and OpenSim it could be an avatar without a real life typist behind it, but chatting away like Siri responding more or less intelligently to questions and remarks. Such a bot could be the representation of a personal assistant.

However, such an avatar does not have to be confined to a traditional computer screen, just imagine what could be possible using Magic Leap or HoloLens. It would make our future digital personal assistant even more interesting…

The anti-Minecraft

Lots of Second Life-people don’t like Minecraft. They consider it too primitive, too childish or whatever. I disagree, I think it’s a fantastic game helping young and not-so-young people all kinds of digital literacies. Joseph Flaherty Wired made me discover another game with Minecraft-like aesthetics, but with themed worlds which can be fully destroyed. It seems the worlds are not persistent, they are generated anew by starting a new session, yet no two sessions can ever be the same.

The new game is called Moonman and was funded via Kickstarter.

William Gibson captures the (gloomy) mood of the time

Don’t miss the latest episode of The Coode Street Podcast as it features author William Gibson. One of the topics is of course his latest book, The Peripheral, a story set in multiple futures. I guess ‘multiple futures’ sounds complex and the first part of the book is indeed bewildering. Wikipedia has a nice entry about The Peripheral (spoiler alert). I found the book intriguing because of the gloomy times we live in – think terrorism, war in Ukraine, societies in crisis, climate and environmental tragedies. In The Peripheral Gibson refers to a major crisis wiping out a big part of humanity, mysteriously called The Jackpot.

There’s no detailed account of what happens during The Jackpot except for some references to the climate. Yet the way in which Gibson talks about the Jackpot seems remarkably realistic:

And first of all that is was no one thing. That is was multicausal, with no particular beginning and no end.

A bit further it is described as

androgenic, systemic, multiplex, seriously bad shit

The climate change, caused by there being too much carbon, is an important driver of the catastrophes. People in the past “fucked it all up” so it is explained, at first they did not know what was going on and then they were unable “to get it together to do anything about it”.

But it’s not only the climate and the climate has multiple consequences. Like I witness it from our newsroom the world is an extremely interdependent, complex and delicate system breaking up at various places – environment, culture, society, economy, finance. It’s not an analysis I talk about here but a certain mood, and Gibson captures that mood brilliantly in his book.

If you wonder what a “peripheral” is in this context: it’s a cyborg avatar that users can connect to from another location. (Wikipedia)

Microsoft’s HoloLens delivers a new level of augmented reality

Dieter Bohn and Tom Warren at The Verge had an amazing mixed realities experience at Microsoft. They tried out the HoloLens, a headset that projects holograms into real space. They report about having Minecraft on a coffee table and about an amazing experience during which an engineer who was communicating via Skype helped a reporter to fix a light switch, not only by talking but also by actually drawing on ‘his reality’. Sounds very weird but have a look at the video:

The applications go far beyond gaming of course: all kinds of support and learning situations could benefit from this.

Immersive journalism puts you in the middle of the news

How would immersive journalism look like? Immersive as in ‘immersing the reader/viewer/participant into a situation in such a way she learns something new’? Actually there exists a website immersivejournalism.com where journalist and scholar Nonny de la Peña discusses her work and ideas about this topic.
I first encountered this work of hers in Second Life in 2010:

But you’ll notice how she evolves and now uses virtual reality tools for her newest projects, about food banks…

…or about Syrian refugees:

Immersive video

Now I learned about a project by another team “which will be centered on live video, delivering an experience that feels more like documentary and photo journalism than a console game.”

They use new experimental cameras which are able to capture live motion 360° and 3D virtual reality footage.

The kit is made from 12-16 cameras mounted to a 3D printed brace, and then stitched onto a virtual sphere to form a 360 degree virtual environment. These new kits are also stereoscopic, adding depth of field.

The project is a partnership between Frontline, The Secret Location, and the Tow Center. You can read more about it on the Tow Center site.

Author Neal Stephenson joins Magic Leap as Chief Futurist

Neal Stephenson is the author of Snow Crash, a book which helped me ‘get’ virtual worlds. Today he announces in a blog post that he agreed to become Chief Futurist of Magic Leap, which is a secretive company working on something which seems to get rid of keyboards and other clunky computer stuff in order to make things appear out of thin air. As Snow Crash was all about virtual reality and augmented reality (in 1992!), it seems to make sense to ask such a visionary author to become Futurist of this secretive company.

This is how Stephenson describes Magic Leap:

Yes, I saw something on that optical table I had never seen before–something that only Magic Leap, as far as I know, is capable of doing. And it was pretty cool. But what fascinated me wasn’t what Magic Leap had done but rather what it was about to start doing.Magic Leap is mustering an arsenal of techniques–some tried and true, others unbelievably advanced–to produce a synthesized light field that falls upon the retina in the same way as light reflected from real objects in your environment. Depth perception, in this system, isn’t just a trick played on the brain by showing it two slightly different images.

This is what our author hopes to do:

I’m fascinated by the science, but not qualified to work on it. Where I hope I can be of use is in thinking about what to do with this tech once it is available to the general public. “Chief Futurist” runs the risk of being a disembodied brain on a stick. I took the job on the understanding that I would have the opportunity to get a few things done.

He adds that Magic Leap is “not exclusively about games. It’s also going to be a great tool for readers, learners, scientists, and artists. Games, however, are a good place to start talking about why this tech is different.”

Keep an eye on his blog!

(Hat tip to Colin Dwyer on the two-way for bringing the news).

Smartphone-based VR headsets open up new possibilities for learning

Google CardboardConnectivism guru Stephen Downes is right to point out alternatives for Oculus Rift such as Google Cardboard. You can buy these things (which allow to convert smartphones into VR devices) from others or build it yourself. I never tried it out myself so I cannot compare with Oculus Rift, but I see the pedagogical advantages of building oneself a virtual headset. Downes refers to a blog post by Donald Clark: 7 learning principles that work in VR (one of those principles being “learning by doing”).

Samsung GearIn the meantime Samsung Gear VR is now officially available in the US, it launches internationally ‘early 2015′, so Road to VR says. The interesting thing about these smartphone-based VR headsets is that they allow for short-term immersive experiences, untied to desktops or laptops so the devices are easier to share.

Immersive experiences for short attention spans

One of my doubts about immersive media such as virtual worlds is that they are often extremely time-consuming – either you ‘get it’ and stay for hours in-world or you don’t and you leave without ever returning. In this new mutation of immersive media however one can dip into an immersive environment – for instance to enjoy a song in a VR setting as suggests Maria Korolov on Hypergrid Business:

Obviously, the killer app for virtual reality is going to be music videos. The length of a song is just about the right amount of time for an easy introduction to VR, and after you’ve watched it, you can pass it around to all your friends.

Now imagine the typical short educational videos used for Massive Open Online Courses (MOOCs) – one could imagine using lightweight mobile VR headsets for accessing short immersive experiences in this context. I first had to smile looking at the smartphone-based headsets, but now I realize they open up very new possibilities.

Robots, VR Headsets and Virtual Worlds are made for each other

People don’t get ideas, they make them. That’s the pedagogy of constructionism explained in the shortest possible way, I guess, and Colin Lewis at RobotEnomics is very good at that. He posted about why employees should be playing with Lego Robots – because it makes it so more obvious what the Internet of Things is all about, more so than by letting people watch Powerpoint presentations.

However, you don’t need to despair if you don’t have a robot around. There’s something called Robot Virtual Worlds which is a high-end simulation environment that enables students, without robots, to learn programming.

Now let’s take another step and use the Oculus Rift, like these guys:

I’ve no experience at all with Robot Virtual Worlds nor do I have any information about which virtual robot programs are compatible with which virtual headset, but it seems obvious that virtual headsets, robots, virtual worlds and programming lessons are made for each other.

Five conditions for a perfect Virtual Reality experience

I had some very nice Oculus Rift experiences. I tried Titans of Space, an exploration of our solar system. The head tracker of the DK2 proved very useful: I could use the dashboard of my small spacecraft by nudging and tilting my head. Here you find a video by VR Review:

You find Titans of Space on the share section of the Oculus site.

Another beautiful experience is Lighthouse Lost Mansion which seems to be an adventure in several episodes. Very beautiful start scene on a rock in the sea with dark skies. I found this download at The Rift Arcade Market where you can actually buy Rift experiences (or sometimes get them for free).

I also continued my exploration of Second Life using Oculus Rift. I went to the Linden Endowment for the Arts (LEA) and walked around at the Welcome Area (SLurl, requires viewer download) and visited Sister Planet (SLurl), to thoroughly enjoy the scenery. Looking up at the vegetation above my head was very nice – really very immersive and realistic. Here you see a picture of the place in a normal view:

Sister Planet on Second Life

I got the idea of visiting these areas on the Oculus Rift DK2 Intergalactic Space Station (SLurl) where you’ll find more suggestions for Rift inspired visits. The Intergalactic Space Station also lives on Facebook.

Even though I’m just a beginning Rift explorer, I’ve some ideas about what is needed for a great virtual reality experience.

– The obvious: being surrounded by a high quality environment.

No hassle. Right now different downloads often require tweaking display settings of your laptop or desktop.

– A bit less obvious, but feasible: great 3D audio.

Haptics! Have a look what this could mean these days (hat tip to Chris Baranluk on NewScientist):

– But what’s also needed is a social dimension. I saw very nice Rift cinemas. You can sit in a very nice virtual cinema and watch a movie, but ultimately what would be fun is meeting others. The social dimension is also very much something the new owner of Oculus Rift, Mark Zuckerberg, wants. Ultimately the Rift must bring us together.

Right now, in this very early phase, most Rift experiences seem to be a bit spooky: the visual effects are powerful, yet without haptics and others who join the experience online these experiences lack some fundamental dimensions.

The fact that one can experiment with the Rift in Second Life is cool for exactly this reason: even though the environment is far from perfect (there is a reason why Linden Lab builds a new virtual platform), the fact one can actually meet other people in en open-ended environment is fabulous.

This could be a competitive advantage for Linden Lab: the fact they already work with this huge virtual world community and have quite some experience with being a platform for many different communities.

Do you see other conditions for a good VR experience? Let me know!

Oculus needs smaller avatars in Second Life

This is the Berlin 1920 area in Second Life is testing for Oculus Rift. They scaled part of the environment to fit the Oculus Rift immersive environment. It also implies one should downsize one’s avatar (avatars are typically unrealistically big in Second Life) – but one can use the Oculus Rift headset everywhere in this virtual world, even though it is not really optimized for it. What I noticed here was that some objects did not render in my Oculus (some furniture such as a couch in an otherwise interesting Berlin 1920 house).

picture of the test area of the Berlin 1920 area in Second Life

This is the notecard about this place:

Welcome to the Oculus Rift and Real Scale Test area.

In this little corner of Second Life you can explore a street, a bar and a house that have all been build to a real world scale.
A lot of places in Second Life have been build to different ideas of scale, often just guesses or estimations based on how large some of the avatars are.
By using a realistic scale, things feel more natural.
We use the scale of the “prim”, the building blocks of Second Life, translating real world scale straight into Second Life Centimeters.

When using the Oculus Rift, realism and realistic scale becomes very important.
You will be seeing Second Life trough the eyes of your avatar, while normally you would see the virtual world trough a camera view high above the head of your avatar.
This makes visiting a lot of places in SL a strange experience as doors and ceilings appear to be made for giants.

This Test area allows you to see what the use of realistic scale looks and feels like without having to change your avatar or your avatar’s clothes so you can visit the actual 1920s Berlin sim.
If you would like to visit an entire city build to this scale and with immersion as one of its main goals, please change into some of the (free) 1920s clothes and get on the train behind the little station.

If you want to return to this place at a later time, you can use this slurl or this landmark;
http://maps.secondlife.com/secondlife/1920s%20Berlin/243/207/753

I’d appreciate it if you wrote down your experiences and shared them with me, perhaps for my blog.
And if you’re interested in discussing the potential of using the Oculus Rift in Second Life, join us on facebook;
https://www.facebook.com/oculuslife


With regards,

Jo Yardley
The 1920s Berlin Project

It sure is a place to keep an eye on…