Friday, February 3, 2017

I can see your halos: Further depiction of forthcoming interfaces

A reader suggested that I follow my previous blogpost with some more concrete descriptions of some of what we might see and do with our graphical interfaces. The following may help to create a more dimensionally rich picture, with some 3-dimensional ideas for organizing data (I previously described several 2-dimensional ideas) and some ideas regarding navigating temporally in addition to spatially.

We could start with the idea that you'll often tend to be particularly interested in what's happening in your immediate geographic area. So the interface might initially display a picture/map of your current location at, say, the neighborhood scale, from which you can zoom out to the scale of continents or galaxies, or zoom in - maybe to the scale of your face, with the help of your device's camera. So you can navigate to any geographic location you like.

Maps/images depicting real-world objects will be connected to images depicting more virtual objects (with the help of something like the Mixed Reality Service); we'll swim smoothly between the worlds.

Toward the edge(s) of the screen can be what we could call the relatively "meta" items. If the main or central part of the screen contains a chunk of cyberspace, filled with contiguous items that are related fairly directly to whatever you're engaging with or exploring at the moment, then out by the perimeter you can have items that are relevant and useful in more general, indirect, abstract ways. So this outer area could be like a halo of meta-items: frequently used resources/tools, updates, messages/alerts, etc. This area can serve functions similar to our current social media, email, browser bookmarks, toolbars, window menus, the array of frequently visited pages that our browser generates when we open a new tab, and whatever else we tend to consult frequently. 

So around the main part of our display, we may have this halo, which we could call the "main halo." Many objects within the display could also sprout halos of their own, containing meta-items specifically relevant to the object, like related topics, or functions that are particularly applicable to the object. A halo can hover a short distance away from its object, or maybe encircle it, more like a hula hoop, or the rings of a planet. And of course items within a halo can have their own halos, and so on.

Consider the complex sequences of images that flow through your mind when you're absorbed in a work of fiction or nonfiction -- not the images your eyes are taking in of the text itself, but the images in your mind of the meanings behind the text, evoked by the text. Well, with sufficiently advanced graphical ontologies, millions of books (and other texts) could be automatically converted to graphics, so that we can imbibe their meanings by traversing a path, and looking, instead of reading. As we traverse these graphicalized works, we can stop and make changes when we see ways the translation can be improved, if there isn't yet AI that's become impeccable at this job.

Now let's say you're traversing a sequence, or thread, of information, and when you're partway through it, you want to stop your traversal temporarily and do something else within the interface. Maybe you're stopping in order to make improvements, as just mentioned; maybe you want to explore more deeply some item in the sequence that you don't understand very well; maybe you're responding or reacting to something that has appeared in your main halo; maybe some tangential, or seemingly unrelated, thing has just occurred to you and you want to follow up on the thought before you forget it. Once you've taken care of the digression, how do you get back to where you were in the original thread? If you remember how you got from that point to where you are now, maybe you can spatially navigate back to it fairly easily. Maybe you zoomed in when you were digressing, and now zooming out will get you right back to the thread. But if the path back to the thread is more complicated, or if you want to go back to someplace more temporally distant, maybe a place to which you hadn't specifically intended to return until now, then you may want to navigate temporally. (You may not know exactly where to go to get back, but you may have a good idea of when you were there.) The system will remember where you've been, so you'll be able to rewind and fast forward through your history. This seems likely to come in handy frequently; you may keep your temporal navigation tools somewhere prominent in your main halo.

While there will be privacy and solitude available in this new environment, it will offer multitudes of new ways to meld with others. We'll have radically more flexible ways to videoconference, we'll be able to "tether" one person's point of view to another's so that the one can follow the other's peregrination through cyberspace in real time (sharing the same trip, as it were), etc. We'll all be inhabiting and co-creating this immense mixed-reality environment. All around us will be structures constructed via massive cooperation of people and machines.

By the way, in writing about the output of these interfaces, I've written mostly or completely about the visual aspects. The visual seems likely to keep playing a prominent role, but there may also be amazing things we'll be doing with sound, and kinesthetic feedback, and other modalities.


Josh Maurice said...

Please share this with, for instance, programmers who might be interested in launching an open-source graphical interface project.

If we start with a platform equivalent to Minecraft or Roblox, in that it allows for creating 3d worlds and interacting with others in them, then we could try to make it possible for objects in the 3d worlds to be nested within other objects indefinitely. We'll want objects to be easily sharable between worlds. We'll let the processes of creating custom worlds, and of connecting and sharing those worlds with others, take place in-game, inside the 3d worlds -- even if there's still a lot of text involved in such processes, we'll be interacting with the text within a larger graphical environment, in such a way that we can build more and more graphical structures to take over more and more of the functions for which we've been using text.

Such an environment, if it becomes the prevailing way for people to use the internet, will get us all, in almost a literal sense, on the same page. It seems to be a realistic way of solving the problems of information poverty and information overload -- allowing us to access and manage the voluminous amounts of information that are relevant to us as navigators through these fascinating and pivotal times. Information will percolate through to everyone, beautifully, appropriately, agreeably

Josh Maurice said...

I gave a kind of 2-dimensional idea of how the "main halo" could work, in this post. We could also imagine it working in a more 3D way, hovering above/around your head, body, or point of view.