I'm the author of the paper. The prototype was never released by Google unfortunately. The implentation we used was a bit hacky, using a visual effects layer over the text editing layer. It was tricky to keep them in sync. A more robust implementation would try to avoid that. I'm happy to discuss specifics in you're interested.
First off thank you for designing this. Both iOS and Android have been focused on streamlining their user experience in the past few years but unfortunately it seems that text editing is just as annoying as before.
In theory how would a "robust implementation" be designed to avoid two layers?
I've given dozens of talks, but this one seems to have struck a chord, as it's my most popular video in quite a while. It's got over 14k views in less than a day.
I'm excited so many people are interested in desktop UX!
I think you did a great job of bringing fairly nuanced problems into perspective for a lot of people who take their interactions with their phone/computer/tablet for granted. That is a great skill!
I think an fertile area for investigation would also be 'task specific' interactions. In XDE[1], the thing that got Steve Jobs all excited, the interaction models are different if you're writing code, debugging code, or running an application. There are key things that always work the same way (cut/paste for example) but other things that change based on context.
And echoing some of the sentiment I've read here as well, consistency is a bigger win for the end user than form. By that I mean even a crappy UX is okay if it is consistent in how its crappy. Heard a great talk about Nintendo's design of the 'Mario world' games and how the secret sauce was that Mario physics are consistent, so as a game player if you knew how to use the game mechanics to do one thing, you can guess how to use them to do another thing you've not yet done. Similarly with UX, if the mechanics are consistent then they give you a stepping off point for doing a new thing you haven't done but using mechanics you are already familiar with.
[1] Xerox Development Environment -- This was the environment everyone at Xerox Business Systems used when working on the Xerox Star desktop publishing workstation.
Fantastic talk, I found myself nodding in agreement a lot. In my research on next-generation desktop interfaces, I was referred to Ink & Switch as well and man, I sure wish they were hiring. I missed out on the Xerox and Bell Labs eras. I'm also reading this book, "Inventing the Future" by John Buck that details early Apple (there's no reason the Jonathan Computer wouldn't sell like hotcakes today, IMHO).
In my downtime I'm working on my future computing concept[1]. The direction I'm going for the UI is context awareness and the desktop being more of an endless canvas. I need to flesh out my ideas into code one of these days.
P.S. Just learned we're on the same Mastodon server, that's dope.
I concur though per my earlier post I do feel "desktop stagnation" is inevitable and we're already there. You were channeling Don Norman https://jnd.org/ in the best of ways.
I'm not into UX much, but listening to someone with such experience and knowledge in their craft, giving a well structured and coherent talk, without shouting or trying to sell anything felt... peaceful?
It's only in the end that I realized I just spent 40 minutes watching the video.
This was a great talk! I'm interested to hear what you believe the desktop UX ('human interface') issues to actually be, today.
Rightly your talk was not about specific issues or specific solutions, but as a desktop user (macOS primarily, Windows secondary but historically, and KDE a distant third), beyond the mishmash of different UIs, i.e. Windows 11 presenting Windows 3.x or just outright dumb decisions such as transparent everything, what is it that you want to solve for people on the desktop space to make them more /productive/ than they currently are? Especially now that our primary vehicle to information creation and sharing is not the desktop, but the web browser alone?
This needs more discussion. Too many don't understand Apple's privacy stance, which has some reasonable elements, is being used to massively foot drag on everyone else trying to build an open web ecosystem.
Passive agressive? (I'm the author) I did research to find a better solution. I did and discovered a wide range of market forces (e.g. people don't like to change) that will make it difficult to ship.
But the last thing I intended was to imply giving up. I said very clearly in the post that my solution, Eloquent, was meant to be one possible solution. There are likely many others! My goal of this post was to clearly describe the problem so people a) understood b) knew there were alternatives and c) we could get fired up to actually solve this (even though it WILL be hard to change)
Aha, I’m glad to hear that! I did misinterpret it after all - thank you for the correction.
I think it was most likely the phrase “shipping something like Eloquent would be challenging”, implying it isn’t going to happen, that gave me that vibe.
That's a fair point, thank you for bringing that up (I'm the author) While these are certainly helpful, the fact that almost no one knows about this proves they aren't really working all that well. Discovery is a key issue that can't be ignored.
This is exactly the point I tried to make in the paper. Teens are using the phone and that's fine, but if you watch them, it's horribly inefficient. I'm not saying stop using the phone, I'm saying your kid can be far more productive.
I'm the author of the paper. That first one though you called out is VERY interesting (the others seem quite close to what is already done today)
We tried pause-to-select and it user tested very poorly as users were thrown into selection far too easily. It was very irritating. However, I can't really criticize it until I've tried it. I'll upgrade my iPad and check it out. Thanks.
I'm the author. I agree, this could be a good way to do it. It's what I meant by a "slow burn" in the post. For example, just start with improvements to tapping and dragging, then add a better magnifier, etc. Just getting rid of some of these ambiguous bugs would be helpful.
I suggest focusing on the minimum viable product: an app with a single text field.
Maybe add the option to load and save txt files, but even clipboard I/O would be sufficient. Just let people play with the editor. If it's actually good, the next steps should become obvious.
