"If we want to order food in a country where we don't know the language at all, we're forced to go into the kitchen and use a see-and-point interface. With a little understanding of the language, we can point at menus to select our dinner from the dining room. But language allows us to discuss exactly what we would like to eat with the waiter or chef."
Ironically, Japanese menus almost universally have pictures of the food, and often (amazingly detailed) plastic models* of the dish in the window.
I frequently wish this was adopted by western restaurants, as being surprised by what actually arrives on my plate after I order is a regular occurrence.
Japanese menus do overwhelmingly include photos, and most of the bigger restaurants have moved to tablets with, of course, images.
The plastic 3d models are mostly in tourist areas.
There are so many restaurants in Japan (especially sushi, soba, tonkatsu, ramen, and izakaya) that have no images whatsoever. Often a handwritten menu or even wall-mounted menus are all that are available.
It's a confirmation bias thing: people come to Japan and go to restaurants with photos on the menu and food models, they think it represents all of Japan. They don't read Japanese, so they may not even know that they walking past a great restaurant without any food displayed outside nor images on menus inside.
> Ironically, Japanese menus almost universally have pictures of the food
For restaurants frequented by foreign tourists, yes. But for most ordinary restaurants, probably not. Handwritten menus with no pictures at all are still very common in Japan.
It may partly a legal thing: Japanese law is big on not-misleading consumers. Depictions are considered close to formal promises of what you'll actually receive, or at least actual ingredients that went into it.
This is opposed to, say... an artistic free-expression of a shared aspirational dream of what the product's platonic ideal, and/or a bunch of things which are meant to evoke the feelings they hope you have in your subjective heart after your joyous nonrefundable purchase.
I'm not sure how well it could be adopted and adapted for American law, but I wish someone'd try.
Note that this is mostly true in cheaper/chain restaurants. Overwhelming majority of izakayas have the menu's to be handwritten / written on the wall without any pictures.
problem with doing the models in western restaurants is they'd have to get them redone every 6 months when they get a new chef because I swear I haven't seen a menu stay consistent for longer than that in ages.
I got to use a real Magic Cap, one of the examples of alternative metaphors, in the article, a black and white view of a room with a desk full of old office oddities. It was the worst user interface that may have ever been designed, like an Alice In Wonderland nightmare. Click an envelope on a desk or a clock, and it starts some other metaphor like an image of a spreadsheet in a dialog, or something, which might appeal to some kind of “grand adventure” logic, but in today’s context…I’ll avoid ending with a negative comment.
I own a Sony Magic Link and you are 100% correct. The UI is the worst sort of point-n-click adventure game memes. Not only do you have to guess what visual elements actually do something you need to figure out what functionality they represent. The spacial metaphor is insipid because it takes a lot of taps to get from one "room" to another.
None of this is helped by how slow the Magic Link is. Supposedly the DataRover 840 was much faster but I've never owned one to tell for sure.
The UI of the Newton MessagePad (I own several) is far from perfect but makes much more sense than MagicCap. It also requires fewer taps to reach different functions.
Every once in a while I'll pull out my Magic Link but the insanity of the UI just inspires me to put it back in a box.
Skeuomorphism was an interesting idea but should have been basically just that -- a design inspiration, not a full fledged embrace of extending old school ways of things.
The folder metaphor for the Mac desktop was reasonable, but it effectively stopped there (as it should), rather than try to embed filing cabinets and document archives into pretty pictures to click and point as if it were a game to be played once rather than a daily driver.
People don't seem to remember, but the finder used to be spatial. You'd open a folder, and a new window would pop up atop your old one.
We can still do this, but thankfully the ui paradigm seems to have advanced to the point that it assumes users don't need to see a stack of every directory they opened. And if they do, Miller columns are superior anyway
LLMs, for better or worse, bring some of that capability to users who aren't fluent in scripting. It allows language centric control, but a couple more layers of abstraction up from direct coding, especially with the rise of agents.
Though, I do find that breaking down instructions into concrete specific steps and validating the LLM output is its own skill that is not too dissimilar to the mindset needed for coding.
This was always the holy grail, we just were never able to achieve this practically.
For example see the early Zork style games, or declarative style languages like SQL. It turns out that it's very tricky to get computers to "understand" language like humans do, and approximating it with pre-defined grammar and vocabulary is not enough.
I think what it describes is about right too: computers programs should have a REPL and we should have agents that can input them for us if we don't want to get into the weeds and wish to automate tasks, in some ways anticipating the browser too.
> Most computer programmers gave up complete control some time ago when they stopped writing in machine language and let assemblers, compilers, and interpreters worry about all the little details.
Ah, there it is. The slippery slope that has stubbornly refused to be slippery for many decades now. Perhaps the author is completely misunderstanding these "metaphors".
What do you mean? With so many average users hopping onboard the LLM train to do what they could basically already do but with less effort (and less control), it seems like the slope's been slippery as predicted.
(And even setting AI aside, I think many people would agree that e.g. Windows 11 gives them less "control" than versions of Windows from decades ago, with the advantage of being harder to break in some ways. Same on the Mac side, and even in the GNU/Linux ecosystem in some ways.)
Abstractions don't give up control. They remove options.
In the best cases, those options were redundant or irrelevant to your goals anyway (compilers most of the time).
In most cases, they add mild inefficiencies (OSes, libraries, frameworks, build tools, etc. and sometimes the compiler).
In the cases of LLMs, WYSIWYG, low-code, etc. you're straight up throwing the baby out with the bathwater and setting the house on fire while you're at it too. Such is impatience and greed.
This distinction between control over the outcome and the available options is no longer as subtle as it once was in the bad old days when everyone was more naive. It is genuinely interesting. I'm not wanting to be negative for the sake of it. I actually think we've had glimpses of more reasonable compromises in the highly constrained by committee environments of app/web dev.
There is a degree to which you can retain control with those higher-level abstractions, but it tends to be just as much or more work to maintain the illusion for their less experienced end users. You end up with more scaffolding than building. This is ultimately why we hire devs anyway and abandon those tools.
"If we want to order food in a country where we don't know the language at all, we're forced to go into the kitchen and use a see-and-point interface. With a little understanding of the language, we can point at menus to select our dinner from the dining room. But language allows us to discuss exactly what we would like to eat with the waiter or chef."
Ironically, Japanese menus almost universally have pictures of the food, and often (amazingly detailed) plastic models* of the dish in the window.
I frequently wish this was adopted by western restaurants, as being surprised by what actually arrives on my plate after I order is a regular occurrence.
I'm fully onboard with see-and-point.
* https://en.wikipedia.org/wiki/Food_model
Japanese menus do overwhelmingly include photos, and most of the bigger restaurants have moved to tablets with, of course, images.
The plastic 3d models are mostly in tourist areas.
There are so many restaurants in Japan (especially sushi, soba, tonkatsu, ramen, and izakaya) that have no images whatsoever. Often a handwritten menu or even wall-mounted menus are all that are available.
It's a confirmation bias thing: people come to Japan and go to restaurants with photos on the menu and food models, they think it represents all of Japan. They don't read Japanese, so they may not even know that they walking past a great restaurant without any food displayed outside nor images on menus inside.
> Ironically, Japanese menus almost universally have pictures of the food
For restaurants frequented by foreign tourists, yes. But for most ordinary restaurants, probably not. Handwritten menus with no pictures at all are still very common in Japan.
I work in food tech in Japan.
> pictures of the food
It may partly a legal thing: Japanese law is big on not-misleading consumers. Depictions are considered close to formal promises of what you'll actually receive, or at least actual ingredients that went into it.
This is opposed to, say... an artistic free-expression of a shared aspirational dream of what the product's platonic ideal, and/or a bunch of things which are meant to evoke the feelings they hope you have in your subjective heart after your joyous nonrefundable purchase.
I'm not sure how well it could be adopted and adapted for American law, but I wish someone'd try.
Note that this is mostly true in cheaper/chain restaurants. Overwhelming majority of izakayas have the menu's to be handwritten / written on the wall without any pictures.
Terry Gillam's take...
https://miro.medium.com/v2/resize:fit:720/format:webp/1*NFPp...
problem with doing the models in western restaurants is they'd have to get them redone every 6 months when they get a new chef because I swear I haven't seen a menu stay consistent for longer than that in ages.
I got to use a real Magic Cap, one of the examples of alternative metaphors, in the article, a black and white view of a room with a desk full of old office oddities. It was the worst user interface that may have ever been designed, like an Alice In Wonderland nightmare. Click an envelope on a desk or a clock, and it starts some other metaphor like an image of a spreadsheet in a dialog, or something, which might appeal to some kind of “grand adventure” logic, but in today’s context…I’ll avoid ending with a negative comment.
I own a Sony Magic Link and you are 100% correct. The UI is the worst sort of point-n-click adventure game memes. Not only do you have to guess what visual elements actually do something you need to figure out what functionality they represent. The spacial metaphor is insipid because it takes a lot of taps to get from one "room" to another.
None of this is helped by how slow the Magic Link is. Supposedly the DataRover 840 was much faster but I've never owned one to tell for sure.
The UI of the Newton MessagePad (I own several) is far from perfect but makes much more sense than MagicCap. It also requires fewer taps to reach different functions.
Every once in a while I'll pull out my Magic Link but the insanity of the UI just inspires me to put it back in a box.
Skeuomorphism was an interesting idea but should have been basically just that -- a design inspiration, not a full fledged embrace of extending old school ways of things.
The folder metaphor for the Mac desktop was reasonable, but it effectively stopped there (as it should), rather than try to embed filing cabinets and document archives into pretty pictures to click and point as if it were a game to be played once rather than a daily driver.
People don't seem to remember, but the finder used to be spatial. You'd open a folder, and a new window would pop up atop your old one.
We can still do this, but thankfully the ui paradigm seems to have advanced to the point that it assumes users don't need to see a stack of every directory they opened. And if they do, Miller columns are superior anyway
This was a thought experiment about UIs if the windows and mouse metaphor hadn’t taken off.
As developers we have the best of both worlds: direct visual manipulation, but also a language-centric control of richer objects in the terminal.
Being able to flip between these has always felt like a superpower.
LLMs, for better or worse, bring some of that capability to users who aren't fluent in scripting. It allows language centric control, but a couple more layers of abstraction up from direct coding, especially with the rise of agents.
Though, I do find that breaking down instructions into concrete specific steps and validating the LLM output is its own skill that is not too dissimilar to the mindset needed for coding.
Mac: "The Power to Be Your Best" (ed. Apple's slogan at the time)
Anti-Mac: “You won't always have to work that hard”
AI (Apple Intelligence) is the Anti-Mac:
https://mjtsai.com/blog/2024/11/12/misguided-apple-intellige...
“It’s really quite a different message than a bicycle for the mind.”
The description of interacting with a computer through language seems oddly prescient of LLMs.
This was always the holy grail, we just were never able to achieve this practically.
For example see the early Zork style games, or declarative style languages like SQL. It turns out that it's very tricky to get computers to "understand" language like humans do, and approximating it with pre-defined grammar and vocabulary is not enough.
I was thinking the same thing.
I think what it describes is about right too: computers programs should have a REPL and we should have agents that can input them for us if we don't want to get into the weeds and wish to automate tasks, in some ways anticipating the browser too.
Great post, thanks for sharing.
I feel that it’s time that we stop spending our focus on designing UX inside boxes, but we are in VR and AR winter at the moment.
> Most computer programmers gave up complete control some time ago when they stopped writing in machine language and let assemblers, compilers, and interpreters worry about all the little details.
Ah, there it is. The slippery slope that has stubbornly refused to be slippery for many decades now. Perhaps the author is completely misunderstanding these "metaphors".
What do you mean? With so many average users hopping onboard the LLM train to do what they could basically already do but with less effort (and less control), it seems like the slope's been slippery as predicted.
(And even setting AI aside, I think many people would agree that e.g. Windows 11 gives them less "control" than versions of Windows from decades ago, with the advantage of being harder to break in some ways. Same on the Mac side, and even in the GNU/Linux ecosystem in some ways.)
Abstractions don't give up control. They remove options.
In the best cases, those options were redundant or irrelevant to your goals anyway (compilers most of the time).
In most cases, they add mild inefficiencies (OSes, libraries, frameworks, build tools, etc. and sometimes the compiler).
In the cases of LLMs, WYSIWYG, low-code, etc. you're straight up throwing the baby out with the bathwater and setting the house on fire while you're at it too. Such is impatience and greed.
This distinction between control over the outcome and the available options is no longer as subtle as it once was in the bad old days when everyone was more naive. It is genuinely interesting. I'm not wanting to be negative for the sake of it. I actually think we've had glimpses of more reasonable compromises in the highly constrained by committee environments of app/web dev.
There is a degree to which you can retain control with those higher-level abstractions, but it tends to be just as much or more work to maintain the illusion for their less experienced end users. You end up with more scaffolding than building. This is ultimately why we hire devs anyway and abandon those tools.