Programming is in tension between the Light Side and the Dark Side.
The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!
The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.
Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
I'll paraphrase someone who commented on HN once (about dynamic vs static typing, IIRC): permissive languages are enablers for solo programmers, but you need more restrictive languages for team programming.
This matches my experience because you don't chose who you work with. Skill level is uneven among the team. Less skilled co-workers will make mistakes that will have more consequences when the language is more permissive.
That's one of the reasons you want code reviews, but then you convert your skilled programmers to teachers. This is not a good trade, because the time they spend improving the skills of other programmers is partially wasted because programmers are free to come and go. AI could change this picture.
The name of the game is therefore to keep your teams small and skilled, but this is can be difficult because of typical company politics - e.g. "bus factor" considerations, shortening time-to-market is often achieved by adding more programmers.
> Less skilled co-workers will make mistakes that will have more consequences when the language is more permissive.
Even the most skilled engineers are going to make these mistakes sometimes. That's the whole point of more restrictive tooling, and why Rust has exploded.
Don't get me wrong. I love Lisp, and use it in some personal projects. But I wouldn't want to use it in must not fail scenarios unless those situations can wait for a human to fix the running image.
Yes. I'm moving the goal post a bit here, but I was actually thinking about design mistakes and dangerous constructs - the kind of thing you do when you write in a "hacky" way - either deliberately or because you don't know better. Although these are not directly under the scope of type checking, a more restrictive language can have a positive influence.
I agree with you. Though I wonder if the divide is more about the size of the program. I've got a couple of large (>100K LoC) solo projects and I like using C++ because it is much easier to catch bugs at compile time.
Static typing is a huge advantage in refactoring, and if you don't refactor a large program as it grows, you end up with a mess.
>This matches my experience because you don't chose who you work with.
That's what job interviews are supposed to be for. It should be easy to hire only the most skilled programmers, especially with the massive number of unemployed programmers out there right now.
Programming skill is similar to driving skill. You can hire the best people in the world and there'll still be incidents. The people who are most skilled may even be more incident-prone because they have the skills to take more risks.
Don't bet the company on no one ever making a mistake. Set up a system where the unintentional risks are minimized and the consequences of inevitable failures are mitigated by seatbelts and insurance policies.
I know this is obvious trolling, but it’s close to a truth. Selecting for a passion for languages that aren’t anyone’s first language gets you polyglots—folks that’ve had to abstract their knowledge and understand what’s an ecosystem feature vs what’s more broadly applicable.
Joking about Rust users rhymes with joking about vegans; it’s true both parties generally can’t keep from talking about their choice. But you do want to hire someone who’s thought about their options and made a decision.
Lisp generally has precise GC, which I'd say makes it light side. It's even relatively type-safe if you count runtime type-checking.
Highly reliable systems are written in Erlang, which if you squint is another Lisp dialect. There's even a sexp-based version called LFE, for Lisp-flavored Erlang. Erlang's key to reliability is error recovery, rather than exceptional levels of error prevention.
I do like your light side/dark side classification.
Neither use as an implementation language nor syntactic similarities make one language a dialect of another. Consider that Java is commonly implemented in C++, and its syntax was designed to be easy to learn for C++ users. Yet nobody calls it a dialect of C++, not even when squinting really hard.
Erlang's computation model is functional, like Lisp, unlike Prolog.
Define a goddamn language, syntax is not enough to define one! What are the semantics? Without that you are just talking about syntax trees like they would mean anything
never understood why people say that: the syntax for defining code seems quite different from the syntax defining data structure. There's no homoiconicity in javascript..
And that's just syntax, it doesn't give you a programming language at all.
JS is a dynamically typed language with prototypical inheritance objects that work like universal key-value maps for the most part. It is also mutable.
Clojure is a dynamically typed language with key-value maps. It is also immutable.
You can surely see where I'm going , the underlying semantic model is the meaningful part. Homoiconicity doesn't give you anything special if your language can parse itself and can eval code. It just makes these completely abstract implementations simpler.
> Homoiconicity doesn't give you anything special if your language can parse itself and can eval code. It just makes these completely abstract implementations simpler.
Well, in a way it does give you something: By making expression of things like macros simpler, it makes them sometimes worthwhile, and makes it a reasonable request to have this kind of meta programming in your language at all. Without homoiconicity such things become even more difficult endeavors and often unjustifiable for the language design and its implementation.
I mentioned in other comment that homoiconicity doesn't necessarily make writing macros simpler. It makes writing trivial toy examples simpler.
But let's compare it to a modern macro system like rust's or scala's, where you get a typed object representation of the AST, and for anything non-trivial you are better off with this latter.
Also, arguably the best is to have certain features in the language itself, that can be used to build proper abstractions - so you don't have to resolve to using macros in its place.
> Also, arguably the best is to have certain features in the language itself, that can be used to build proper abstractions - so you don't have to resolve to using macros in its place.
I don't agree, because that would mean, that the language must be huge, or grow huge over time, or alternatively be extremely abstract at its core, to be able to fit every use-case. Furthermore, so far I have not seen a language, in which the language designers managed to pull it off, so I tend to think that what already has been successfully pulled off, which is macros for language extensions, is the way to go.
Also this seems to be arguing from a limiting idea about what macros do. Macros are not always the right solution for any problem, in fact often they are not, but there are things you simply cannot do otherwise (without syntactic clutter), like for example changing the order of evaluation.
I also don't agree with your point about only making toy examples easier to write. For example I have written macros for implementing new define forms, which allow to specify contracts for function arguments and return values, or a macro for automatically defining functions that communicate to API routes, based on the function name, which I used to implement a proof of concept docker client for Scheme. Those are not toy examples, but real world applications, where a small macro can have big effect.
If you think macros need to be big and elaborate and complicated and otherwise are toys, then you don't really understand the power of macros. One of my favorite macros is the following threading macro:
Small, but a great addition to the code, that improves readability in many places of the code. It doesn't have to be big or long, in order to not be a toy example, but actually be a useful macro.
Well, my main point is not to never use macros, but that in certain cases a language-native feature that can also solve a problem you would use a macro for, it's probably better (better debugging, error messages, etc).
And my other point was macro systems in other languages, which I believe are better, in part due to not having homoiconicity, like Scala or rust.
Yes, that is true. I'm not big on the idea that Lisp is defined by parentheses. The implementation strategies are another way to look at it. That doesn't capture it either, but it's an angle to try.
So then what it is? Because otherwise it's a magical nothing-term to which everything lisp people like applies, but no criticism can ever reach it because "that's not really lisp, see it's different in this other implementation"
In my mind, it's 2 distinct criteria: interactivity and the manipulation of symbols (whether those are implemented as symbols or identifiers doesn't really matter). I don't know about Erlang, but from the admittedly little JS I've written, I believe it achieves both criteria, even if it's worse at them than a "proper" Lisp.
I think the big thing is that Lisp is (aside from mutable variable assignment) basically all declarative, rather than the imperative paradigm.
Even without static types, and even allowing macro craziness, there's just such a stronger baseline of declarative and functional thinking, you're off to such a good start in clearer thinking and reasoning about a program.
This is just not true. The Lisp family contains all conceivable variants of languages, from statically typed Coalton, to dynamically typed Scheme. From functional, immutable by default Clojure to the imperative-style of Common Lisp.
There are prologs, constraint solvers, ffis, JavaScript alternatives, garbage collected and not garbage collected languages that call themselves lisps.
I won't disagree but most books and people who made lisp in the early decades were somehow hinting at not manipulating state but trying to represent ideas and operators to raise the level of expressiveness without shooting yourself in the foot (as much as possible)
I think this depends on the Lisp, no? AFAIK Common Lisp supports a lot of imperative style programming--besides all the mutation/assignment functions, there's the `prog` macro that lets you use goto, `do`/`do*` to iterate over groups of statements, or even the `loop` macro. OTOH the Scheme-style Lisps are much more declarative thanks to TCO and a community that prefers the functional/declarative programming style.
But again, I suppose all the Lisp forms return values, and quibbling about the declarative : imperative :: expressions : statements mapping is just petty semantics
> But again, I suppose all the Lisp forms return values, and quibbling about the declarative : imperative :: expressions : statements mapping is just petty semantics
I would argue that it's very much not just semantics, or at the very least it's certainly not petty. The distinction has a noticeable effect on what the experience of writing code and on the reliability and related properties of the written code.
Yes and also how you use it. The more you squeeze performance out of CL the closer the code looks to C in s-expression format, the less you squeeze it the more it looks like a DSL. The bridge between the two is macros so that you can have both the performance and expressiveness.
And you have a good point. Maybe it's a weakness in the metaphor if one can see it either way.
I fundamentally believe that there is no perfect language, instead one merely chooses a set of trade-offs. What's interesting about Lisp is that it is simultaneously well-respected and yet not widely used in productions (compared to C, C++, Python, JavaScript, and even Rust).
It's almost like Lisp chose a particularly extreme set of trade-offs that yield great power but also repel large swathes of programmers. I think PG might say that it repels dumb programmers, but maybe not.
Our entire profession has been moving more and more in the "bondage-and-discipline" direction over the past 10 years or so, tracking with broader sociological trends that are also justified with arguments involving "safety". That by itself can explain why Lisp has such low adoption. While Lisp has never been the most-popular language, it was never as unpopular as it is today. If that pendulum swings the other way, Lisp will get more popular again.
I think it’s just that the syntax is more difficult to grok. When I first saw code (PHP and JS) it was fairly easy to read and figure out what was going on. I’m now experienced but I’ve never tried Lisp, but reading Lisp code takes considerably more effort than when I first saw the PHP/JS. I’m sure with very little experience it becomes just as easy, but to the novel eye I think it’s simply less intuitive to reason about.
Light and dark is propaganda depending on which side you're on.
I'm with you, on the dark side I've seen people slowly turn into crabs, screeching about memory safety and static types. Can you imagine that? Now stick those wings on me, I'm going to fly close to the sun.
Programming is in tension between the Light Side and the Dark Side.
The Light Side believes in giving people freedom and power to achieve freedom, to allow themselves to express themselves better through programming, to become better programmers through the tools the light side provides; in the Light Side believes in programming as a way of thinking.
The Dark Side believes that people are incapable of handling freedom and must be restricted and controlled under the service of various organizations - generally companies. The companies will control access to programming and control the programmers via this access. The dark side does not believe in making programmers better programmers, it believes in managing programmers. The dark side does not believe in helping programmers to think, it believes in thinking for them.
I'm not big on static type checking in most situations -- adequate testing should find the type errors too -- but one place it looks very useful is avoiding data races in multithreaded programs. See Rust.
What side for python? Haskell? Idris? Clean? C? K?...
white/dark side is too much of a simplification to me.
Programming language equilibrium is a very complex subject and fascinating one.
After years of experimenting with many of them, it is all a matter of context of usage, personal preferences, objectives, etc.. there is no such thing as "the best programming language ever", neither "best for light side programming" nor "best for dark side programming".
Similarly, there is no such things as: Light side/Dark side in artistic painting. There are tons of paints some are clearly greater than others, but there is no such thing as "the best painting most beautiful for everyone seeing this paint comparatively to any pain".
Malbolge is not an instrument for building, come on programmer.
C is clearly the dark side with to chance to protest.
The only way to consider K as dark side is its speed constraints relative to my favorite PL which is J. K is so fast because its interpreter fits into CPU cache.
Haskell is a border case between sides because the idea of PL is so light that using so much light in programming leans to be really dark. Haskell extracts the cognitive friction from the code environment to the types environment, programming in paper is not a joke for Haskell.
If there is any language that has ever been, Python is the very definition of light side, so much so that anything that needs to be done in the cover of night, is purposefully directed elsewhere by rather shady dark siders (C).
I have always assumed that Alan Kay compared Lisp to Maxwell's Equations because of the similarity between the interplay of eval and apply in Lisp on the one hand, and the interplay of the electric and magnetic fields in Maxwell's equations on the other.
But there are two types of “programming”: 1) programming per se (just you writing code for fun and perhaps building your own tools) and 2) a team of people with real-world constraints writing software for others for revenue
It’s clear why it feels so good to use something like Lisp for personal projects. It’s also clear why the vast majority of companies don’t use it
You have it inverted. Your "Light Side" is what previous generations of programmers called "Bondage and Discipline", because this philosophy assumes that a good programming language has built-in shackles to impose the will of a so-called "benevolent dictator" on all programmers.
It isn't nearly as much of a trade-off as people say. Languages like Haskell are both remarkably expressive and provide a lot of safety. (And, of course, languages like Python, Java and Go are the opposite.)
Language design has almost nothing to do with language popularity.
People learn JavaScript or TypeScript because they want to write web apps. Swift or Objective C to write iOS or Mac apps. SQL because there’s a database they need to get data out of. Python because there’s a machine learning library they need to use. Etc etc.
Language design is far down the list of priorities.
For one, there might be trade-offs against dimensions other than safety and expressiveness. Performance, corporate support, libraries, popularity, learning curves, similarity to other languages...
For two, popularity is simply not strong correlated to quality. Popularity is a social function driven by, well, social factors. It's heavily path-dependent and noisy. There is absolutely no guarantee, not even close, that the "best" thing in any sense will be the most popular, or popular at all.
I would refine that: Language design has only indirect relevance to language popularity.
People go where the money is, hence python. But python is widespread because there is employment in it. That adoption by employers follows a network effect, but it originally came about through the language design.
Early-adopter => mainstream => long tail
Adoption hinges on that jump from evangelists to the mass market, and that implies:
- attracts early adopters because of language design
- early adopters can pitch the language to their bosses by showing cleat benefits, also because of language design
The most obvious exception is javascript, which was inflicted on the world by browsers. Even so, it had to be sufficiently shit for the php crowd to feel at home. (Unsure which came first, but you get my point.)
Php is exactly popular due to design. It was an exact fit for the myspace dotcom era, and there were no json APIs to illuminate its shonkiness. (I am still angry that [ ] serialises to either [ ] or { } depending on a fucking global variable. I cannot think, being maximally charitable, of any value that comes from conflating arrays and hashmaps. No, i do not want an array containing the keys 0 and "0". I can count on the fingers of one foot the times i have wanted that.)
Sorry - my therapist encourages me to rant about php. She said it's part of healing. I haven't even told her about back-end php yet.
When Python got its popularity, programming was an in-demand skill, so the thing that determined adoption was being easy for beginners to learn. Python was extremely beginner-friendly, so it became popular.
Now, the thing that determines popularity is popularity. So Python still wins, even though its beginner friendliness isn't so important anymore.
It may very well be the case that it makes trade-offs that turn of lots of programmers but I don't think it's popularity directly implies it by definition, rather just that it implies there's something about it that prevents lots of programmers from taking it up en masse, and I think that's perhaps just the fact that it is so different from what everyone's used to and it takes some time to adapt to it enough to be able to appreciate the tradeoffs.
for a while there's been a strong cultural tendency in programming to mistrust people and trust tools (often even blindly because tools are just assumed to make no mistakes), and so expressiveness has been sacrificed for safety. There can be something to this but it's also self-fulfilling, if you strip people of agency of course they'll unlearn to program.
I've always thought it's a misanthropic philosophy and sucked much joy out of programming but also there's something to be said that there's safety in the expressiveness of Lisp. I read an article a while ago that found a robust correlation simply between length of a code base and errors regardless of languages used. And given how succinct and clear Lisp codebases can be that's valuable in itself.
we're kind of at the logical endpoint of this now with gigantic slop codebases that nobody understands just held together by 20 different tools, and if you ask me if I had to pick one of those or something one tenth the size written by a guy or girl who has been writing Lisp for ten years I'd say thank you I'll go with #2
The issue is also social, the average software engineer will treat Lisp code as a liability because most people really don't know what to do with it. In the mind of a professional SE, code is "maintainable" only if it's writing in Javascript or some other commercial language.
Yep, you have to use C# and Java because they are safe. If you want anything extra, you need to use $library, because only $library's maintainer is trusted with the dark corners of the language. Look how powerful we are, millions of libraries. Look at those Lisp guys and their sorry examples of package managers. /s
If there's one thing I've been happy with learning Common Lisp and Clojure is that you only want libraries for things like algorithms, data formats, and protocols/interfaces. Mostly because those often have standards. Anything else should be a snippet to copypaste even when presented as a full program.
That is a very powerful idea, but unfortunately it cannot be realized with a fixed set of language rules. Some invariants are universal, but some are bound to the domain. Some projects require, say, an int (EvenInt) to take on only even numbers and others odds (OddInt). You can imagine the possibilities here are unbounded ("the numbers should be prefixed by numbers that are divisible by my age squared").
Ideally your base language has the expressive power to formulate new abstractions and constraints. This fundamentally requires the Dark Side. Once the proper primitives are in place a different "language" - this can be literally or figuratively - is used to express the interplay between those primitives.
In effect you are starting with the most general (say, all of Lisp), restrict it to become more and more and specific - only "these modules" - until it cannot be reduced any further. If you can bring your domain down into being expressible as, say, a single config file, that'd be quite ideal. If you don't need the powers of abstraction to express your solution then exposing said powers would only invite in trouble. The mathematical equivalent of introducing unnecessary variables.
As a programmer you are free to traverse the journey from say all of Lisp to a JSON config file in whatever way you please.
The "Light Side" people have converged, or try to converge, on some intermediate state between full powers of abstraction and configuration only. I think this is useful because many if not all problems travel through that intermediate landscape on the way "down" (into their specificity). For example what you call "types" is a significant restriction on your freedom but at the same time it's so enormously general that this restriction can be considered a worthwhile default because just about any problem I can think of can potentially benefit from that restriction.
All this is to say that I don't think it's a dichotomy so much as two interacting polarities whose interplay gives each its strength.
Lisp is a particularly minimal base introducing very few restrictions of its own. But it's not the only one. Forth would be one in another direction.
Well put. Minor quibble: I see what you're driving at with "inexpressibility" but i think i could write EvenInt and OddInt in F#. I don't know Haskell, but i believe you would have to work hard to find a domain constraint that you can't declare in the Haskell type system.
You are, of course, right about EvenInt being easy to express in a type system. In fact, I believe type systems are generally so expressive as to allow just about any constraint to be expressed (eventually..).
I notice a lot of arguments on this topic resolve to "but Turing", which is not completely uncalled for but I think misses the point a bit. Not because it is wrong, but because it highlights the wrong property.
I don't doubt, say, Brainfucks ability to express any arbitrary computation, but I do doubt its ability to do so sanely. Now you may say Haskell's type system is very clean, but some constraints will definitely push it out of its comfort zone. I'm not saying it won't be able to express them, but I am claiming there will be dragons. One example of a constraint that's at least awkward is when the absence of a fixed type is part of the design. There are of course myriad solutions to this problem but they generally all require not quite so straightforward constructions that eventually might make sense with enough exposure, but whose complexity can actually be disproportionate to the value of the guarantee.
It's very much a testament to the genius of languages like F# and Haskell that you have to think hard of practical counter-examples.
That said, I actually think the Dark/Light polarity rears its head again even deep inside Haskell as any sufficiently complicated software system encounters barriers it needs to overcome and those require carefully constructed escape hatches: unsafePerformIO, metaprogramming, that is to say, the (relative) Dark Side.
In that metaphor undisciplined use of Lisp is like a vast, dark gravitational field of possibility and Haskell is like a sea of light with some carefully marked dark patches.
I think of C as more of a dark side language because it's basically a portable assembler. Lisp at least prevents you from doing the stupidest things like overrunning array bounds or writing to freed pointers*, which C is perfectly happy to let you do.
* Of course there are no user-visible pointers** in Lisp so the concept of free vs. not free doesn't come up because memory allocation and garbage collection are automatic.
**Under the hood most everything in Lisp is a pointer, but the user cannot see them or mess with them.
I dislike when people use the term “power” to describe making code slightly shorter to type. Operator overloading is not in any meaningful sense more powerful than a language which lacks it.
You say that because you lack experience in writing programs for scientific-technical computing, where an abundance of complicated mathematical formulae are necessary.
Most programmers have seen only programs that do more data movement and comparison than actual computation, but nonetheless there are programs with abundant computations, the kind of programs that were originally written in Fortran, but nowadays they may be ported to other languages, or alternatives for them may be now written in more modern languages.
In such programs, overloaded operators do not make programs "slightly shorter to type", but they reduce the amount of text at least an order of magnitude or even much more.
The main advantage is not that you type less, but that you can read the source of some function in one page, so you can see it in its entirety, instead of having the source spread on many pages, forcing you to wander through all those pages continuously, when you try to understand what it does and whether it does it correctly.
I have written such programs, for instance which were full of long formulae where most variables were complex vectors or complex matrices (for computing some radiated electromagnetic fields with the method of boundary elements). Without the operator overloading of C++, reading them would have been extremely tedious and maintaining them would have been very error prone.
For myself, any programming language that lacks operator overloading, e.g. Java, is disqualified, because it definitely is not "powerful" enough.
Operator overloading also increases the safety of a programming language, because you can define distinct data types for each kind of physical quantity, allowing the compiler to detect and reject the invalid operations. Without operator overloading that would inflate too much the size of the source text.
> where an abundance of complicated mathematical formulae are necessary
I think what you mean to say is, a lot of linear algebra is involved. So multi-dimensional structures like matrices and vectors are expected to have linear arithmetic operators defined over them, because they're used quite a lot.
You don't need operator overloading for that. Your language should be providing them as primitive structures and extending operators over them. Think like how C has a polymorphic addition, where '+' doesn't discriminate between integers, pointers and floats.
There are a lot of problems with user-facing operator overloading, but I think Haskell is actually a really great example of how to do it sanely. At the point in which you have type-families, you can properly restrict the semantic domain of a polymorphic function but still keep it an open set. The addition operator being restricted over types belonging to the Num type-family as an example trivially allows you to use '+' for matrix addition.
Now you've avoided the worst part of user-facing operator overloading (juniors creating incoherent DSLs by abusing the mechanism) while still providing the mechanism where it's useful.
I agree with you on overloading - the real word here should be "expressivity", which is defined by language constructs that could NOT be syntactic sugared into a local context. Those may occasionally be useful, though not by itself. E.g. gotos are more expressive by the above definition, yet it was largely deemed to not be a useful "power".
Certain kinds of GOTOs are indeed proven mathematically to be necessary to write certain programs, in the sense that eliminating the GOTOs can be done only by making the program more inefficient, i.e. by increasing the number of executed instructions and the amount of memory that is used.
Because of this, any decent programming language must include GOTOs, but in many modern languages they are not named GOTOs, because of the bad reputation of the word.
For instance the language Scheme does not have GOTOs, but it has mandatory tail call optimization, which means that a "tail call" is just an alternative name for "GOTO". For example, this allows the writing of a state machine in Scheme, exactly like it would be written in a language with GOTO, but using tail calls instead of GOTOs.
Other languages have labelled loops and they allow exit a.k.a. break with a label and next a.k.a. cycle a.k.a. continue with a label. Such instructions with labelled targets are just GOTOs with a bad placement of the label, which makes reading the source more tedious than with a classic GOTO.
For providing the benefits of GOTO, a restricted variant is sufficient, i.e. a GOTO that may jump only forwards and which cannot jump inside nested blocks.
With operator overloading you can provide same numeric operation interface to library types such as big integers. Sure, you can also define methods and make sure your primitive types also define those methods but now you have some confusions (like equals vs == in Java).
Typical ignorant take. Lisp macros allow you to create restricted DSLs to prevent bugs from being expressible, including full static type and structured programming systems.
Basically no one understands this until they reach the 'lisp enlightenment', so the decades of beating this drum falls on deaf ears. AI is here now so it's time to let it all go.
Of course you feel this way. You just finished reading SICP or maybe you binged PG essays last weekend. But eventually you'll read Simon Peyton Jones and start screaming about functional programming and algebraic type systems. Then, if you're lucky, you'll get a real job and realize that languages are just a tiny part of software engineering.
You've got a good point. Code generally needs to be read more than it is written. At the very least, you realize this as you try to revisit something you had done previously and are left scratching your head. Encapsulating the complexity in a custom DSL can be great for simplifying all the code you write in the DSL. But it also raises the stakes - if you ever have to revisit the DSL implementation the complexity is there lurking.
I would be very interest in seeing how "getting lisp" enables you to write software that is more successful than the C and C++ software that runs the world. Perhaps you have written software in Lisp demonstrating this? Something you can show us?
> Alan Kay says a lot of problems with C/C++ go away when using the higher math more easily expressible in Lisp and like minded languages
That doesn't match my experience at all. Math and numbers are in general not a problem except when (say) implementing triangle hit detection in a game (something I did) where the precision of 32 bit floats does matter.
> Jürgen Schmidhuber says the problem with computers from a mathematician's point of view are the numbers.
That is probably true if you are a mathematician. Most people aren't.
Having said that, LEAN (for example) is written in C++ and used heavily by mathematicians.
Go ate a big lunch from Java, C and a bit of C++ as a systems' language, which Go was designed as "C sucessor without the C++ bullshit". Even the most modern C (Plan9/9front, not what the ANSI C comitee vomits in every iteration) it's a very different beast.
Still, Common Lisp it's in places where trying to build such kind of software in C/C++ would be a reciper for disasters.
> Common Lisp it's in places where trying to build such kind of software in C/C++ would be a reciper for disasters.
I personally never blame the tools. I have written fairly complex software in 68000 assembler and it (1) worked fine and (2) was highly maintainable. However it was of course much slower to modify than using a higher level language.
"Of course that's your contention. You just got finished readin' some Marxian historian -- Pete Garrison probably. You're gonna be convinced of that 'til next month when you get to James Lemon, and then you're gonna be talkin' about how the economies of Virginia and Pennsylvania were entrepreneurial and capitalist way back in 1740. That's gonna last until next year -- you're gonna be in here regurgitating Gordon Wood, talkin' about, you know, the Pre-revolutionary utopia and the capital-forming effects of military mobilization." Good Will Hunting (1997)
In an awkward, error-prone, non-mathematically rigorous and easily subverted manner. That whole "make invalid states unrepresentable" thing relies upon the fact the semantic structures you're doing it with can't be shadowed silently by other parts of the code, and the safety constraint itself is guaranteed to be completely pure and deterministic. Lisp macros and reader macros fail at that criteria. They aren't any different from any other collection of procedures and their collective interface. Actually, they're a fair bit worse since the natural cognitive overhead from juggling multiple layers of evaluative indirection lends itself to blindspots of edge cases and unsoundness.
I think "macros are a safety mechanism" might qualify for the top 3 "new lisper mania" things I've ever read.
It can be made as mathematically rigorous as anything computable, and as non-awkward as is possible to represent. You either reify the invariants and semantics as statically analysed syntax or they are implicit in your collections of procedures and objects which is far more cognitive load. If you need to judge multiple levels of abstraction while using the DSL then the DSL is the wrong fit for the problem. There is no 'evaluative indirection' because a macro is a _compiler_ and the underlying code can be extremely alien to the DSL semantics.
Any argument against macros must be levelled against compilers in general, as _they are the same thing_. Rust is compiler as a safety mechanism, not different from macros as a safety mechanism.
It's actually harder to make something not Turing complete - in and of itself this property is absolutely useless and tells you nothing about how practical something is.
yes it's desirable to use non turing complete languages as less shit can go wrong, which is why you need macros. You seem to over estimate the difficulty of writing compiler macros, remember you only need to go from the DSL -> Lisp, not lower it into machine code. The DSL's compose perfectly as well, lower one to another or combination of them + lisp.
Well, compilers have layers - you ain't outputting assembly from the AST layer directly, usually there is some kind of IR. Like LLVM have plenty of layers.
And for a simple DSL you can usually just use a sufficiently expressive language, no need for macros. Like kotlin/Scala has html DSLs that look pretty decent.
For anything more complex though, you can't just "compose stuff", a type system is a global property so for that you literally have to write something way more complex than a local macro should contain.
these interpreted DSLs aren't at all what I'm talking about. The macro system should be able to compile arbitrary text into anything in the simplest possible way and have it seamless with the rest of the code + able to call into it and build off it.
A type system is paired with a language, not a global property. You can have language L1 with typesystem T1 compile into (L2,T2), etc. These typesystems can be domain specific and arbitrary as well.
After reading your posts I recommend you go ahead and learn common lisp macros, it will be more informative than more discussion.
Lisp macros used to be the one advantage Lisp had over other programming languages. However nowadays macros are common. There are even languages that has more than one flavour of macro system (Haskell has both a typed and non-typed flavour of macros).
However I personally prefer custom code generators instead of macros. The problem with macros, for the kind of large scale systems I work on, is understanding macros with N layers of abstraction, and also the compile/run time cost of using them.
Also, you can write code generators in any language and generate code for any language. Which is a huge advantage. I (for example) use code generators that generate C++, Java, Typescript, SQL, PDF's, interface descriptions, protocol specs etc. Whatever is required by a customer or other members of the team I can generate without demanding that they use a specific programming language.
Don't get me wrong. I love playing around with macros. However I have decided not to use them for real work.
I use plenty of DSL's in my daily work maintaining 1 million+ lines of C++ code used by very large companies around the world. No macros needed or wanted. More than 90% of the production code is generated from DSL's.
Outside of C++, Haskell has many great examples of powerful DSL's implemented without macros.
The problem with macros is the compile/run time cost. I could have implemented those DSL's using C++ templates (which are Turing complete by the way) but I prefer not to.
The inflection of the copula "can be" is exactly the problem, as is the 'anything computable'. There's a reason Isabelle is used as a proof assistant, and Prolog isn't. The former guarantees sound, mathematical rigor that can't be trivially violated. The latter doesn't. Prolog is much more powerful and flexible than Isabelle. Power and flexibility are disastrous for safety. As someone else said elsewhere in this thread, safety is about restrictions.
In this case, it actually runs deeper than this, there are structural issues which prevent them from being sound right from the beginning. It's the mere fact that the whole behavior of your DSL can be silently changed without touching the module it's defined in, or the file it's being imported into. A well-meaning junior, in a completely different department, can completely destroy your invariants in a completely unrelated part of the codebase. Whether by shadowing runtime functions, or a package-level collision that exists upstream. Even Scheme's hygienic macros don't actually solve this, they just make it less likely. At the point in which you have no behavioral guarantees of semantics, you do not have a safety mechanism at all.
> There is no 'evaluative indirection' because a macro is a _compiler_ and the underlying code can be extremely alien to the DSL semantics.
This is a trivial contradiction. That is precisely evaluative indirection. Yes, stacking multiple compilers on top of each other, especially when their semantics are non-isomorphic, is a massive burden of mental overhead. Given you already have this problem with the mapping between Lisp and binaries, adding even more complicated layers on top of it is a recipe for disaster. Particularly because: you are going to make mistakes in your logic. Macros provide no means to stop you from blowing your own leg off when writing them. That is not what they are designed for.
Finally, you cannot escape the awkwardness of the underlying metaprogramming system, you will always have to wrestle with quotation and quasiquotation. You have to define a DSL before you can use it.
Only at the point in which you treat and use Lisp like an unserious toy, rather than the powerful tool that it is, can the fantasy of "macros are a safety mechanism" begin to make sense.
Sure, and you may as well write full systems in Minecraft and create your own type system inside and whatnot.
Macros can make some small specific uses safe, indeed. But they are not comparable to languages with type systems where everything is type safe, unless you literally have written a new type system and compiler to begin with. Which is trivially true in every other language, C can be 100% safe with a much better type system, I just read a text block and compile it as rust code! Wow!
I have no horses in this race and I'm sorry if I misunderstand but I don't think he means to say Lisp is by nature superior to say Haskell. I take it as a statement on the generality inherent in its design. It is by its very nature very low on restrictions. Which is to say your freedom of expression is very nearly as complete as it'll ever be.
Sure you can beat Minecraft into Lisp and then Lisp into Haskell - as Turing made sure of - but you'll be battling a lot of dragons along the way. It's beautiful that you can beat Haskell into letting go of its type system by, say, creating a C compiler in it. It actually still amazes me the universe has this property.
In general I see "building" (I rather call it "conjuring") a system as going from the most general (all of your programming "language") to the specific (your solution). I can see how the most general of languages makes for a comfortable starting point of that downward journey.
I can also see how not starting from The Universe In All Its Infinity Glory (Lisp) but from The Planet Earth (say, Haskell) is helpful if you want to conjure something made for humans, but if you want to go beyond the mortal plane.. (I'm not saying that's a good idea by the way).
My issue is that a language is just as much about what it disallows, as much as about what it allows.
It's absolutely trivial to allow everything, you just have to make a Turing-complete extension either by design or accidentally.
It's much much harder to carefully choose more limiting primitives that only allow you to build stuff that keep certain important properties about them. You can't subtract stuff after the fact.
E.g. something like rust's borrow checker could not be realistically implemented as "a lisp" macro (what does lisp even mean in this case). For that to work you would literally build a rust compiler over s-expressions and you might as well leave off the s-expressions at that point.
And every part that was not compiled by that macro or whatever would be unsafe and could not interact with the "rust-lisp" compiled parts, as you can no longer assert anything about them (there is nothing mandating the usual single writer, xor multiple read-only references laws)
I agree. Restrictions are core to software development. In fact without them I don't think there is development in the first place. As you know I see software development as going from the general (the full bandwidth of the computational substrate available to you) to the specific (the absolute minimum - if any - computational structure you need to get what you're after).
Rust is more restrictive than Lisp. These restrictions are sane and productive for most general software development in 2026. They form an excellent base to start your work in this day and age. Starting with raw Lisp feels like starting with the general notion of mobility instead of just getting in a car. The car is not inferior at all. It's a set of restrictions on the most general notion of mobility that in fact buy you a lot of comfort like knowing it doesn't need to eat hay or take a poo in the street. The freedom they take away, they give back in strong, useful guarantees about stuff you actually care about ("getting places").
I can see how Rust, Haskell or whatever provide a set of comfortable, general-enough restrictions to make life for the common developer easier and more productive.
This being HN I do want to allow myself some contrarianism and point out that finding the most minimal, yet most general computational substrate is not at all trivial. It's in some ways equivalent to finding the most minimal set of invariants ("laws of physics") that can describe the most wide range of phenomena ("nature"). Which is to say I find it pretty damn impressive. So I only "disagree" with this:
> It's absolutely trivial to allow everything
I don't disagree because it is wrong, it's clearly not. Turing shows all languages collapse into one when it comes to computational power and I don't doubt that. But I do know there is a distance between the computational substrate, the primitives of the language, and the structures it expresses. Let me give an example.
A spreadsheet can represent a computer game. A game engine can represent a spreadsheet. But if you build Excel inside Unity you haven't made Unity spreadsheet-like at the substrate level; you have encoded spreadsheet semantics in Unity's primitives. Functionally they may be equivalent, and that equivalence is beautiful and Turing can tell you all about it, but the relationship between the native primitives and the concepts being expressed is different.
Turing makes building endless towers that are functionally equivalent possible, but what I mean is that there is an architectural difference between a system directly built out of the least semantically committed substrate available to it ("native") or "embedded" in simulations of substrates built on top of those primitives.
I think Lisp is close to the "least semantically committed substrate" you'll ever come across in practice. It's certainly not the only one, but it's a particularly clean and relatively practical one. One can say, for example, raw lambda calculus is equally uncommitted but it's kind of .. hairy. Lisp's primitives are basically the machinery of symbolic manipulation itself. It's bloody amazing.
Sorry for the verbiage. I went way overboard and meandered into foggy, mystical meadows full of mysterious entities so feel free to ignore.
Thank you for the reply, and I definitely agree with you on many points. I absolutely don't want to take away from Lisp that it is somehow uniquely elegant, and does have a mathematical beauty to it, this is certain.
I just feel this (and its practical ramifications) are often overexaggerated.
> I just feel this (and its practical ramifications) are often overexaggerated.
I dunno. Can't fully agree or disagree. Nominally, yes, you really don't need s-expressions and homoiconicity for creating reflective, self-hosting runtimes - live redefinition is possible in Erlang, Pharo, Ruby. Metaprogramming ergonomics - sure they are cheap in Lisps, but even Lispers try to avoid reaching there, Clojure specifically recommends thinking twice, although projects like Hyperfiddle prove macros absolutely can be very powerful. Syntax, mathematical beauty, yada-yada - that's all "poetry", much of the real world operates on tons of very ugly yet functioning code, right? So, really Lisp-shmisp, whatever, no?
In practice though, Lispers are enormously pragmatic - I'm not self-referencing here, I have worked with some. It's incredible how rapidly they can build things, prototyping on the fly. How quickly they can move between different runtimes - I've seen codebases sharing ideas between absolutely dissimilar platforms. It is inspiring how undogmatic they could be - they easily move between modes - data/code, FP/OOP, interactive/compiled, etc. They have good understanding of type systems and some even know good deal of theorem provers. For whatever reasons, Lispers are conspicuously, disproportionately effective, and this is true. Sure "citation needed" here, but this is my empirical, anecdotal observation working in different groups, using distinct language stacks for many years.
The causation probably runs through the programmer, not the program. It's not like Lisp unequivocally emits good engineers, maybe it's that a substrate with minimal syntax and maximal malleability trains a particular disposition - they treat everything as reshapeable material, distrust dogma because the language never enforced one, reach for the smallest thing that works because the language makes "the smallest thing" actually small? Maybe Lisp doesn't make anyone write better programs, it just makes it cheap to keep changing your mind? Perhaps cross-runtime fluency, undogmatism, and rapid prototyping are all just "cheap to change your mind" in a way?
Could be selection bias - maybe Lisp just attracts curious, theory-literate, undogmatic people? I don't know. What I observed is for whatever reason Lisp typically attracts older, more experienced engineers. And therefore all the "Lisp propaganda" comes from them, and demographically that's a small subset of overall community and maybe that why it often feels like overexaggerated rhetoric?
Having that natively available as plain Common Lisp code is a very different thing from your "just read a text block and compile it as rust code!" concept. Or at least, the tool chain gymnastics required to make a C program emit its own source in another language... Why?
Well, I was talking about the fundamental part. Of course you can have more practical implementations, like Scala or rust's macro system, that gives you a proper typed AST.
Type safety is a domain specific concern, 99% of the time your domain will not have 'types', often it won't have notions of things at all. But any kind of type safety can be added at any point with macros. Something like rust would be a huge project but it doesn't get easier than macros, see Coalton, and they compose with one another. When you run into the same requirements in another language you are simply stuck source pre-processing. There is no stuck in lisp.
There are plenty of languages with macros though, nothing special about lisps.
And no, language semantics absolutely don't compose. Like you can't just do some kind of optimization in one place if you do some mutation on it in another place. Optimizations work on global assumptions that every part of the codebase have to abide by. Any part not doing it will make the whole thing crumble - and "there is no stuck in lisp" is exactly why you can't have your cake and eat it too.
Don't miss out, we had new excellent editors and tools being released in the last months:
Mine: a complete, single-download application that comes with everything needed to experience the interactive and incremental development programming workflow, including hot-reloading and on-the-fly debugging. For CL and Coalton.
The website seems to have a bug with syntax highlighting. Pieces of code included in the post text are black, you can still see the actual text if you select it with your mouse. Same bug on Chrome desktop and on Safari on iPad
This is the classic example of why you should always set the foreground color when changing the background color. Otherwise you end up with <code> blocks that are black-on-black. It probably works fine if your browser is in dark mode.
A workaround is to open one of the code elements in the browser's debugger and add color: white to the :not(pre) > code style.
Now, it looks fine to me too. I tested it in Chrome,Firefox, Safari on macOS and on Safari on an iPad yesterday when I wrote my comment and in all cases there were black lines all over the the place. The author corrected the problem, see https://news.ycombinator.com/item?id=48855382
There are many articles extolling the virtues of Lisp. I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Articles like this, and the PG articles it references, amount to "if you know, you know". I understand the appeal and I understand the explicit and implicit arguments this article is making.
Computer programming has matured quite a bit in the past 60 years. I would like to see more articles that are more considered in their examination.
As a language Lisp is great (though the ecosystem is limited). It has two flaws-- it's very open-ended so unless you're talented and disciplined you can fall into a rabbit hole of hacking fun Lisp stuff and not actually getting any work done. Other languages have this problem, but Lisp I think more so.
The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.
I started with pascal and then C++. And then discovered Emacs and Lisp. Boy, that was a revelation!
Never ever in my life shipped a line of lisp code to production. My real life code was always Python or C++ or Java or C. All of my lisp was toys and emacs tweaks (50k loc in my config!).
But most of production code is gone from my life. I am a mid-level engineering manager now. I mostly write texts, or messages, or emails, or slides... and still use emacs and lisps for fun and profit and competitive programming.
Not a single production-grade lisp LOC in 20+ years. But, OTOH, i contributed to emacs, tinkered with compilers and interpreters and prog. lang. internals - all because Lisps made it interesting for me.
And, in a way, this brought me closer than ever to the job of my dreams: i work for a major player in static analysis space.
Your wording sounds like it implies that Lisp "got stuck" somewhere in the past, no?
Clojure, Clojurescript, Clojure-Dart, Fennel, Jade, Jank, Jolt, Coalton - these are relatively recent (and still developing) languages, and this is just off the top of my head, there are so many more.
Lisp is not a programming language (in a sense), it's an idea. It influenced pretty much every single PL we use today and continues to do so. You can't really "level-heatedly" criticize an idea, it's like criticizing I dunno, group theory. You can though debate about merits of a specific implementation of it.
> Your wording sounds like it implies that Lisp "got stuck" somewhere in the past, no?
Pretty much. See pg's famous "Blub Paradox" where he sees Lisp as the top of a tower of lesser languages. He doesn't recognize that Lisp might at best be called a limit ordinal, to use math jargon. That is, Lisp is just another Blub, and the Lisp zealots haven't figured that out.
You don't necessarily want to keep going further and further up, of course. Lisp still has fascination. But e.g., in Common Lisp (I mean just the stuff in the CL spec, no ad-hoc extensions allowed) you can't write anything resembling an OS. You can in Scheme, using continuations to handle process switching.
Going further up, Lisp doesn't make it easy to ensure the absence of particular behaviours in a program, what TAPL calls the purpose of a type system. Tony Morrison has a semi-realistic example of what static types can get you:
Your lisp criticism is orthogonal to the idea of lisp itself. Lisps like Coalton (lambda calculus) or Shen (sequent calculus) are strongly typed.
AI is an interesting point too where being closer to a raw AST is likely an advantage because it can focus more on the semantics instead of the syntax.
You can wrap parentheses around anything and call it Lisp, but I specified Common Lisp and pg's article clearly wasn't contemplating anything like Shen. Surface syntax is near irrelevant though.
I feel you're now contradicting your own statements now. You said: "[maybe] Lisp is just another Blub", but when pointed out that Lisp (and Lispers) is evolving, you're saying "pg clearly meant [the other Lisp]"...
Of course we're not talking about "parenthesis". Clojure even added square brackets for arguments and destructuring, it's not about the syntax, I agree on that.
pg's notion was psychological, not about languages per se: a programmer sitting in language X can't perceive power above X, only below. It's about a fixed vantage point.
Your "Lisp is just another Blub" would be true if Lispers were stuck looking up from Lisp-as-it-was and failing to see higher. If Lispers were trapped in the Blub position, they couldn't have deliberately imported ideas that sit "above" classic Lisp. But they keep adapting: Clojure added persistent immutable data structures; brought CSP/channels; introduced structural contracts like Spec and Malli. With Coalton and Shen they are explicitly reaching for the static-types level you say they can't see. Racket's whole "language-oriented programming" perhaps a level above any perceived Blub.
Maybe what you see ain't a Lisp ceiling? Could it be that some powers must live in the substrate - and Lisp's distinctive traits are precisely what makes building that new substrate cheap?
Most Lispers I met do actively write in multiple languages all the time. They tend to borrow ideas from other PLs instead of loathing everything else that's not their favorite. They prefer Lispy syntax, but won't reject a language to achieve a goal - they'd pick everything - runtime, tooling, etc. and try to find a Lispy syntax that sits atop. While preserving all the remaining semantics. And then they'd argue that syntax does not make a language. Perhaps, Lispers are the largest demographic of polyglot programmers in the global community.
Well, Haskell/OCaml campers would probably say "it ain't FP, like at all, rofl..." Truly, it's absolutely pointless mental exercise to cherry pick features of any given PL and compare with another one, with no regard to overall experience (which often is very subjective).
Switching between different Lisp dialects is far less mentally taxing, even when they operate in completely dissimilar runtimes. I have seen days when I needed to jump between CL, Clojure, Fennel, Elisp and Janet and from all practical points it felt like almost using the same language everywhere. While switching even between JS and TS is enormously vexing for me in comparison. Although I have programmed in both for far longer than any Lisp.
> I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Standardized Concurrency is basic table-stakes for a language today. CL does not have a standardized async/await or concurrency model. The standard hasn't been updated since 1995 so it will never happen.
And it remains in a state of almost getting to the point of generic collections (like C++, Clojure, many others) using the standard functions, but not quite getting there. There are functions (not generic functions) which operate on sequences, but no standard way to extend what types are considered sequences (as one example). It makes sense that the 1995 version of the spec would be incremental, but without a further development (or consensus amongst the implementations if no official standard) it's difficult to continue moving forward to make better use of the language's generic function capabilities.
That's where Clojure, Julia, and many others have advanced beyond what CL (by the standard) provides. The language needed at least one or two more standards to drive those features forward, but never got them.
None of the functions operating on sequences are generic, so you cannot create your own type and do a `defmethod` and have them automatically adapt to it. And unlike `print-object` which gets used by format and print and others, there's also no "protocol" (or whatever term you like) generic function you can implement which will make your collection acceptable as a sequence.
You could build something like this, and maybe shadow the standard functions so that it's not a new set of functions from a user perspective, but it's a notable absence in a language with a rich OO system through CLOS. It makes sense that it wasn't in the original spec, but I still think it would have been something that would have been added if the development of CL (as a standard) had continued.
The latest finalised submission to the Common Lisp Document Repository (confusingly called CDR instead of CLDR) dates from August 4, 2013. CDRs are the equivalent to the SRFIs from the Scheme world. You could argue that they are not THE standard, but they are A standard. And considering the original standardization supposedly cost 400,000 USD (about 900,000 USD in today's money), I can see why there hasn't been another official ANSI standard.
Since my job involves a lot of Unicode stuff, I personally would have been confused if it had been called the CLDR, thinking that it was referring the Common Locale Data Repository. (I have to look something up in the CLDR about once a month on average).
And since the Common Lisp Document Repository appears to have been created in 2006 (the first document at https://cdr.common-lisp.dev/index_files/final.html is a description of the rationale for its existence, dated August 2006) while the Common Locale Data Repository was created in 2003, the Common Locale Data Repository had the CLDR acronym first. So kudos to the Common Lisp Document Repository folks for not overloading the acronym but finding an alternative.
I see. Wasn't aware of this. However, it does not appear that an official standard for concurrency exists within the Common Lisp Document Repository (CDR). To be honest, CL implementers should just solicit donations for producing a standard.
There are also other basic things needed for adoption - standard APIs for database providers, etc
Certain people seem to find it mentally appealing. I would liken it to one of the really niche music genres. Only a small number of people like it, but they do so very intensely. The Stockhausen of programming.
That's basically all it is. A fandom. A remarkably enduring one, but not one which has ever broken out, and therefore is unlikely to ever do so. Now in an even worse position: if you are armed with an LLM and therefore uninterested in the code itself, why on earth would you direct it to write in Lisp?
I've been programming Lisp too long to appreciate it's features. It is not until I watch in shock and horror how people sabotage themselves in the popular and economically less risky languages that I suddenly come to appreciate Lisp again.
There are some truly powerful and unique things about Lisps, but I wish articles like this would stop including REPLs and hot-reloading. The former have been table stakes for interpreted languages (and some compiled ones!) for years, and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
> The former have been table stakes for interpreted languages
I used to think so too back in the day when I was getting into Clojure. It was much later when I realized that when Lisp people talk about the REPL they’re usually talking not so much about the interactive CLI where you can evaluate commands easily but more so the ability to connect your program to a live session where you can quickly evaluate forms within your text editor and in the context of your running application, which enables much more interactive development than in other interpreted languages.
the python debugger is cool but is an inferior experience. It isn't only about a REPL, but about the editing experience, and about the language being thought from the ground up for long-living programs. See: https://mikelevins.github.io/posts/2020-12-18-repl-driven/
No. There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages - Ruby, Python, Haskell, etc. Every single stage there in Read-Eval-Print-Loop differs.
Man, it gets so exhausting trying to convince every PL critic who grabs a single (or a couple) of axis of any language and tries to dispute the value of a language without ever understanding the holistic, overall experience working with it.
Like, I don't understand, do people think that tons of Clojurians or Common Lispers who fall in love with the language after decades of working and getting seasoned in literally dozens of different PLs are on some kind of delusional trip or something?
Guys, just take a gander at Clojurians Slack; see what people are working on, what kind of stuff they're building; check their profiles. Many of them are the battle-scarred veterans of coding. Sure, some of them may have wrong opinions, but surely they can't be all wrong, can't they?
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
Smalltalk is not homoiconic, and it's REPL experience is equivalent (I'd argue somewhat better, but that's mostly a tooling thing, see the commercial CL implementations as examples of improvements over SBCL + Emacs + Slime). Homoiconicity is not the trait that makes the CL REPL experience better than others, it's that it includes a very good debugger, the compiler, hot code reloading, the ability to redefine classes and update current instances, and so on.
That's the tooling, not the language, that provides the experience. Nothing about being non-homoiconic prevents other languages from having a comparable (or even better) experience.
Nothing about being statically/dynamically typed. Nothing about being functional/OOP/relational/logic. Nothing about being pure/side-effecty/strict/lazy. Nothing about being compiled/interpreted. Nothing about imperative/procedural/stack-oriented.
You can pick just about any single or (few) aspects about any language. Heck, it doesn't even have to be a programming language and you can find things to complain about.
You know that there are three genuine, true, legit ways to build robust, bug-free, performant software? Three! The problem? Nobody knows what they are and that's why we are all doomed to keep bashing on everyone else's choices and opinions.
"Better experience" is not dictated by "features". Better experience comes, well, with experience.
I suppose it's my own fault. I tried focusing on "holistic, overall experience", yet still picked a single aspect to chime in.
I wrote GGP as a Lisp user and enjoyer—not pretending to deep experience, but definitely am well acquainted. And neither the REPL nor live reloading are important differentiators. Neither is the ability to attach a REPL to a running program: plenty of languages have that built in (Erlang, Ruby), and many others support it through popular third party tooling (e.g. Pyrasite for Python, and—ironically—the class executor for the Clojure REPL for arbitrary non-Clojure JVM programs). Many Lisps’ REPL tooling is very nice, and the language lends itself well to REPL-oriented development. But those aren’t “uniquely Lisp” features as you and many others claim.
Homoiconicity is a "uniquely Lisp" feature and it doesn't seem like you've fully grokked the implications and differences between homoiconic and non-homoiconic. I'm not making this up - every step there in Read-Eval-Print-Loop does differ. That is easily verifiable info.
Eval differs very slightly from, say, an interpreter with a very non-syntactic AST, due as you say to homoiconicity. Neither read, print, nor loop differ in any Lisp-specific or qualitative way. And the differences in eval’s behavior don’t change anything significant about the REPL user’s experience (other than that they’re writing Lisp). So what do you mean?
Specifically, cached state and behaviors still need to be reconciled with new inputs/overrides during “eval”, and no Lisp has an easy answer to those (neither, as far as I know, does any other language). Avoidance of closure state via late binding and a convention of reliance on very simple data structures help Lisp’s REPL/hot-patch story, and that is indeed nice, but it’s far from “guaranteed ease/safety of REPL patching” or a totally different paradigm.
> don’t change anything significant about the REPL user’s experience
Perhaps you have never tried/heard of nextjournal/clerk, scicloj/clay, djblue/portal, vlaaad/reveal or just simply tried building a simple web scraper with Playwright running on nbb. Oh, and hyperfiddle/electric - something like that would not be very trivial without homoiconicity.
Lol, how are you so smart and figured just everything out, so already? I'd be so bored to ever touch the keyboard again. Bless Turing I'm so stupid and simple and keep trying different things just to see how they differ.
I wish someone would just write a clear description of that magic version of REPL that isn't a REPL. Maybe also a demo. And how to get the actual pristine source out once you've done development on a running image. Maybe we could be enlightened. The complaining clearly isn't doing it. Articles like this one didn't succeed either. Pointing at Slack surely isn't going to do it.
Tell me your knowledge of Ruby is surface level without telling me...
I used Clojure when it first came out, I've used Common Lisp for years, I've also used languages like Smalltalk and many others.
Yes, Lisp environments are nice, but sometimes I think Lispers are so insular they don't realize that other languages have similar things. R, Julia and Ruby have similar environments. Smalltalk is next level.
And homoiconicity is great for macros and parsing but it's late-binding that enables the live programming behaviour, which isn't exclusive to Lisp.
Also if you'd ever gotten deep into say, SB-ALIEN, you'd know the limitations too; Common Lisp isn't magic, it can't just redefine say, instantiated structs in memory. It relies on pointers then switching references on the fly.
> they don't realize that other languages have similar things
Yes they do. God, how did you (and apparently you're not alone) read my rant and still got it 100% backwards? I specifically hinted about not picking a single aspect of a language - REPL or whatever. What is so confusing about my wording on "holistic, overall experience working with it"?
Apart from your rant, this is the most pertinent part of what you said:
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
And it's false. There's nothing special about a REPL in a homoiconic language. Homoiconicity has nothing to do with a REPL or tools.
I do however think SBCL has a very nice environment with great tools (especially it's compiler) and that's why I've used it a lot over the years, but I really think you underestimate the environments in other languages (including the "holistic, overall experience").
> And it's false. There's nothing special about a REPL
You are wrong. You might be correct about the workflow but still wrong about the semantics. Pedantically, your notion is false, because homoiconism gives the REPL a genuine capability (program-as-manipulable-data across the read/eval boundary) that non-homoiconic REPLs lack.
I did not say anything about s-expressions. I said "homoiconicity". S-expressions are not an absolute requirement for homoiconic features. You keep poking me with your "knowledge" as if I'm trying to sell you snake oil and you feel obligated to defend and retaliate. I don't need to prove anything to you, I'm not your ignorant professor attempting to instil you with some defunct truths that exist solely in my head. Do your own research. Or don't, who cares.
You sweet, summer child, just so you know, I have existed for nearly half a century in this world where the larger part of it I have spent dealing in computing. I have seen and dealt with more programming languages (including Ruby) than you can count with your fingers and toes and that number keeps growing still. I'm not advocating for any particular language, runtime, framework or paradigm - do use whatever your heart desires.
I'm simply pointing out that there is a meaningful difference in REPLs in homoiconic and non-homoiconic PLs. You don't have to listen to me, this is easily verifiable information. Google it, ask LLMs, try it yourself, or ignore the notion entirely - it's completely up to you. But let's not get too intimate and slide into insulting one another - you have no idea who I am and what cults I'm specifically fond of.
Apparently you're not experienced enough to have used a Lisp that doesn't have a REPL, nor a non-Lisp that does have a good one.
You obviously haven't implemented a Lisp either, because then you'd understand what homoiconicity actually gives you. And also how a REPL is implemented (since you wouldn't just get one for free, it's a tool to implement).
You're wrong on every count about my experience. And once again, please: I'm just saying that stages in Read-Eval-Print-Loop does have differences in homoiconic and non-homoiconic languages. THAT'S ALL I SAID. There's zero controversial taste in that statement. None. I'm not bashing on Ruby, or Python, or any language you favor. I'm not telling you to use this or that. I'm just pointing out at specific differences that exist.
Implementing C-like language in Lisp runtime - your interpreter reads C-like source, parses it to your own AST structs, and evaluates those. The host being homoiconic says nothing about the guest. You would have to write a guest REPL anyway (if you want one).
Rhombus built on Racket, yes, means it reuses Racket's machinery: the macro expander, the module system, the compiler, the runtime. But it doesn't mean it inherits Racket's parenthesized syntax.
Racket's expander does not operate on text. It operates on syntax objects (shrubbery). It has its own REPL, because the Read stage has to parse shrubbery syntax, not s-expressions. So, my original point stands - REPLs do have differences for homo and and non-homoiconic PLs.
I don't know who that is. I don't play stupid social games to win stupid social prizes. Why would I create a new account and yet keep replying with another one? I have single accounts pretty much everywhere, except Reddit - they shadowbanned my first one when I posted while connected to a VPN (apparently that's not allowed). You can easily use some OSINT tools to confirm that.
Da fuk you're talking about? Where the heck am I being an asshole? I have not used a foul language or insult in this thread once, even when someone accused me of being in a cult. Don't patronize me, even if you bought a moral authority on a discount - I'm not your fucking child. No matter how many times you say "please" and call me "asshole" I won't correct anything in my behavior because there's nothing to correct to begin with. What "disservice"? a) I could't care less about Lisp's popularity b) You seriously think someone, anyone would be like "don't look at Lisp... because there's one jerk in this specific HN thread. He said something in 2026. Eight levels nested down..."
> and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
For what it's worth, hot reload is very widely used in Dart/Flutter. When you are writing UI code using a reactive style framework where rendering the UI appears to be "generate a new UI from scratch on each frame", it's more straightforward to have an intuition about what does and doesn't get reloaded.
It's not perfect, of course. But it works really well for the kind of changes you make when iterating on a user experience.
I do think it is worth highlighting how many advanced parts of hot-reloading have already been covered in Common Lisp. Same with highlighting how the REPL is largely not used to directly type into, but is instead a very powerful interface for tools to interact with a running image.
But, again agreed that simply these existing are not that notable today.
Not all Lisp dialects have well-developed hot patching systems. The OOP system has to be carefully designed for it. What if a class definition is superseded by a reload, but there are existing instances? The Common Lisp dialect of Lisp has useful answers to questions like this, but not necessarily every Lisp you come across.
Even Bash supports hot-reloading. I've developed modules that are updated in place by sourcing:
Generally speaking, the pain of handling hot patching depends highly on the structure of your codebase and the reason for hot patching to begin with. For a structure-of-arrays architecture, or for fixing logic errors at run-time, it's really no big deal at all.
It is limited utility, but if you start out from the mindset it's something you'll use a lot, those limitations start disappearing rather quickly.
Can you expand on that? I am mildly experienced with Clojure, and much more so with the JVM and I see no reason for that to happen (besides perhaps some kind of module shenanigans or importing stuff that requires special flags?)
> After getting comfortable reading code with so many parentheses
I never managed to get over the ().
Ruby has a very flexible syntax, compared to many other languages,
in that you can omit syntax in many cases. For instance, using
() for method calls is largely, for the most part, optional. So
when I have the python code:
cat = Cat()
cat.meow()
I find it worse than the ruby code:
cat = Cat.new
cat.meow
(Though you can use () in ruby there too; but most people
won't do so as there is little point in that.)
This is a superficial issue though. Python's biggest mistake
is to require implicit self. It always feels as if I need to
hand-hold python and trying to explain to it what an object
is and what self is. In ruby I don't have that issue. Note
that I find both languages fine, but ruby is "more" object
oriented than python is, for many reasons.
Lisp is quite different though. I had some exposure to it
via scheme and while it can be fun, I feel that lisp is a
worse programming language than either ruby or python. The
old game haxima/nazghul was given up eventually, primarily
due to lack of time, but also because the author was no
longer convinced of scheme (the core of the engine is in
C if I recall correctly). He tried to switch to python,
though, which did not work (but, again, I think it was mostly
due to lack of time). The scheme code in haxima/nazghul
was quite interesting (https://sourceforge.net/projects/nazghul/
if you want to look at the scheme code there), but I much
prefer either ruby or python there. Although it would be
interesting to have DSLs that would really focus on the
game or project at hand, like in the old Zak McKracken game.
As a non-Ruby user, this confuses me. Why can't `Cat.new` be a function reference? Does Ruby explicitly disallow this (i.e. passing around a function)?
In commercial practice DSLs are an anti-pattern. Someone will create an under-documented DSL that only they understand and move on. The following programmers have to try to decipher this strange language. It is almost always a better idea to use standard language structures and features. Code is read more than it is written.
In Lisp, those following programmers can simply macroexpand. With modern tools they can even do it inside their editor, co-located as replacement text in the same source file, connected to a live Lisp environment, if they wish.
A great example is Common Lisp's own LOOP macro - if someone's usage is difficult to understand, you simply ask Lisp to expand it into the more verbose non-LOOP fundamental calls, and you don't need to understand anything about LOOP. In fact you can replace the LOOP form with the expanded code.
That's a key difference from DSL's in most other languages where the DSL code is really data structures that are interpreted.
I’m not referring to trivial functions nor classes. I talking about things like overloading operators, or changing the default idioms of the language. This means that a person can’t read the code without fully understanding the entire class structure. For the most part DSLs are highly personal. What one person thinks is a great formalism that make it easy to reason about the problem, to everyone else is spaghetti code.
I always thought that the "answer" to programming would be that one day everyone would use Lisp and with awesome tooling and libraries things would be wonderful. In fact, my plan for retirement was to build high-quality libraries for a Lisp language to accelerate this process.
Does the rise of AI bring an end to this dream? Is that, once again, we have solved the problem by adding more cruft? Rather than a superintelligent AI writing in the best programming language available, we're going to just spam lots of Python code until it works?
Does this matter? I don't know, I just wanted a world of elegance.
had become a mainstream Mac product rather than being co-opted for NeXT and use w/ Objective-C.
LISP clicked with me (when taking a comparative language class in college I was the only one who managed to do all of the LISP homework) and learning it was a lot of help in using TeX.
I just wish that there was:
- a nice native or cross-platform GUI toolkit for it which was opensource
- an easy way to distribute projects as stand-alone compiled code
Get Quicklisp and after that, Ultralisp because the last CLX library it's much faster even on shitty computers las my n270 netbook. Under ECL it almost ran without lag (just a slight one), and with SBCL
it was almost as fast as Tk under TCL.
Languages are never the answer. Lisp isn't superior to Visual Basic. They are both Turing complete. Humans can express good ideas in each. It's better to think of languages as tools. An impact drill is not superior to a simple rotary drill. You will have better luck building a deck with an impact drill, but both will do the job.
Personally, I grew beyond this. I tried Common Lisp, Forth and Haskell. I enjoyed books On Lisp and Let Over Lambda.
Now I think the best programming language is Barry Jay's Triage Calculus, which is close to combinatory logic or untyped lambda calculus.
But unlike lambda calculus (which is easily expressed in TC), triage calculus has a built-in quoting and introspection (similar to Lisp's CAR and CDR), which lets you easily add typechecking. So in TC, you can have any syntax you want, just by building the correct abstraction. It's truly an ultimate programming language.
TC shows that there is a false dichotomy between programming language features and syntax on one hand, and the function/API definitions in the standard library or user code on the other hand. Every time you write a piece of code intended to be reusable, you're adding to the language. It can all be expressed as some term in TC.
All the syntactic squibbles are a matter of habit. Although one syntactic feature you really want in TC is the let over lambda abstraction, because it's somewhat annoying to order combinators by hand.
Yes, although I wish he would call it properly triage calculus (see https://treecalcul.us/specification/), because although the languages are almost identical in power, the original tree calculus and triage calculus rules lead to different programs.
I must admit - I still don't understand macros. I get that they're code that's generated at compile time. But I don't understand how that's different than a function which evaluates other functions. I guess the latter would actually be evaluated at runtime? I think I get it conceptually but I'm not sure I have the muscle memory to reach for them. Anybody here have an "ah hah!" Moment with macros?
> But I don't understand how that's different than a function which evaluates other functions.
You want a function that takes in arguments, but does not evaluate the arguments when called. So:
func(foobar(), foobar())
Normally, foobar() will be called twice - at the time of the call. With macro expansion, you can ensure that it's not the result of calling foobar that goes into the func, but this expression.
A canonical example is if you want to write an if/then/else function:
if(condition, then_path, else_path)
It's quite possible that the else_path is invalid and will terminate the program if the condition is true. But if you wrote a function this way, it will evaluate both then_path and else_path - not something you want to do in a regular if expression!
The point is that in most/all languages, if the condition is true, the else branch is not evaluated. And it's usually OK to put code in there that can crash when the condition is true, because we know it won't be evaluated.
But if you make an if function like I did above in, say, Python, both the then_path and the else_path are evaluated before the decision is made.
> Technically you can accomplish the same thing in languages with first class functions if the caller wraps their code in a lambda.
True. For languages that have lambdas (which I guess is most of them nowadays).
Well, as mentioned that's why you create two lambdas as the branches. This is pretty much what thunks are in some languages (like Haskell), so this example doesn't strictly require macros per se.
Well, if we have a compiler that sees a constant value, it can completely optimize out one branch. But it of course depends on the semantics of the language.
But yeah you are right in case of a naive compiler.
Sure, macros are functions that take functions as input, and produce new functions as output. But they take the function's symbols as input and produce a new set of symbols. So a macro can extend the syntax of the language without having to modify the core language system. Anyway, what's unique about Lisp macros vs say, Rust macros, or C style preprocessors, is "homoiconicity". The data structure that a Lisp macro takes as an input, the lisp code, is the same data structure that the language uses normally (S-expressions, lists...), so writing a macro requires few new language skills compared to writing normal lisp (again, compare writing Rust macros, a dark art in comparison).
At the same time, you get a mushy tree as input, instead of a properly typed AST "object" with accessible "function name", "method parameters" everything that you can just refer to.
It will be the third arguments' 2nd arg with s-exprs, following your arbitrary pattern you figured you want to use. So it's very arguable which is easier to use, sure for some "party tricks" lisps will win, for anything more useful my vote would go with rust/Scala macros for sure. So again, you win nothing by homoiconicity, the by-the-compiler implemented parse function is more complex in rust, which is 100% abstracted away.
If you haven't read it, I'd suggest taking a look at Paul Graham's book On Lisp [0]. He says better, and with more examples than I'd provide in a comment block, what I'd write on the subject. Jump to chapter 8 for his discussion on the topic, referring back to chapter 7 if you find the macro definitions difficult to read.
I think the best explanation, how Lisp macros are different from most other macro system is, that a Lisp macro is a function executed during compilation. It gets fed the parameters passed to it in literal form and then computes the source to actually compile by the compiler.
The power of this is that this function is also written in plain Lisp and it also sees all definition already existing in your Lisp image. Also, it is completely unlimited in power. So while being run, or as it is called "during macro expansion" the function could google for one of the things passed and use the result to create the output. Of course, this is not a practical example, but shows how flexible the system is.
There's a bit of a mental model flip to make maybe.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
The arguments presented to a macro don't have to be valid code. Your "function evaluating functions" are all individually valid functions you've composed at runtime.
(defmacro foo [code] ,,,)
(foo "<some totally different language in a string>")
=> actual, compiled Clojure function built up by `foo` parsing the string, producing a Clojure list, and calling `(eval the-list)` and returning it.
I passed a string to `foo` in the example, but it could actually be anything the Clojure reader can parse even if it is semantically invalid—arbitrary Clojure data.
function can do what a macro does, but it must run every single time while the macro can run just one time during the final code generation.
Let's say you have some Java-style a + b. It needs to work on native strings (concatenation) along with various kinds of ints and floats. The issue is that each one of these works differently internally, so the system is going to look at the incoming symbols and metasymbols (inferred type data) and is going to dynamically change between different functions like ADD_STR, ADD_INT8, ADD_INT32, ADD_FLOAT32, etc. If it did this at runtime by introspecting the type with some `switch(incomingType)` statement which involves all kinds of extra data and branches that clog the cache and create branches (two of the worst things you can do for performance). Instead, the macro (though they probably don't call it that) looks at everything and hardwires the correct output.
This hardwiring and its associated performance is the difference between a function and a macro. A macro `foo!(a b)` can inline into the current function while the function alternative must save all the registers to the stack and create a new stack frame which is very expensive.
The beauty of lisp macros is how functions and macros look the same. You may think that `(+ a b)` is the same as your favorite languages' `a + b`, but most lisps also allow `(+ a b c)` too. `+` is a macro rather than a function and it knows that `(+ a b c)` needs to convert to something more like `(+ a (+ b c))` and a typed lisp may go further to something like `(+f64 a (+f64 b c))`.
Open notepad and solve your problem in some language you make up on the fly, with macros that's now valid lisp code. A macro is a compiler and nothing more.
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
If it helps, https://taeric.github.io/CodeAsData.html was my attempt at exploring "code as data" and what makes lisp different here. My specific focus was more to point out that "eval" in lisp doesn't just take in a string. But I think the same general points remain.
Many problems can be solved with either macros or higher-order-functions. The advantage of macros is two-fold (macros also have disadvantages, and Lisp allows you to use either solution; deciding which to use is often a matter of taste):
1. Syntax can be more familiar
2. Performance (to the extent that there is overhead for functions-calling-functions).
For example, let's consider a hypothetical lispy language that doesn't have a short-circuiting "and" operator. Macros would let you implement something to used like this:
(and (foo x) (bar y))
Higher order functions would require you to do something more like:
(and (lambda () (foo x) (lambda () (bar y))
More noise, and more work for the optimizer (or in the worst-case, more work at run-time). Languages that rely heavily on higher-order-functions will tend to have terser syntax for anonymous functions. For example in javascript you might do:
Macros are conceptually similar to FEXPRs in that they act like functions that don't evaluate their operands and return code. The (only?) difference is that macro are all expanded before execution/runtime.
one way to see it is that it's a function that runs at compile time. for instance instead of dumping magic numbers/tables in a codebase you could put the code and substitute to their value at compile time.
but also it can change your code, so you get to do all the java annotation magic stuff.
You do not need macros for anything. They are not the tool to "extend the language", but instructions for the compiler. And if the system does not even have a compiler, macros are useless, often incredibly stupid.
If you have an interpreter only, no compiler, using macros for metaprogramming anwyay at least prepares you for the eventuality that one day there will be a compiler. The macros will Just Work as before, only the expanded code is now processed by compiling.
Suppose you reject the idea that there will ever be a compiler. Macros are still useful for doing "compiler-like things" in the context of interpretation, like transforming code into something that will interpret better.
We can regard the interpreter as a virtual machine acting on a representation of the code; the macro system lets you manipulate the representation in a pre-computed pass, which has no further cost at run time.
If you have a code expansion pass on top of an interpreter, for supporting macros, you can use that as an excuse to perform built-in code transformations that are not macros; those enable you to have more flexibility in how special forms are implemented.
In TXR Lisp case is a macro (family) which performs certain optimizations like recognizing values in a range and emitting a table switch. This works interpreted or compiled:
It is faster to do some checks and interpret the sys:switch special form to dispatch by a numeric index than to do a large number of exhaustive comparisons.
Ultimately, the way we optimize interpretation is by compiling, but it can still be worth it to have better interpretation here and there.
You don't want to do this kind of optimization at run-time; you don't want the interpreter to be evaluating the condition "are these cases integers (or characters) in a tight range?". That's a property of the syntax in which the cases are constants; it wants to be pre-computed once.
TLDR, but you are (probably) describing somekind of builtin property.
I am just saying that the end-user do not need macros for much anything, unless she has some specific optimization in mind. Bloody annoying when they start making cryptic macros to "extend the language".
"Need" is a very tenuous word, because most of any tool stack consists of benefits.
End users benefit from things that they don't strictly need.
Once you start talking about what people don't need, it's hard to do it in a way such that they need Lisp but don't need macros.
Sometimes "need" is about dependencies; i.e. things you "need" are provided by upstreams; you stop needing things that you made yourself.
E.g. "I don't need objects. Because, well, I have lambdas and macros, and with that control over syntax and semantics, I made my own objects. So, strictly speaking, I do need objects, just don't need them from you or anyone else."
People disagree over what constitutes a benefit; something someone finds beneficial in their work is crap to someone else, and the easiest way to disparage someone else's beneficial thing is resort to the necessity rhetoric.
"Everything that I found beneficial in conducting my successful XYZ project, which got to a level of complexity and quality in so many years of effort and lines of code, was obviously necessary; all else is unnecessary."
A really sneaky way to do that is to repeatedly acknowledge the benefit of something while emphasizing the lack of necessity. With a sprinkle of FUD about possible harms that outweigh benefits ...
I really do not understand this whining and downvoting.
In so-called real life somebody has to live with the malformed macro, whose only purpose is to beautify the code.
I even made un-macrofier, which deleted all macros and replaced instances with macroexpand, because slight adjustement of code resulted mysterious errors cause by nasty macroes.
And lets not talk about Common Lisp loop macro, which was probably scooped up from latrine of hell by the Devil himself.
Nobody has to live with a malformed macro because they can just observe what code expansion it is performing and write their own which does it better. (Speaking of CL loop, some people have resorted to this solution of actually dragging an implementation of loop into their application, to avoid issues with a vendor's loop, or behavioral differences among different ones---yes, loop has this, unfortunately).
The malformed macro situation is a lot better than the situation of a malformed API function to which you don't have source code.
Anything that is formed can be malformed. DNA can be malformed, leading to malformed fingers someone actually has to live with; so let's not have fingers.
As a professional - assuming you mean “which can I put on my resume to get a job working in that language” - probably (not necessarily!) none, or maybe Clojure.
As a hobbyist, probably all of them. It’s worth at least trying to go through SICP in Scheme (use Racket with the SICP language), it’s worth learning CL (use SBCL) to fully appreciate everything the Lisp world has to offer without compromises, it’s worth learning Clojure because it’s, frankly, a little bit cleaner and more elegant than CL in many respects.
Yeah but apart from SBCL, Viaweb, Hacker News, Emacs, Clojure, Scheme, Racket, garbage collection, macros, homoiconicity, the REPL, S-expressions, symbolic computation, what has Lisp ever done for us?
I respect Guy Steele immensely, but his view of Java as being halfway to Lisp is sign that he had jumped the shark by that point, and find it hard to take the rest of his work seriously.
Yes, to a certain degree, although it does take initial effort to make it happen.
LLMs do somewhat okay job when you use them with Lisp, treating it just like any other PL, which is roughly the Unix/pipe model - batch-style. Agent spawns process -> reads stdout/stderr -> spawns next process. State lives in-between the calls and in files. Each tool invocation is stateless.
Things get far more interesting when you give an LLM a true Lisp REPL. LLM stops guessing and starts empirically analyzing current state of things and produces working solution faster, costing far less tokens. And you get to watch it solve things interactively, e.g. I often let AI poke through our UI (via Playwright-driven Clojurescript REPL), while monitoring situation in k8s - through nrepl port, connected to Clojure REPL. It literally interactively walks the DOM, finds the selectors, clicks buttons, etc. All without restarts, complex state management and all.
Depends very much on your harness and effective use of tools. For vibe-coding, stick to Python.
It can get good with the right setup. I made Codex work with GToolkit (similarly underrepresented, but technically impressive and a seemingly good fit for LLMs), but it took a lot of tweaking of the project structure, extensive instructions in AGENTS.md, and some custom skills (some borrowed from Gt4Llm, the GT's built-in model harness). Out of the box, it burned tokens and took way too long to implement even basic things.
YMMV - as with everything LLM-related - but I think without a similar setup, an agent instructed to write CL/Clj/Scheme/Racket will have the same issues. It might be better in an established, large project - but starting from an empty Git repo, I suspect you'll have to fight your way to productivity.
I've found LLMs to be bad at balancing parenthesis. I've also found them to be less likely to hallucinate library types in dynamic languages, they tend to hallucinate arguments to library functions/methods instead.
Because you treating Lisp just like any other (non-homoiconic) PL. Give an agent a true Lisp REPL to mess around, and you'd be surprised. Things get very interesting. I still don't understand why more people don't do that - isn't that obvious first thing anyone should figure out? Like I can't even imagine working with Lisp without a REPL and structural editing - I'd immediately fail at balancing parens. Why do you expect a [dumber] machine would do any better?
Yeah. In my GToolkit setup, I started with giving it eval. At first, it tried to "rebuild the world" for each task, but we gradually settled on a set of images for specific tasks that can be rebuilt as a separate step and are otherwise cached. That gave us fast eval, and it kinda snowballed from there. I thought I would need to implement some IPC into a live image, but the startup is fast enough that it doesn't matter too much. The agent now has both the textual source (in Tonel) on disk and can easily query a live image via CLI.
What do you think about making the agent write type annotations? There are built-in forms in CL, and in Smalltalk, I settled for pragmas for now. They are not checked, but since I started using them, I think the rate of one- or two-shotting solutions has gone up.
assuming you're using state of the art llms, when they start failing with paren balancing it means they're approaching the limits of their context window and a new session should be started.
I use LLM_based coding agents frequently with Lisp languages and my way of working is different with Lisp languages: 1) if generated code ever has a syntax error, I like to quickly fix the syntax error myself. 2) for some reason I usually prefer to run tests myself in another terminal (with Python, Typescript, etc. I let the coding harness run tests).
I haven’t let coding harnesses run REPLs. When I do let a coding harness run tests I specify in, for examp,e, my Common Lisp skills file to run ‘sbcl -load …” so bash test commands are one liners.
It would be interesting to work on skills and a harness to use REPLs - nothing bad about that idea, I just haven’t tried it.
Current SOTA models work great with Emacs Lisp and Common Lisp. Giving access to the REPL enables them to do an amazing job but they still work well with Unix input/output static code processing paradigm.
yes if you run a repl server like slynk the llms will happily communicate with your running program to develop and test things. It's often quite creepy to witness.
That might be, on the other side, I consider Lisp highly readable, but quite a few modern languages quite unreadable. For example parsing type signatures with a complex syntax.
Any style of writing you are not familiar with will require a lot of training. Try reading anything written in the Greek alphabet for example.
Its general purpose. For now I'm trying to build out all the basic language features. Once its built I'll use it as my daily driver for whatever I'm coding.
It's a lisp so it can be used for anything since macros have been implemented. You can create your own syntax using macros.
I played a little bit around with LISP but I was astounded when I saw that you run programs using the live REPL. I thought, how can you reliably deploy a finished program if it can get messed with at any time? Why isn't there a way to compile to a binary?
> I was astounded when I saw that you run programs using the live REPL.
That's one way, you stopped too early in your investigation though.
You can produce binaries, though the precise mechanism will vary by your implementation. And there's no reason to use the REPL for it. You can create an executable file with something like this:
(defun main () ...) ; do whatever you need in here for program launch
(sb-ext:save-lisp-and-die "my-program" :executable t :toplevel #'main)
And then `sbcl --load program.lisp` (or whatever you name it) and it'll produce a binary for you. Other implementations will have other methods of achieving the same thing.
Or, if you don't need a binary, you can have something like this:
(defun main () ...)
(main)
And then run `sbcl --load program.lisp`. That will compile and execute it without ever invoking the REPL.
(NB: Using a function named main isn't strictly necessary, I named it that for the example. Name it whatever you want.)
I’m guessing there’s some way for a Lisp program to load a DLL and call a function in it so writing a Windows application using the Windows App SDK is likely possible.
The one part I can’t imagine is how do you pass a Lisp function as a callback?
https://www.quicklisp.org/beta/UNOFFICIAL/docs/cffi/doc/Tuto... has some pointers (which might be harder to understand unless you have read the rest of https://www.quicklisp.org/beta/UNOFFICIAL/docs/cffi/doc/Tuto... first). The short version (which I might be mangling in my attempt to shorten it) is, since Lisp is so very good at creating code, you have your Lisp program create a C function that will be passed as the callback to the API. Then the C function that you created will convert the C data structures to Lisp data structures, and call the Lisp function with those Lisp data structures.
If you're really good at Lisp, you can write that automatic-callback-creation function yourself. But if you're really, really good at Lisp, you will just use one of the ones that other people have already written, e.g. `define-alien-callable` from https://www.sbcl.org/manual/#Calling-Lisp-From-C or some other similar FFI.
Yes. Kandria [1] is a great example of someone doing exactly this. There's a bunch of information on the development here [2] and the code is open source [3].
I really love the idea of lisp but feels like the learning curve is steep. Sure you can learn basic lisp in an hour or so but then? You need to learn emacs, all of the plugins, module systems.
> [...] among its extensibility, its interactive environment, the REPL, and a lot of other features we haven’t touched yet. It is the combination of all of them that makes Lisp programming what it is.
Agreed. However, although the alternatives are few, they do exist. Today, I'd like to convince you (whether you're OP or a commenter) to give one of them a try. I'm talking about GToolkit[1]: Smalltalk/Pharo-based reimagining of Smalltalk as a productive environment for modern system design, analysis, and implementation. It's based on Pharo and its VM, but with GT-specific extensions and replacements, developed on GitHub in both Smalltalk and Rust.
I used both Common Lisp and Smalltalk over the years for some of my side projects. Technically, the environments are comparable: image-based, live, interactive development is central to both. Lisp is easier to fit into modern workflows: it's still just files on disk. If you ignore the REPL and treat the image as a bundler, you can have a Java- or Python-like development workflow. You can gradually adopt the more interactive ways of working with the codebase. Smalltalk was historically more of an "all or nothing" approach, but nowadays it supports Git-based workflows as first-class, with GToolkit providing additional tools directly. GT is tied to a single IDE, but that IDE is genuinely powerful, pragmatic, and easy to customize: the entire IDE is Smalltalk code[2] that lives in the image beside your code, so you can live-edit any part of it at any time.
The languages differ, most obviously in the object model (single inheritance and message sends in Smalltalk, and multiple inheritance with multimethods in CLOS), but I don't think one is strictly better than the other for the vast majority of code. You just need to structure your code differently. Both systems are very dynamic, so neither is like Java or C++. After quite a few projects in both, I am convinced that this difference doesn't matter.
Both languages have very simple syntax. Smalltalk doesn't have macros; instead, it has an extensible/replaceable compiler for method bodies. This capability is used, for example, for compiling grammars into parsers/lexers in SmaCC. It's not as convenient for control-flow abstraction as CL's macros, but Smalltalk tends to make them out of blocks (lexical closures with non-local exits) plus actions on thisContext. In practice, I never found either language too limiting in what I could express, and both have small, regular basic syntax.
TL;DR: Common Lisp and GToolkit Smalltalk offer comparable technical merits, with the most pronounced difference being GT's built-in, high-quality IDE, which open source Lisp lacks.
The IDE (and the capability to build cross-platform, but natively rendered GUI apps) is the major selling point of GToolkit, but the bigger reason to consider it is social. Lisp had a bit of a renaissance from 2005 to 2015, but ~it has since died down~ (EDIT: After checking a bit, it's more like there are similar levels of development now, but it's just varied, and I personally hit a few areas where nothing new happened for a while. Or in other words, the hopes of CL getting catapulted to mainstream by the wave of adoption didn't materialize, and the adoption seems to have largely plateaued, with a slight upward trend since then.) A lot of great things for Lisp appeared in that time: ASDF and Quicklisp being prime examples. Unfortunately, the momentum was lost, and since then, the rate of development has plummeted. It's still a solid proposition because of its stability, which means a lot of old code still works perfectly well, but it's a double-edged sword: it would take a hundred miracles in a row for CL to get a plausible M:N concurrency story, for example. This isn't true for Smalltalk, and doubly so.
There's a small but active community around Pharo, mostly academics and hobbyists (IME; and it's from a decade ago, so YMMV). Pharo itself is already a "Smalltalk-like" language, and it regularly gets features that are not in original Smalltalk (stateful traits and slots (reification of class and instance variables) are good examples). It continues to evolve, and each version brings notable improvements to the language, the VM, and the class library. In CL, since there are many implementations maintained by various groups and the standard is set in stone, adopting extensions to the standard is incredibly hard and time-consuming. You can, of course, commit to a single implementation, but there are only 2 or 3 implementations that are actively developed and trying to do something "new"; they are all experimental and incomplete. And while SBCL has quite a few nice features, its codebase was a bit hard for me to grok (read: days on end of banging my head against a wall, then giving up; I'm much more comfortable messing with Smalltalk bytecode compiler).
Secondly, GToolkit is driven by a group of coders[3] who have established a profitable consultancy. It's being developed around a single vision that the authors swear helps them achieve their projects' goals. Even if you're not sold on moldable development[4] as a methodology, GToolkit gives you all the tools you might need in a modern development and then adds tons of domain-specific tools and utilities that might be useful after just a bit of adjustment. The whole environment is built to make such adjustments as painless as possible, too.
The only problem I experience with GToolkit is the lack of structured, book-like documentation. The GToolkit book is closer to the PHP wiki or a bundle of tutorials than something like the Rust book. It's not that the docs aren't there - they often are, but finding them effectively is a bit challenging. I found that simply cloning all the repos that GT consists of (tens, if not hundreds, but there's automation for that) and pointing an agent to the directory with them is often enough to quickly find what I'm looking for (if the built-in Spotter fails and I'm too lazy to construct an in-image search).
TL;DR: GToolkit/Pharo move much faster than the CL world, and while the number of maintainers might be similar (honestly hard to estimate, though), the effort on the GT side is more centralized and, in my opinion, heads in a better direction. If you want an extensible, live, dynamic environment that shares many of Common Lisp's strengths, but is more polished, more actively maintained, and feels more modern, take a look at GToolkit.
(Disclaimer: no affiliation, just a programmer who likes to explore unpopular languages).
All three major programming environments at Xerox PARC, shared similar concepts.
Interlisp-D, Smalltalk, Mesa (XDE) which evolved into Cedar.
If you read Xerox papers about all of them, there are several quotes on how relevant it was to share the same programming experience across environments.
Which is why, given their linage, JVM and CLR are the closest big mindshare ecosystems that somehow still have traces of those features when using their IDEs and runtimes, even without being a proper Smalltalk or Lisp.
> how relevant it was to share the same programming experience across environments.
Right. If you view the language as just a part of a bigger "programming experience", you can do with a worse language with better tooling (pathological example: (lack of) namespaces in many Smalltalks to this day). If you focus on language design above all else, you end up with either something pretty on paper (but never implemented) or something practical and (kind of) elegant, but it'll take 40 years to get the tooling to where you want it (and still won't cover many important capabilities).
Also, it's easier to swap languages if you focus more on tooling. From what I read, many Lisp and Smalltalk IDEs/environments were (and some still are) polyglot. I sadly didn't have access to them at the time, but I imagine switching from Turbo Pascal to Turbo C++ was easier than going from Turbo C++ to Visual C++? That said, tooling tends to break if the language actively fights it, so it's not like PL design is completely irrelevant.
> Which is why, given their linage, JVM and CLR are the closest big mindshare ecosystems that somehow still have traces of those features when using their IDEs and runtimes
Yes. The only problem is that those concepts/features become niche black magic (somewhat unavoidable for advanced features in "big mindshare ecosystem"). You can definitely compile Java classes dynamically at runtime, install them, and use them normally; however, outside of frameworks/IDEs/JVM languages, the ClassLoader is used mostly for loading static assets, and even simple reflection is viewed with suspicion.
Using Smalltalk or Lisp directly puts those powerful concepts "front and center", which a) is good for education; and more importantly, b) feels good when things click :) It could be an advantage for small groups of programmers, but from what I understand, it becomes a liability in large organizations. Pharo is especially bad at it, and GToolkit doesn't fix everything (they do a good job at providing tools, but don't want to maintain too large a patch set for Pharo, which is a good trade-off given their limited resources). But I'm replying to "Why Lisp (or When)", and in this context, Smalltalk is (IMO) the better direct alternative than JVM/CLR.
I am fascinated by lisp but only had the time to mess with clojure for about a year.
My weird take is that lisp would have a better shot if one would replace the parentheses with square brackets. It might just be me but the round parenthesis make it visually painful to look at code blocks. I feel square brackets would fix that.
It's really fun: I read so many propaganda on Lisp here that by now we should all be coding in it. Don't we get perhaps weekly posts like this?
But ... somehow almost everyone simply ignores this propaganda --- that's actually the way we ought to handle propaganda! Funny that it works with Lisp, but not with many political topics where it can be shown that propaganda really works.
My experience with Lisp is somewhat special-cornered. I only know Emacs-Lisp, and that not great. And for me, Lisp an out, outdated and confusing thing.
I don't know ANY other programming language that elected assembler mnemonics to be "high"-level expression: car and cdr.
While I love Emacs, they claim all the time that is has 100% discovery through build in doc. But that is only 70% true. It doesn't exist in Lisp like it does in other languages. For example, I have an association list. Now, what functions exist that uses them? I can't use the equivalent for (dir alist), alist.dir(), or use some LSP for that. I can't even look in the symbols, because operators are written like assoc or assq with not logic. And then there are either prefixes to them (rassoc) or postfixes (assoc-string).
That there is no namespace is soooo 1970. I mean, even Turbo-Pascal had namespaces.
Typechecking is nonexistant.
At least we have byte-compiling.
And note that I don't croak about the parenthesis. While I find it mildy weird that I as a human program in almost-AST, Emacs has superp support for the parenthesis. So people croaking about them just had never had a good editor IMHO.
Now, one can argue that Emacs Lisp != Common Lisp. But these propaganda articles don't tend to promote one specific Lisp, they only talk in generals. So I can apply this general principle and compare it to the tiny corner of these awful language family.
Programming is in tension between the Light Side and the Dark Side.
The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!
The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.
Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
I'll paraphrase someone who commented on HN once (about dynamic vs static typing, IIRC): permissive languages are enablers for solo programmers, but you need more restrictive languages for team programming.
This matches my experience because you don't chose who you work with. Skill level is uneven among the team. Less skilled co-workers will make mistakes that will have more consequences when the language is more permissive.
That's one of the reasons you want code reviews, but then you convert your skilled programmers to teachers. This is not a good trade, because the time they spend improving the skills of other programmers is partially wasted because programmers are free to come and go. AI could change this picture.
The name of the game is therefore to keep your teams small and skilled, but this is can be difficult because of typical company politics - e.g. "bus factor" considerations, shortening time-to-market is often achieved by adding more programmers.
> Less skilled co-workers will make mistakes that will have more consequences when the language is more permissive.
Even the most skilled engineers are going to make these mistakes sometimes. That's the whole point of more restrictive tooling, and why Rust has exploded.
Don't get me wrong. I love Lisp, and use it in some personal projects. But I wouldn't want to use it in must not fail scenarios unless those situations can wait for a human to fix the running image.
Yes. I'm moving the goal post a bit here, but I was actually thinking about design mistakes and dangerous constructs - the kind of thing you do when you write in a "hacky" way - either deliberately or because you don't know better. Although these are not directly under the scope of type checking, a more restrictive language can have a positive influence.
I agree with you. Though I wonder if the divide is more about the size of the program. I've got a couple of large (>100K LoC) solo projects and I like using C++ because it is much easier to catch bugs at compile time.
Static typing is a huge advantage in refactoring, and if you don't refactor a large program as it grows, you end up with a mess.
Clojure programmer here. Refactoring is the only time I miss static typing.
>This matches my experience because you don't chose who you work with.
That's what job interviews are supposed to be for. It should be easy to hire only the most skilled programmers, especially with the massive number of unemployed programmers out there right now.
Programming skill is similar to driving skill. You can hire the best people in the world and there'll still be incidents. The people who are most skilled may even be more incident-prone because they have the skills to take more risks.
Don't bet the company on no one ever making a mistake. Set up a system where the unintentional risks are minimized and the consequences of inevitable failures are mitigated by seatbelts and insurance policies.
Maybe it's trivial what I'm saying, but hiring the most skilled available person does not solve the skill disparity.
Just look for the people who like Rust and filter them out
I know this is obvious trolling, but it’s close to a truth. Selecting for a passion for languages that aren’t anyone’s first language gets you polyglots—folks that’ve had to abstract their knowledge and understand what’s an ecosystem feature vs what’s more broadly applicable.
Joking about Rust users rhymes with joking about vegans; it’s true both parties generally can’t keep from talking about their choice. But you do want to hire someone who’s thought about their options and made a decision.
Lisp generally has precise GC, which I'd say makes it light side. It's even relatively type-safe if you count runtime type-checking.
Highly reliable systems are written in Erlang, which if you squint is another Lisp dialect. There's even a sexp-based version called LFE, for Lisp-flavored Erlang. Erlang's key to reliability is error recovery, rather than exceptional levels of error prevention.
I do like your light side/dark side classification.
> Erlang, which if you squint is another Lisp dialect
Prolog disagrees. IIRC the first versions of Erlang were written in Prolog, and you can still see its influence in the syntax.
Neither use as an implementation language nor syntactic similarities make one language a dialect of another. Consider that Java is commonly implemented in C++, and its syntax was designed to be easy to learn for C++ users. Yet nobody calls it a dialect of C++, not even when squinting really hard.
Erlang's computation model is functional, like Lisp, unlike Prolog.
Correct, although I would assert Prolog shares many concepts with Lisp.
It does. In Prolog, data is code is data as well.
If you squint enough, JS is a lisp..
Define a goddamn language, syntax is not enough to define one! What are the semantics? Without that you are just talking about syntax trees like they would mean anything
never understood why people say that: the syntax for defining code seems quite different from the syntax defining data structure. There's no homoiconicity in javascript..
And that's just syntax, it doesn't give you a programming language at all.
JS is a dynamically typed language with prototypical inheritance objects that work like universal key-value maps for the most part. It is also mutable.
Clojure is a dynamically typed language with key-value maps. It is also immutable.
You can surely see where I'm going , the underlying semantic model is the meaningful part. Homoiconicity doesn't give you anything special if your language can parse itself and can eval code. It just makes these completely abstract implementations simpler.
> Homoiconicity doesn't give you anything special if your language can parse itself and can eval code. It just makes these completely abstract implementations simpler.
Well, in a way it does give you something: By making expression of things like macros simpler, it makes them sometimes worthwhile, and makes it a reasonable request to have this kind of meta programming in your language at all. Without homoiconicity such things become even more difficult endeavors and often unjustifiable for the language design and its implementation.
I mentioned in other comment that homoiconicity doesn't necessarily make writing macros simpler. It makes writing trivial toy examples simpler.
But let's compare it to a modern macro system like rust's or scala's, where you get a typed object representation of the AST, and for anything non-trivial you are better off with this latter.
Also, arguably the best is to have certain features in the language itself, that can be used to build proper abstractions - so you don't have to resolve to using macros in its place.
> Also, arguably the best is to have certain features in the language itself, that can be used to build proper abstractions - so you don't have to resolve to using macros in its place.
I don't agree, because that would mean, that the language must be huge, or grow huge over time, or alternatively be extremely abstract at its core, to be able to fit every use-case. Furthermore, so far I have not seen a language, in which the language designers managed to pull it off, so I tend to think that what already has been successfully pulled off, which is macros for language extensions, is the way to go.
Also this seems to be arguing from a limiting idea about what macros do. Macros are not always the right solution for any problem, in fact often they are not, but there are things you simply cannot do otherwise (without syntactic clutter), like for example changing the order of evaluation.
I also don't agree with your point about only making toy examples easier to write. For example I have written macros for implementing new define forms, which allow to specify contracts for function arguments and return values, or a macro for automatically defining functions that communicate to API routes, based on the function name, which I used to implement a proof of concept docker client for Scheme. Those are not toy examples, but real world applications, where a small macro can have big effect.
If you think macros need to be big and elaborate and complicated and otherwise are toys, then you don't really understand the power of macros. One of my favorite macros is the following threading macro:
Small, but a great addition to the code, that improves readability in many places of the code. It doesn't have to be big or long, in order to not be a toy example, but actually be a useful macro.Well, my main point is not to never use macros, but that in certain cases a language-native feature that can also solve a problem you would use a macro for, it's probably better (better debugging, error messages, etc).
And my other point was macro systems in other languages, which I believe are better, in part due to not having homoiconicity, like Scala or rust.
> If you squint enough, JS is a lisp..
Yes, that is true. I'm not big on the idea that Lisp is defined by parentheses. The implementation strategies are another way to look at it. That doesn't capture it either, but it's an angle to try.
So then what it is? Because otherwise it's a magical nothing-term to which everything lisp people like applies, but no criticism can ever reach it because "that's not really lisp, see it's different in this other implementation"
In my mind, it's 2 distinct criteria: interactivity and the manipulation of symbols (whether those are implemented as symbols or identifiers doesn't really matter). I don't know about Erlang, but from the admittedly little JS I've written, I believe it achieves both criteria, even if it's worse at them than a "proper" Lisp.
It isn't, Dylan and Julia are two Lisps with Algol like syntax.
Indeed there was a proposal to add Algol like syntax to Lisp,
https://en.wikipedia.org/wiki/LISP_2
JS is Self, which is a dialect of Smalltalk. So not really Lisp imo
I think the big thing is that Lisp is (aside from mutable variable assignment) basically all declarative, rather than the imperative paradigm.
Even without static types, and even allowing macro craziness, there's just such a stronger baseline of declarative and functional thinking, you're off to such a good start in clearer thinking and reasoning about a program.
This is just not true. The Lisp family contains all conceivable variants of languages, from statically typed Coalton, to dynamically typed Scheme. From functional, immutable by default Clojure to the imperative-style of Common Lisp.
There are prologs, constraint solvers, ffis, JavaScript alternatives, garbage collected and not garbage collected languages that call themselves lisps.
I won't disagree but most books and people who made lisp in the early decades were somehow hinting at not manipulating state but trying to represent ideas and operators to raise the level of expressiveness without shooting yourself in the foot (as much as possible)
I think this depends on the Lisp, no? AFAIK Common Lisp supports a lot of imperative style programming--besides all the mutation/assignment functions, there's the `prog` macro that lets you use goto, `do`/`do*` to iterate over groups of statements, or even the `loop` macro. OTOH the Scheme-style Lisps are much more declarative thanks to TCO and a community that prefers the functional/declarative programming style.
But again, I suppose all the Lisp forms return values, and quibbling about the declarative : imperative :: expressions : statements mapping is just petty semantics
> But again, I suppose all the Lisp forms return values, and quibbling about the declarative : imperative :: expressions : statements mapping is just petty semantics
I would argue that it's very much not just semantics, or at the very least it's certainly not petty. The distinction has a noticeable effect on what the experience of writing code and on the reliability and related properties of the written code.
Yes and also how you use it. The more you squeeze performance out of CL the closer the code looks to C in s-expression format, the less you squeeze it the more it looks like a DSL. The bridge between the two is macros so that you can have both the performance and expressiveness.
I like your metaphor but I'd argue that trusting programmers with power is the Light Side.
Relevant PG: https://paulgraham.com/langdes.html
That's a great essay--thanks.
And you have a good point. Maybe it's a weakness in the metaphor if one can see it either way.
I fundamentally believe that there is no perfect language, instead one merely chooses a set of trade-offs. What's interesting about Lisp is that it is simultaneously well-respected and yet not widely used in productions (compared to C, C++, Python, JavaScript, and even Rust).
It's almost like Lisp chose a particularly extreme set of trade-offs that yield great power but also repel large swathes of programmers. I think PG might say that it repels dumb programmers, but maybe not.
Our entire profession has been moving more and more in the "bondage-and-discipline" direction over the past 10 years or so, tracking with broader sociological trends that are also justified with arguments involving "safety". That by itself can explain why Lisp has such low adoption. While Lisp has never been the most-popular language, it was never as unpopular as it is today. If that pendulum swings the other way, Lisp will get more popular again.
I think it’s just that the syntax is more difficult to grok. When I first saw code (PHP and JS) it was fairly easy to read and figure out what was going on. I’m now experienced but I’ve never tried Lisp, but reading Lisp code takes considerably more effort than when I first saw the PHP/JS. I’m sure with very little experience it becomes just as easy, but to the novel eye I think it’s simply less intuitive to reason about.
You mixed up Light and Dark. Creator-knows-best bondage-and-discipline is the Dark side. Trusting people with power is the Light side.
Light and dark is propaganda depending on which side you're on.
I'm with you, on the dark side I've seen people slowly turn into crabs, screeching about memory safety and static types. Can you imagine that? Now stick those wings on me, I'm going to fly close to the sun.
Programming is in tension between the Light Side and the Dark Side.
The Light Side believes in giving people freedom and power to achieve freedom, to allow themselves to express themselves better through programming, to become better programmers through the tools the light side provides; in the Light Side believes in programming as a way of thinking.
The Dark Side believes that people are incapable of handling freedom and must be restricted and controlled under the service of various organizations - generally companies. The companies will control access to programming and control the programmers via this access. The dark side does not believe in making programmers better programmers, it believes in managing programmers. The dark side does not believe in helping programmers to think, it believes in thinking for them.
Kind of agree, but in my view preventing the programmer to make mistakes is futile. I have seen awful stuff in languages made to prevent errors.
It's much better to give all the power to the programmer, to allow him to fix his mistakes rather than fantasising about preventing them.
I'm not big on static type checking in most situations -- adequate testing should find the type errors too -- but one place it looks very useful is avoiding data races in multithreaded programs. See Rust.
... or Ada.
I wonder what side you would put Malbolge,
What side for python? Haskell? Idris? Clean? C? K?...
white/dark side is too much of a simplification to me.
Programming language equilibrium is a very complex subject and fascinating one.
After years of experimenting with many of them, it is all a matter of context of usage, personal preferences, objectives, etc.. there is no such thing as "the best programming language ever", neither "best for light side programming" nor "best for dark side programming".
Similarly, there is no such things as: Light side/Dark side in artistic painting. There are tons of paints some are clearly greater than others, but there is no such thing as "the best painting most beautiful for everyone seeing this paint comparatively to any pain".
[^Malbolge]: https://en.wikipedia.org/wiki/Malbolge
[^Clean]: https://en.wikipedia.org/wiki/Clean_(programming_language)
[^K]: https://en.wikipedia.org/wiki/K_(programming_language)
Malbolge is not an instrument for building, come on programmer.
C is clearly the dark side with to chance to protest.
The only way to consider K as dark side is its speed constraints relative to my favorite PL which is J. K is so fast because its interpreter fits into CPU cache.
Haskell is a border case between sides because the idea of PL is so light that using so much light in programming leans to be really dark. Haskell extracts the cognitive friction from the code environment to the types environment, programming in paper is not a joke for Haskell.
Python is definitely Light Side
If there is any language that has ever been, Python is the very definition of light side, so much so that anything that needs to be done in the cover of night, is purposefully directed elsewhere by rather shady dark siders (C).
> Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
This isn't so far off, considering that some people consider the "Lisp in Lisp" bit from the 1.5 manual to be the "Maxwell's Equations in Software":
https://michaelnielsen.org/ddi/lisp-as-the-maxwells-equation...
I have always assumed that Alan Kay compared Lisp to Maxwell's Equations because of the similarity between the interplay of eval and apply in Lisp on the one hand, and the interplay of the electric and magnetic fields in Maxwell's equations on the other.
Also because lisp core can be reduced to a few "equations" (at least that was my understanding)
Maybe machine language is the formless chaos and Lisp is the purest manifestation of order. All other languages are points in between.
That makes no sense without specifying what Lisp actually is.
S-expressions are a generic tree, that's not a language, that's just syntax.
Is it CL, Clojure, scheme?
Like these don't even share a basic object model, it's like getting the AST of Haskell and C and then talking about how good that AST is.
See Cardelli's breakdown in his famous article "Typeful programming":
http://www.lucacardelli.name/Papers/TypefulProg.pdf
Page 52 perhaps.
But there are two types of “programming”: 1) programming per se (just you writing code for fun and perhaps building your own tools) and 2) a team of people with real-world constraints writing software for others for revenue
It’s clear why it feels so good to use something like Lisp for personal projects. It’s also clear why the vast majority of companies don’t use it
You have it inverted. Your "Light Side" is what previous generations of programmers called "Bondage and Discipline", because this philosophy assumes that a good programming language has built-in shackles to impose the will of a so-called "benevolent dictator" on all programmers.
It isn't nearly as much of a trade-off as people say. Languages like Haskell are both remarkably expressive and provide a lot of safety. (And, of course, languages like Python, Java and Go are the opposite.)
Haskell is currently at #18 in LangPop: https://langpop.com/rankings
Almost by definition that implies that it makes some trade-offs that turn off lots of programmers. Still more popular than Lisp, though.
Language design has almost nothing to do with language popularity.
People learn JavaScript or TypeScript because they want to write web apps. Swift or Objective C to write iOS or Mac apps. SQL because there’s a database they need to get data out of. Python because there’s a machine learning library they need to use. Etc etc.
Language design is far down the list of priorities.
That's fair, but I was responding to the claim that you don't need to make a trade off. If that were true, then Haskell would be much more popular.
Clearly there's a trade off or else everyone would just use Haskell.
That doesn't follow.
For one, there might be trade-offs against dimensions other than safety and expressiveness. Performance, corporate support, libraries, popularity, learning curves, similarity to other languages...
For two, popularity is simply not strong correlated to quality. Popularity is a social function driven by, well, social factors. It's heavily path-dependent and noisy. There is absolutely no guarantee, not even close, that the "best" thing in any sense will be the most popular, or popular at all.
I would refine that: Language design has only indirect relevance to language popularity.
People go where the money is, hence python. But python is widespread because there is employment in it. That adoption by employers follows a network effect, but it originally came about through the language design.
Early-adopter => mainstream => long tail
Adoption hinges on that jump from evangelists to the mass market, and that implies:
- attracts early adopters because of language design - early adopters can pitch the language to their bosses by showing cleat benefits, also because of language design
The most obvious exception is javascript, which was inflicted on the world by browsers. Even so, it had to be sufficiently shit for the php crowd to feel at home. (Unsure which came first, but you get my point.)
Php is exactly popular due to design. It was an exact fit for the myspace dotcom era, and there were no json APIs to illuminate its shonkiness. (I am still angry that [ ] serialises to either [ ] or { } depending on a fucking global variable. I cannot think, being maximally charitable, of any value that comes from conflating arrays and hashmaps. No, i do not want an array containing the keys 0 and "0". I can count on the fingers of one foot the times i have wanted that.)
Sorry - my therapist encourages me to rant about php. She said it's part of healing. I haven't even told her about back-end php yet.
When Python got its popularity, programming was an in-demand skill, so the thing that determined adoption was being easy for beginners to learn. Python was extremely beginner-friendly, so it became popular.
Now, the thing that determines popularity is popularity. So Python still wins, even though its beginner friendliness isn't so important anymore.
I've never much luck selling a language on cleat benefits alone, but I guess there's not many climbers in my department.
Those are track cleats, because sports metaphors drive middle management
That argument seems crazy to me. 18th is so good. There are literally thousands of programming languages that it is competing with.
It may very well be the case that it makes trade-offs that turn of lots of programmers but I don't think it's popularity directly implies it by definition, rather just that it implies there's something about it that prevents lots of programmers from taking it up en masse, and I think that's perhaps just the fact that it is so different from what everyone's used to and it takes some time to adapt to it enough to be able to appreciate the tradeoffs.
Only if you knew the power of the Lisp side…
for a while there's been a strong cultural tendency in programming to mistrust people and trust tools (often even blindly because tools are just assumed to make no mistakes), and so expressiveness has been sacrificed for safety. There can be something to this but it's also self-fulfilling, if you strip people of agency of course they'll unlearn to program.
I've always thought it's a misanthropic philosophy and sucked much joy out of programming but also there's something to be said that there's safety in the expressiveness of Lisp. I read an article a while ago that found a robust correlation simply between length of a code base and errors regardless of languages used. And given how succinct and clear Lisp codebases can be that's valuable in itself.
we're kind of at the logical endpoint of this now with gigantic slop codebases that nobody understands just held together by 20 different tools, and if you ask me if I had to pick one of those or something one tenth the size written by a guy or girl who has been writing Lisp for ten years I'd say thank you I'll go with #2
The issue is also social, the average software engineer will treat Lisp code as a liability because most people really don't know what to do with it. In the mind of a professional SE, code is "maintainable" only if it's writing in Javascript or some other commercial language.
Yep, you have to use C# and Java because they are safe. If you want anything extra, you need to use $library, because only $library's maintainer is trusted with the dark corners of the language. Look how powerful we are, millions of libraries. Look at those Lisp guys and their sorry examples of package managers. /s
If there's one thing I've been happy with learning Common Lisp and Clojure is that you only want libraries for things like algorithms, data formats, and protocols/interfaces. Mostly because those often have standards. Anything else should be a snippet to copypaste even when presented as a full program.
> Do not allow a bug to be expressible
That is a very powerful idea, but unfortunately it cannot be realized with a fixed set of language rules. Some invariants are universal, but some are bound to the domain. Some projects require, say, an int (EvenInt) to take on only even numbers and others odds (OddInt). You can imagine the possibilities here are unbounded ("the numbers should be prefixed by numbers that are divisible by my age squared").
Ideally your base language has the expressive power to formulate new abstractions and constraints. This fundamentally requires the Dark Side. Once the proper primitives are in place a different "language" - this can be literally or figuratively - is used to express the interplay between those primitives.
In effect you are starting with the most general (say, all of Lisp), restrict it to become more and more and specific - only "these modules" - until it cannot be reduced any further. If you can bring your domain down into being expressible as, say, a single config file, that'd be quite ideal. If you don't need the powers of abstraction to express your solution then exposing said powers would only invite in trouble. The mathematical equivalent of introducing unnecessary variables.
As a programmer you are free to traverse the journey from say all of Lisp to a JSON config file in whatever way you please.
The "Light Side" people have converged, or try to converge, on some intermediate state between full powers of abstraction and configuration only. I think this is useful because many if not all problems travel through that intermediate landscape on the way "down" (into their specificity). For example what you call "types" is a significant restriction on your freedom but at the same time it's so enormously general that this restriction can be considered a worthwhile default because just about any problem I can think of can potentially benefit from that restriction.
All this is to say that I don't think it's a dichotomy so much as two interacting polarities whose interplay gives each its strength.
Lisp is a particularly minimal base introducing very few restrictions of its own. But it's not the only one. Forth would be one in another direction.
Well put. Minor quibble: I see what you're driving at with "inexpressibility" but i think i could write EvenInt and OddInt in F#. I don't know Haskell, but i believe you would have to work hard to find a domain constraint that you can't declare in the Haskell type system.
You are, of course, right about EvenInt being easy to express in a type system. In fact, I believe type systems are generally so expressive as to allow just about any constraint to be expressed (eventually..).
I notice a lot of arguments on this topic resolve to "but Turing", which is not completely uncalled for but I think misses the point a bit. Not because it is wrong, but because it highlights the wrong property.
I don't doubt, say, Brainfucks ability to express any arbitrary computation, but I do doubt its ability to do so sanely. Now you may say Haskell's type system is very clean, but some constraints will definitely push it out of its comfort zone. I'm not saying it won't be able to express them, but I am claiming there will be dragons. One example of a constraint that's at least awkward is when the absence of a fixed type is part of the design. There are of course myriad solutions to this problem but they generally all require not quite so straightforward constructions that eventually might make sense with enough exposure, but whose complexity can actually be disproportionate to the value of the guarantee.
It's very much a testament to the genius of languages like F# and Haskell that you have to think hard of practical counter-examples.
That said, I actually think the Dark/Light polarity rears its head again even deep inside Haskell as any sufficiently complicated software system encounters barriers it needs to overcome and those require carefully constructed escape hatches: unsafePerformIO, metaprogramming, that is to say, the (relative) Dark Side.
In that metaphor undisciplined use of Lisp is like a vast, dark gravitational field of possibility and Haskell is like a sea of light with some carefully marked dark patches.
you understand. It's also endlessly funny to me it requires anonymous accounts to speak the truth about programming languages.
Hah, didn't think of it this way... To me it was if it allows live updates (+support for them) or not :)
I think of C as more of a dark side language because it's basically a portable assembler. Lisp at least prevents you from doing the stupidest things like overrunning array bounds or writing to freed pointers*, which C is perfectly happy to let you do.
* Of course there are no user-visible pointers** in Lisp so the concept of free vs. not free doesn't come up because memory allocation and garbage collection are automatic.
**Under the hood most everything in Lisp is a pointer, but the user cannot see them or mess with them.
I dislike when people use the term “power” to describe making code slightly shorter to type. Operator overloading is not in any meaningful sense more powerful than a language which lacks it.
You say that because you lack experience in writing programs for scientific-technical computing, where an abundance of complicated mathematical formulae are necessary.
Most programmers have seen only programs that do more data movement and comparison than actual computation, but nonetheless there are programs with abundant computations, the kind of programs that were originally written in Fortran, but nowadays they may be ported to other languages, or alternatives for them may be now written in more modern languages.
In such programs, overloaded operators do not make programs "slightly shorter to type", but they reduce the amount of text at least an order of magnitude or even much more.
The main advantage is not that you type less, but that you can read the source of some function in one page, so you can see it in its entirety, instead of having the source spread on many pages, forcing you to wander through all those pages continuously, when you try to understand what it does and whether it does it correctly.
I have written such programs, for instance which were full of long formulae where most variables were complex vectors or complex matrices (for computing some radiated electromagnetic fields with the method of boundary elements). Without the operator overloading of C++, reading them would have been extremely tedious and maintaining them would have been very error prone.
For myself, any programming language that lacks operator overloading, e.g. Java, is disqualified, because it definitely is not "powerful" enough.
Operator overloading also increases the safety of a programming language, because you can define distinct data types for each kind of physical quantity, allowing the compiler to detect and reject the invalid operations. Without operator overloading that would inflate too much the size of the source text.
> where an abundance of complicated mathematical formulae are necessary
I think what you mean to say is, a lot of linear algebra is involved. So multi-dimensional structures like matrices and vectors are expected to have linear arithmetic operators defined over them, because they're used quite a lot.
You don't need operator overloading for that. Your language should be providing them as primitive structures and extending operators over them. Think like how C has a polymorphic addition, where '+' doesn't discriminate between integers, pointers and floats.
There are a lot of problems with user-facing operator overloading, but I think Haskell is actually a really great example of how to do it sanely. At the point in which you have type-families, you can properly restrict the semantic domain of a polymorphic function but still keep it an open set. The addition operator being restricted over types belonging to the Num type-family as an example trivially allows you to use '+' for matrix addition.
https://hackage-content.haskell.org/package/matrix-0.3.6.4/d...
Now you've avoided the worst part of user-facing operator overloading (juniors creating incoherent DSLs by abusing the mechanism) while still providing the mechanism where it's useful.
I agree with you on overloading - the real word here should be "expressivity", which is defined by language constructs that could NOT be syntactic sugared into a local context. Those may occasionally be useful, though not by itself. E.g. gotos are more expressive by the above definition, yet it was largely deemed to not be a useful "power".
Certain kinds of GOTOs are indeed proven mathematically to be necessary to write certain programs, in the sense that eliminating the GOTOs can be done only by making the program more inefficient, i.e. by increasing the number of executed instructions and the amount of memory that is used.
Because of this, any decent programming language must include GOTOs, but in many modern languages they are not named GOTOs, because of the bad reputation of the word.
For instance the language Scheme does not have GOTOs, but it has mandatory tail call optimization, which means that a "tail call" is just an alternative name for "GOTO". For example, this allows the writing of a state machine in Scheme, exactly like it would be written in a language with GOTO, but using tail calls instead of GOTOs.
Other languages have labelled loops and they allow exit a.k.a. break with a label and next a.k.a. cycle a.k.a. continue with a label. Such instructions with labelled targets are just GOTOs with a bad placement of the label, which makes reading the source more tedious than with a classic GOTO.
For providing the benefits of GOTO, a restricted variant is sufficient, i.e. a GOTO that may jump only forwards and which cannot jump inside nested blocks.
With operator overloading you can provide same numeric operation interface to library types such as big integers. Sure, you can also define methods and make sure your primitive types also define those methods but now you have some confusions (like equals vs == in Java).
Maybe you've never been in a codebase that overloads the comma operator.
Check out Clojure spec. The light dark bifurcation doesn’t hold up.
it's dark side-ish at the ast level mostly, which is not the same as the dark side of mutable state over byte arrays
Typical ignorant take. Lisp macros allow you to create restricted DSLs to prevent bugs from being expressible, including full static type and structured programming systems. Basically no one understands this until they reach the 'lisp enlightenment', so the decades of beating this drum falls on deaf ears. AI is here now so it's time to let it all go.
Of course you feel this way. You just finished reading SICP or maybe you binged PG essays last weekend. But eventually you'll read Simon Peyton Jones and start screaming about functional programming and algebraic type systems. Then, if you're lucky, you'll get a real job and realize that languages are just a tiny part of software engineering.
I'm excited for you to experience that journey.
Wadler's old article comparing Miranda (a Haskell forebear) to Scheme might be a better place to start.
https://www.cs.kent.ac.uk/people/staff/dat/miranda/wadler87....
This was a very pleasant read.
Do you like Apple ]['s?
You've got a good point. Code generally needs to be read more than it is written. At the very least, you realize this as you try to revisit something you had done previously and are left scratching your head. Encapsulating the complexity in a custom DSL can be great for simplifying all the code you write in the DSL. But it also raises the stakes - if you ever have to revisit the DSL implementation the complexity is there lurking.
cute but you don't get lisp yet.
I would be very interest in seeing how "getting lisp" enables you to write software that is more successful than the C and C++ software that runs the world. Perhaps you have written software in Lisp demonstrating this? Something you can show us?
The reference text on time keeping used to be implemented in Lisp.
Alan Kay says a lot of problems with C/C++ go away when using the higher math more easily expressible in Lisp and like minded languages.
Jürgen Schmidhuber says the problem with computers from a mathematician's point of view are the numbers.
> Alan Kay says a lot of problems with C/C++ go away when using the higher math more easily expressible in Lisp and like minded languages
That doesn't match my experience at all. Math and numbers are in general not a problem except when (say) implementing triangle hit detection in a game (something I did) where the precision of 32 bit floats does matter.
> Jürgen Schmidhuber says the problem with computers from a mathematician's point of view are the numbers.
That is probably true if you are a mathematician. Most people aren't.
Having said that, LEAN (for example) is written in C++ and used heavily by mathematicians.
Go ate a big lunch from Java, C and a bit of C++ as a systems' language, which Go was designed as "C sucessor without the C++ bullshit". Even the most modern C (Plan9/9front, not what the ANSI C comitee vomits in every iteration) it's a very different beast.
Still, Common Lisp it's in places where trying to build such kind of software in C/C++ would be a reciper for disasters.
https://www.lispworks.com/success-stories/index.html
> Common Lisp it's in places where trying to build such kind of software in C/C++ would be a reciper for disasters.
I personally never blame the tools. I have written fairly complex software in 68000 assembler and it (1) worked fine and (2) was highly maintainable. However it was of course much slower to modify than using a higher level language.
M68k asm can be easier than some C++ templates.
I see what you did there.
"Of course that's your contention. You just got finished readin' some Marxian historian -- Pete Garrison probably. You're gonna be convinced of that 'til next month when you get to James Lemon, and then you're gonna be talkin' about how the economies of Virginia and Pennsylvania were entrepreneurial and capitalist way back in 1740. That's gonna last until next year -- you're gonna be in here regurgitating Gordon Wood, talkin' about, you know, the Pre-revolutionary utopia and the capital-forming effects of military mobilization." Good Will Hunting (1997)
In an awkward, error-prone, non-mathematically rigorous and easily subverted manner. That whole "make invalid states unrepresentable" thing relies upon the fact the semantic structures you're doing it with can't be shadowed silently by other parts of the code, and the safety constraint itself is guaranteed to be completely pure and deterministic. Lisp macros and reader macros fail at that criteria. They aren't any different from any other collection of procedures and their collective interface. Actually, they're a fair bit worse since the natural cognitive overhead from juggling multiple layers of evaluative indirection lends itself to blindspots of edge cases and unsoundness.
I think "macros are a safety mechanism" might qualify for the top 3 "new lisper mania" things I've ever read.
It can be made as mathematically rigorous as anything computable, and as non-awkward as is possible to represent. You either reify the invariants and semantics as statically analysed syntax or they are implicit in your collections of procedures and objects which is far more cognitive load. If you need to judge multiple levels of abstraction while using the DSL then the DSL is the wrong fit for the problem. There is no 'evaluative indirection' because a macro is a _compiler_ and the underlying code can be extremely alien to the DSL semantics.
Any argument against macros must be levelled against compilers in general, as _they are the same thing_. Rust is compiler as a safety mechanism, not different from macros as a safety mechanism.
Again, fallacy by Turing completeness.
It's actually harder to make something not Turing complete - in and of itself this property is absolutely useless and tells you nothing about how practical something is.
yes it's desirable to use non turing complete languages as less shit can go wrong, which is why you need macros. You seem to over estimate the difficulty of writing compiler macros, remember you only need to go from the DSL -> Lisp, not lower it into machine code. The DSL's compose perfectly as well, lower one to another or combination of them + lisp.
Well, compilers have layers - you ain't outputting assembly from the AST layer directly, usually there is some kind of IR. Like LLVM have plenty of layers.
And for a simple DSL you can usually just use a sufficiently expressive language, no need for macros. Like kotlin/Scala has html DSLs that look pretty decent.
For anything more complex though, you can't just "compose stuff", a type system is a global property so for that you literally have to write something way more complex than a local macro should contain.
>kotlin/Scala has html DSLs
these interpreted DSLs aren't at all what I'm talking about. The macro system should be able to compile arbitrary text into anything in the simplest possible way and have it seamless with the rest of the code + able to call into it and build off it.
A type system is paired with a language, not a global property. You can have language L1 with typesystem T1 compile into (L2,T2), etc. These typesystems can be domain specific and arbitrary as well.
After reading your posts I recommend you go ahead and learn common lisp macros, it will be more informative than more discussion.
> go ahead and learn common lisp macros
Lisp macros used to be the one advantage Lisp had over other programming languages. However nowadays macros are common. There are even languages that has more than one flavour of macro system (Haskell has both a typed and non-typed flavour of macros).
However I personally prefer custom code generators instead of macros. The problem with macros, for the kind of large scale systems I work on, is understanding macros with N layers of abstraction, and also the compile/run time cost of using them.
Also, you can write code generators in any language and generate code for any language. Which is a huge advantage. I (for example) use code generators that generate C++, Java, Typescript, SQL, PDF's, interface descriptions, protocol specs etc. Whatever is required by a customer or other members of the team I can generate without demanding that they use a specific programming language.
Don't get me wrong. I love playing around with macros. However I have decided not to use them for real work.
I use plenty of DSL's in my daily work maintaining 1 million+ lines of C++ code used by very large companies around the world. No macros needed or wanted. More than 90% of the production code is generated from DSL's.
Outside of C++, Haskell has many great examples of powerful DSL's implemented without macros.
The problem with macros is the compile/run time cost. I could have implemented those DSL's using C++ templates (which are Turing complete by the way) but I prefer not to.
The inflection of the copula "can be" is exactly the problem, as is the 'anything computable'. There's a reason Isabelle is used as a proof assistant, and Prolog isn't. The former guarantees sound, mathematical rigor that can't be trivially violated. The latter doesn't. Prolog is much more powerful and flexible than Isabelle. Power and flexibility are disastrous for safety. As someone else said elsewhere in this thread, safety is about restrictions.
In this case, it actually runs deeper than this, there are structural issues which prevent them from being sound right from the beginning. It's the mere fact that the whole behavior of your DSL can be silently changed without touching the module it's defined in, or the file it's being imported into. A well-meaning junior, in a completely different department, can completely destroy your invariants in a completely unrelated part of the codebase. Whether by shadowing runtime functions, or a package-level collision that exists upstream. Even Scheme's hygienic macros don't actually solve this, they just make it less likely. At the point in which you have no behavioral guarantees of semantics, you do not have a safety mechanism at all.
> There is no 'evaluative indirection' because a macro is a _compiler_ and the underlying code can be extremely alien to the DSL semantics.
This is a trivial contradiction. That is precisely evaluative indirection. Yes, stacking multiple compilers on top of each other, especially when their semantics are non-isomorphic, is a massive burden of mental overhead. Given you already have this problem with the mapping between Lisp and binaries, adding even more complicated layers on top of it is a recipe for disaster. Particularly because: you are going to make mistakes in your logic. Macros provide no means to stop you from blowing your own leg off when writing them. That is not what they are designed for.
Finally, you cannot escape the awkwardness of the underlying metaprogramming system, you will always have to wrestle with quotation and quasiquotation. You have to define a DSL before you can use it.
Only at the point in which you treat and use Lisp like an unserious toy, rather than the powerful tool that it is, can the fantasy of "macros are a safety mechanism" begin to make sense.
This is the "fallacy by Turing completeness".
Sure, and you may as well write full systems in Minecraft and create your own type system inside and whatnot.
Macros can make some small specific uses safe, indeed. But they are not comparable to languages with type systems where everything is type safe, unless you literally have written a new type system and compiler to begin with. Which is trivially true in every other language, C can be 100% safe with a much better type system, I just read a text block and compile it as rust code! Wow!
I have no horses in this race and I'm sorry if I misunderstand but I don't think he means to say Lisp is by nature superior to say Haskell. I take it as a statement on the generality inherent in its design. It is by its very nature very low on restrictions. Which is to say your freedom of expression is very nearly as complete as it'll ever be.
Sure you can beat Minecraft into Lisp and then Lisp into Haskell - as Turing made sure of - but you'll be battling a lot of dragons along the way. It's beautiful that you can beat Haskell into letting go of its type system by, say, creating a C compiler in it. It actually still amazes me the universe has this property.
In general I see "building" (I rather call it "conjuring") a system as going from the most general (all of your programming "language") to the specific (your solution). I can see how the most general of languages makes for a comfortable starting point of that downward journey.
I can also see how not starting from The Universe In All Its Infinity Glory (Lisp) but from The Planet Earth (say, Haskell) is helpful if you want to conjure something made for humans, but if you want to go beyond the mortal plane.. (I'm not saying that's a good idea by the way).
My issue is that a language is just as much about what it disallows, as much as about what it allows.
It's absolutely trivial to allow everything, you just have to make a Turing-complete extension either by design or accidentally.
It's much much harder to carefully choose more limiting primitives that only allow you to build stuff that keep certain important properties about them. You can't subtract stuff after the fact.
E.g. something like rust's borrow checker could not be realistically implemented as "a lisp" macro (what does lisp even mean in this case). For that to work you would literally build a rust compiler over s-expressions and you might as well leave off the s-expressions at that point.
And every part that was not compiled by that macro or whatever would be unsafe and could not interact with the "rust-lisp" compiled parts, as you can no longer assert anything about them (there is nothing mandating the usual single writer, xor multiple read-only references laws)
I agree. Restrictions are core to software development. In fact without them I don't think there is development in the first place. As you know I see software development as going from the general (the full bandwidth of the computational substrate available to you) to the specific (the absolute minimum - if any - computational structure you need to get what you're after).
Rust is more restrictive than Lisp. These restrictions are sane and productive for most general software development in 2026. They form an excellent base to start your work in this day and age. Starting with raw Lisp feels like starting with the general notion of mobility instead of just getting in a car. The car is not inferior at all. It's a set of restrictions on the most general notion of mobility that in fact buy you a lot of comfort like knowing it doesn't need to eat hay or take a poo in the street. The freedom they take away, they give back in strong, useful guarantees about stuff you actually care about ("getting places").
I can see how Rust, Haskell or whatever provide a set of comfortable, general-enough restrictions to make life for the common developer easier and more productive.
This being HN I do want to allow myself some contrarianism and point out that finding the most minimal, yet most general computational substrate is not at all trivial. It's in some ways equivalent to finding the most minimal set of invariants ("laws of physics") that can describe the most wide range of phenomena ("nature"). Which is to say I find it pretty damn impressive. So I only "disagree" with this:
> It's absolutely trivial to allow everything
I don't disagree because it is wrong, it's clearly not. Turing shows all languages collapse into one when it comes to computational power and I don't doubt that. But I do know there is a distance between the computational substrate, the primitives of the language, and the structures it expresses. Let me give an example.
A spreadsheet can represent a computer game. A game engine can represent a spreadsheet. But if you build Excel inside Unity you haven't made Unity spreadsheet-like at the substrate level; you have encoded spreadsheet semantics in Unity's primitives. Functionally they may be equivalent, and that equivalence is beautiful and Turing can tell you all about it, but the relationship between the native primitives and the concepts being expressed is different.
Turing makes building endless towers that are functionally equivalent possible, but what I mean is that there is an architectural difference between a system directly built out of the least semantically committed substrate available to it ("native") or "embedded" in simulations of substrates built on top of those primitives.
I think Lisp is close to the "least semantically committed substrate" you'll ever come across in practice. It's certainly not the only one, but it's a particularly clean and relatively practical one. One can say, for example, raw lambda calculus is equally uncommitted but it's kind of .. hairy. Lisp's primitives are basically the machinery of symbolic manipulation itself. It's bloody amazing.
Sorry for the verbiage. I went way overboard and meandered into foggy, mystical meadows full of mysterious entities so feel free to ignore.
Have a nice weekend!
Thank you for the reply, and I definitely agree with you on many points. I absolutely don't want to take away from Lisp that it is somehow uniquely elegant, and does have a mathematical beauty to it, this is certain.
I just feel this (and its practical ramifications) are often overexaggerated.
> I just feel this (and its practical ramifications) are often overexaggerated.
I dunno. Can't fully agree or disagree. Nominally, yes, you really don't need s-expressions and homoiconicity for creating reflective, self-hosting runtimes - live redefinition is possible in Erlang, Pharo, Ruby. Metaprogramming ergonomics - sure they are cheap in Lisps, but even Lispers try to avoid reaching there, Clojure specifically recommends thinking twice, although projects like Hyperfiddle prove macros absolutely can be very powerful. Syntax, mathematical beauty, yada-yada - that's all "poetry", much of the real world operates on tons of very ugly yet functioning code, right? So, really Lisp-shmisp, whatever, no?
In practice though, Lispers are enormously pragmatic - I'm not self-referencing here, I have worked with some. It's incredible how rapidly they can build things, prototyping on the fly. How quickly they can move between different runtimes - I've seen codebases sharing ideas between absolutely dissimilar platforms. It is inspiring how undogmatic they could be - they easily move between modes - data/code, FP/OOP, interactive/compiled, etc. They have good understanding of type systems and some even know good deal of theorem provers. For whatever reasons, Lispers are conspicuously, disproportionately effective, and this is true. Sure "citation needed" here, but this is my empirical, anecdotal observation working in different groups, using distinct language stacks for many years.
The causation probably runs through the programmer, not the program. It's not like Lisp unequivocally emits good engineers, maybe it's that a substrate with minimal syntax and maximal malleability trains a particular disposition - they treat everything as reshapeable material, distrust dogma because the language never enforced one, reach for the smallest thing that works because the language makes "the smallest thing" actually small? Maybe Lisp doesn't make anyone write better programs, it just makes it cheap to keep changing your mind? Perhaps cross-runtime fluency, undogmatism, and rapid prototyping are all just "cheap to change your mind" in a way?
Could be selection bias - maybe Lisp just attracts curious, theory-literate, undogmatic people? I don't know. What I observed is for whatever reason Lisp typically attracts older, more experienced engineers. And therefore all the "Lisp propaganda" comes from them, and demographically that's a small subset of overall community and maybe that why it often feels like overexaggerated rhetoric?
> unless you literally have written a new type system and compiler
This is entirely possible with plain lisp macros. See https://coalton-lang.github.io
Having that natively available as plain Common Lisp code is a very different thing from your "just read a text block and compile it as rust code!" concept. Or at least, the tool chain gymnastics required to make a C program emit its own source in another language... Why?
Well, I was talking about the fundamental part. Of course you can have more practical implementations, like Scala or rust's macro system, that gives you a proper typed AST.
Type safety is a domain specific concern, 99% of the time your domain will not have 'types', often it won't have notions of things at all. But any kind of type safety can be added at any point with macros. Something like rust would be a huge project but it doesn't get easier than macros, see Coalton, and they compose with one another. When you run into the same requirements in another language you are simply stuck source pre-processing. There is no stuck in lisp.
There are plenty of languages with macros though, nothing special about lisps.
And no, language semantics absolutely don't compose. Like you can't just do some kind of optimization in one place if you do some mutation on it in another place. Optimizations work on global assumptions that every part of the codebase have to abide by. Any part not doing it will make the whole thing crumble - and "there is no stuck in lisp" is exactly why you can't have your cake and eat it too.
[dead]
Don't miss out, we had new excellent editors and tools being released in the last months:
Mine: a complete, single-download application that comes with everything needed to experience the interactive and incremental development programming workflow, including hot-reloading and on-the-fly debugging. For CL and Coalton.
https://coalton-lang.github.io/20260424-mine/
OLIVE: a new hand-made plugin for VSCode.
ICL: a new REPL for the terminal and the browser with advanced features.
as a bonus: a JupyterLite kernel based on JSCL that runs 100% in the browser.
find them all: https://lispcookbook.github.io/cl-cookbook/editor-support.ht...
I can't find the source for `mine`. Do you have a link?
Looks like Mine is part of Coalton's repo. https://github.com/coalton-lang/coalton/blob/main/mine/READM...
The website seems to have a bug with syntax highlighting. Pieces of code included in the post text are black, you can still see the actual text if you select it with your mouse. Same bug on Chrome desktop and on Safari on iPad
Thanks for reporting. I fixed the error as soon as I read your comment.
This is the classic example of why you should always set the foreground color when changing the background color. Otherwise you end up with <code> blocks that are black-on-black. It probably works fine if your browser is in dark mode.
A workaround is to open one of the code elements in the browser's debugger and add color: white to the :not(pre) > code style.
MacOS Chrome? The code blocks look fine to me in linux {firefox, chromium, brave}. I think they're all using Qt.
Now, it looks fine to me too. I tested it in Chrome,Firefox, Safari on macOS and on Safari on an iPad yesterday when I wrote my comment and in all cases there were black lines all over the the place. The author corrected the problem, see https://news.ycombinator.com/item?id=48855382
Looks fine to me in Chrome on Mac.
I get this on both FF and Chrome.
My first thought was that the article was redacted lol
metoo
There are many articles extolling the virtues of Lisp. I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Articles like this, and the PG articles it references, amount to "if you know, you know". I understand the appeal and I understand the explicit and implicit arguments this article is making.
Computer programming has matured quite a bit in the past 60 years. I would like to see more articles that are more considered in their examination.
As a language Lisp is great (though the ecosystem is limited). It has two flaws-- it's very open-ended so unless you're talented and disciplined you can fall into a rabbit hole of hacking fun Lisp stuff and not actually getting any work done. Other languages have this problem, but Lisp I think more so.
The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.
Heh, i want to both disagree and agree.
I started with pascal and then C++. And then discovered Emacs and Lisp. Boy, that was a revelation!
Never ever in my life shipped a line of lisp code to production. My real life code was always Python or C++ or Java or C. All of my lisp was toys and emacs tweaks (50k loc in my config!).
But most of production code is gone from my life. I am a mid-level engineering manager now. I mostly write texts, or messages, or emails, or slides... and still use emacs and lisps for fun and profit and competitive programming.
Not a single production-grade lisp LOC in 20+ years. But, OTOH, i contributed to emacs, tinkered with compilers and interpreters and prog. lang. internals - all because Lisps made it interesting for me.
And, in a way, this brought me closer than ever to the job of my dreams: i work for a major player in static analysis space.
So yes, you are right and wrong at the same time.
> programming has matured quite a bit
Your wording sounds like it implies that Lisp "got stuck" somewhere in the past, no?
Clojure, Clojurescript, Clojure-Dart, Fennel, Jade, Jank, Jolt, Coalton - these are relatively recent (and still developing) languages, and this is just off the top of my head, there are so many more.
Lisp is not a programming language (in a sense), it's an idea. It influenced pretty much every single PL we use today and continues to do so. You can't really "level-heatedly" criticize an idea, it's like criticizing I dunno, group theory. You can though debate about merits of a specific implementation of it.
> Your wording sounds like it implies that Lisp "got stuck" somewhere in the past, no?
Pretty much. See pg's famous "Blub Paradox" where he sees Lisp as the top of a tower of lesser languages. He doesn't recognize that Lisp might at best be called a limit ordinal, to use math jargon. That is, Lisp is just another Blub, and the Lisp zealots haven't figured that out.
You don't necessarily want to keep going further and further up, of course. Lisp still has fascination. But e.g., in Common Lisp (I mean just the stuff in the CL spec, no ad-hoc extensions allowed) you can't write anything resembling an OS. You can in Scheme, using continuations to handle process switching.
Going further up, Lisp doesn't make it easy to ensure the absence of particular behaviours in a program, what TAPL calls the purpose of a type system. Tony Morrison has a semi-realistic example of what static types can get you:
http://blog.tmorris.net/posts/understanding-practical-api-de...
Link still works but TLS certificate expired in 2025, tsk tsk.
Your lisp criticism is orthogonal to the idea of lisp itself. Lisps like Coalton (lambda calculus) or Shen (sequent calculus) are strongly typed.
AI is an interesting point too where being closer to a raw AST is likely an advantage because it can focus more on the semantics instead of the syntax.
You can wrap parentheses around anything and call it Lisp, but I specified Common Lisp and pg's article clearly wasn't contemplating anything like Shen. Surface syntax is near irrelevant though.
I feel you're now contradicting your own statements now. You said: "[maybe] Lisp is just another Blub", but when pointed out that Lisp (and Lispers) is evolving, you're saying "pg clearly meant [the other Lisp]"...
Of course we're not talking about "parenthesis". Clojure even added square brackets for arguments and destructuring, it's not about the syntax, I agree on that.
Last I checked, both of these were built on top of Common Lisp/SBCL.
pg's notion was psychological, not about languages per se: a programmer sitting in language X can't perceive power above X, only below. It's about a fixed vantage point.
Your "Lisp is just another Blub" would be true if Lispers were stuck looking up from Lisp-as-it-was and failing to see higher. If Lispers were trapped in the Blub position, they couldn't have deliberately imported ideas that sit "above" classic Lisp. But they keep adapting: Clojure added persistent immutable data structures; brought CSP/channels; introduced structural contracts like Spec and Malli. With Coalton and Shen they are explicitly reaching for the static-types level you say they can't see. Racket's whole "language-oriented programming" perhaps a level above any perceived Blub.
Maybe what you see ain't a Lisp ceiling? Could it be that some powers must live in the substrate - and Lisp's distinctive traits are precisely what makes building that new substrate cheap?
I'm unfamiliar with Coalton but the Blub article describes the proverbial smug Lisp weenie. I'd agree with you that #notalllispers.
Most Lispers I met do actively write in multiple languages all the time. They tend to borrow ideas from other PLs instead of loathing everything else that's not their favorite. They prefer Lispy syntax, but won't reject a language to achieve a goal - they'd pick everything - runtime, tooling, etc. and try to find a Lispy syntax that sits atop. While preserving all the remaining semantics. And then they'd argue that syntax does not make a language. Perhaps, Lispers are the largest demographic of polyglot programmers in the global community.
What idea? S-expressions?
Because that's all these languages share. What's common in CL and Clojure? They are as dissimilar as C and Scala.
> What's common in CL and Clojure?
homoiconicity, macros, functional bias, REPL - to count just a few.
I would argue the functional bias when you have so much mutability.
Also, only the first one is remotely unique.
Well, Haskell/OCaml campers would probably say "it ain't FP, like at all, rofl..." Truly, it's absolutely pointless mental exercise to cherry pick features of any given PL and compare with another one, with no regard to overall experience (which often is very subjective).
Switching between different Lisp dialects is far less mentally taxing, even when they operate in completely dissimilar runtimes. I have seen days when I needed to jump between CL, Clojure, Fennel, Elisp and Janet and from all practical points it felt like almost using the same language everywhere. While switching even between JS and TS is enormously vexing for me in comparison. Although I have programmed in both for far longer than any Lisp.
> I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Standardized Concurrency is basic table-stakes for a language today. CL does not have a standardized async/await or concurrency model. The standard hasn't been updated since 1995 so it will never happen.
And it remains in a state of almost getting to the point of generic collections (like C++, Clojure, many others) using the standard functions, but not quite getting there. There are functions (not generic functions) which operate on sequences, but no standard way to extend what types are considered sequences (as one example). It makes sense that the 1995 version of the spec would be incremental, but without a further development (or consensus amongst the implementations if no official standard) it's difficult to continue moving forward to make better use of the language's generic function capabilities.
That's where Clojure, Julia, and many others have advanced beyond what CL (by the standard) provides. The language needed at least one or two more standards to drive those features forward, but never got them.
I don't quite understand what you mean by generic collections in this case. Do you want to restrict a list to only contain one specific type?
I wrote that quickly and should have edited it for clarity. What I meant was a new collection type that could be treated as a sequence.
https://www.lispworks.com/documentation/HyperSpec/Body/17_a....
None of the functions operating on sequences are generic, so you cannot create your own type and do a `defmethod` and have them automatically adapt to it. And unlike `print-object` which gets used by format and print and others, there's also no "protocol" (or whatever term you like) generic function you can implement which will make your collection acceptable as a sequence.
You could build something like this, and maybe shadow the standard functions so that it's not a new set of functions from a user perspective, but it's a notable absence in a language with a rich OO system through CLOS. It makes sense that it wasn't in the original spec, but I still think it would have been something that would have been added if the development of CL (as a standard) had continued.
Oh, I understand. For what it's worth, some implementations do have that as an extension, although I've personally never used it. See http://www.sbcl.org/manual/#Extensible-Sequences.
> The standard hasn't been updated since 1995
The latest finalised submission to the Common Lisp Document Repository (confusingly called CDR instead of CLDR) dates from August 4, 2013. CDRs are the equivalent to the SRFIs from the Scheme world. You could argue that they are not THE standard, but they are A standard. And considering the original standardization supposedly cost 400,000 USD (about 900,000 USD in today's money), I can see why there hasn't been another official ANSI standard.
Since my job involves a lot of Unicode stuff, I personally would have been confused if it had been called the CLDR, thinking that it was referring the Common Locale Data Repository. (I have to look something up in the CLDR about once a month on average).
And since the Common Lisp Document Repository appears to have been created in 2006 (the first document at https://cdr.common-lisp.dev/index_files/final.html is a description of the rationale for its existence, dated August 2006) while the Common Locale Data Repository was created in 2003, the Common Locale Data Repository had the CLDR acronym first. So kudos to the Common Lisp Document Repository folks for not overloading the acronym but finding an alternative.
I see. Wasn't aware of this. However, it does not appear that an official standard for concurrency exists within the Common Lisp Document Repository (CDR). To be honest, CL implementers should just solicit donations for producing a standard.
There are also other basic things needed for adoption - standard APIs for database providers, etc
https://github.com/CodyReichert/awesome-cl lists some concurrency packages, such as https://github.com/sionescu/bordeaux-threads and https://github.com/sharplispers/lparallel, as being "so widespread and solid that they became community standards. You can't be wrong with them." I don't know enough to know whether or not they're right about those being widespread, but if that list is wrong then I'm sure someone will chime in to say so.
So while the official standard may not have concurrency, that does not mean that there isn't a de facto community standard way to do it.
CL may not but other lisps do. Clojure’s concurrency is just so hawt
The lisp curse: https://www.winestockwebdesign.com/Essays/Lisp_Curse.html
[dead]
Here you go: https://www.marktarver.com/bipolar.html
It's been discussed many times on HN.
and strangely relevant again in the age of LLMs - https://news.ycombinator.com/item?id=48126840
There seems to be a Lisp .. handedness?
Certain people seem to find it mentally appealing. I would liken it to one of the really niche music genres. Only a small number of people like it, but they do so very intensely. The Stockhausen of programming.
That's basically all it is. A fandom. A remarkably enduring one, but not one which has ever broken out, and therefore is unlikely to ever do so. Now in an even worse position: if you are armed with an LLM and therefore uninterested in the code itself, why on earth would you direct it to write in Lisp?
https://www.tamaspapp.eu/post/common-lisp-to-julia/
I mentioned this already but you might like: https://www.cs.kent.ac.uk/people/staff/dat/miranda/wadler87....
I've been programming Lisp too long to appreciate it's features. It is not until I watch in shock and horror how people sabotage themselves in the popular and economically less risky languages that I suddenly come to appreciate Lisp again.
There are some truly powerful and unique things about Lisps, but I wish articles like this would stop including REPLs and hot-reloading. The former have been table stakes for interpreted languages (and some compiled ones!) for years, and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
> The former have been table stakes for interpreted languages
I used to think so too back in the day when I was getting into Clojure. It was much later when I realized that when Lisp people talk about the REPL they’re usually talking not so much about the interactive CLI where you can evaluate commands easily but more so the ability to connect your program to a live session where you can quickly evaluate forms within your text editor and in the context of your running application, which enables much more interactive development than in other interpreted languages.
you can do that in a python debugger?
the python debugger is cool but is an inferior experience. It isn't only about a REPL, but about the editing experience, and about the language being thought from the ground up for long-living programs. See: https://mikelevins.github.io/posts/2020-12-18-repl-driven/
[dead]
You've been able to do this in Ruby since I can remember. Not a lot of editors take advantage of it though...
No. There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages - Ruby, Python, Haskell, etc. Every single stage there in Read-Eval-Print-Loop differs.
Man, it gets so exhausting trying to convince every PL critic who grabs a single (or a couple) of axis of any language and tries to dispute the value of a language without ever understanding the holistic, overall experience working with it.
Like, I don't understand, do people think that tons of Clojurians or Common Lispers who fall in love with the language after decades of working and getting seasoned in literally dozens of different PLs are on some kind of delusional trip or something?
Guys, just take a gander at Clojurians Slack; see what people are working on, what kind of stuff they're building; check their profiles. Many of them are the battle-scarred veterans of coding. Sure, some of them may have wrong opinions, but surely they can't be all wrong, can't they?
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
Smalltalk is not homoiconic, and it's REPL experience is equivalent (I'd argue somewhat better, but that's mostly a tooling thing, see the commercial CL implementations as examples of improvements over SBCL + Emacs + Slime). Homoiconicity is not the trait that makes the CL REPL experience better than others, it's that it includes a very good debugger, the compiler, hot code reloading, the ability to redefine classes and update current instances, and so on.
That's the tooling, not the language, that provides the experience. Nothing about being non-homoiconic prevents other languages from having a comparable (or even better) experience.
> Nothing about being non-homoiconic
Nothing about being statically/dynamically typed. Nothing about being functional/OOP/relational/logic. Nothing about being pure/side-effecty/strict/lazy. Nothing about being compiled/interpreted. Nothing about imperative/procedural/stack-oriented.
You can pick just about any single or (few) aspects about any language. Heck, it doesn't even have to be a programming language and you can find things to complain about.
You know that there are three genuine, true, legit ways to build robust, bug-free, performant software? Three! The problem? Nobody knows what they are and that's why we are all doomed to keep bashing on everyone else's choices and opinions.
"Better experience" is not dictated by "features". Better experience comes, well, with experience.
I suppose it's my own fault. I tried focusing on "holistic, overall experience", yet still picked a single aspect to chime in.
[flagged]
Huh? First time I hear that my account is "regarded" and not just, but "widely" too. What part of your fragile ego my words hurt, I wonder?
I wrote GGP as a Lisp user and enjoyer—not pretending to deep experience, but definitely am well acquainted. And neither the REPL nor live reloading are important differentiators. Neither is the ability to attach a REPL to a running program: plenty of languages have that built in (Erlang, Ruby), and many others support it through popular third party tooling (e.g. Pyrasite for Python, and—ironically—the class executor for the Clojure REPL for arbitrary non-Clojure JVM programs). Many Lisps’ REPL tooling is very nice, and the language lends itself well to REPL-oriented development. But those aren’t “uniquely Lisp” features as you and many others claim.
Homoiconicity is a "uniquely Lisp" feature and it doesn't seem like you've fully grokked the implications and differences between homoiconic and non-homoiconic. I'm not making this up - every step there in Read-Eval-Print-Loop does differ. That is easily verifiable info.
Eval differs very slightly from, say, an interpreter with a very non-syntactic AST, due as you say to homoiconicity. Neither read, print, nor loop differ in any Lisp-specific or qualitative way. And the differences in eval’s behavior don’t change anything significant about the REPL user’s experience (other than that they’re writing Lisp). So what do you mean?
Specifically, cached state and behaviors still need to be reconciled with new inputs/overrides during “eval”, and no Lisp has an easy answer to those (neither, as far as I know, does any other language). Avoidance of closure state via late binding and a convention of reliance on very simple data structures help Lisp’s REPL/hot-patch story, and that is indeed nice, but it’s far from “guaranteed ease/safety of REPL patching” or a totally different paradigm.
> don’t change anything significant about the REPL user’s experience
Perhaps you have never tried/heard of nextjournal/clerk, scicloj/clay, djblue/portal, vlaaad/reveal or just simply tried building a simple web scraper with Playwright running on nbb. Oh, and hyperfiddle/electric - something like that would not be very trivial without homoiconicity.
Lol equivalents to all those exist in at least half a dozen non-Lisp languages.
Lol, how are you so smart and figured just everything out, so already? I'd be so bored to ever touch the keyboard again. Bless Turing I'm so stupid and simple and keep trying different things just to see how they differ.
I wish someone would just write a clear description of that magic version of REPL that isn't a REPL. Maybe also a demo. And how to get the actual pristine source out once you've done development on a running image. Maybe we could be enlightened. The complaining clearly isn't doing it. Articles like this one didn't succeed either. Pointing at Slack surely isn't going to do it.
Tell me your knowledge of Ruby is surface level without telling me...
I used Clojure when it first came out, I've used Common Lisp for years, I've also used languages like Smalltalk and many others.
Yes, Lisp environments are nice, but sometimes I think Lispers are so insular they don't realize that other languages have similar things. R, Julia and Ruby have similar environments. Smalltalk is next level.
And homoiconicity is great for macros and parsing but it's late-binding that enables the live programming behaviour, which isn't exclusive to Lisp.
Also if you'd ever gotten deep into say, SB-ALIEN, you'd know the limitations too; Common Lisp isn't magic, it can't just redefine say, instantiated structs in memory. It relies on pointers then switching references on the fly.
[flagged]
> they don't realize that other languages have similar things
Yes they do. God, how did you (and apparently you're not alone) read my rant and still got it 100% backwards? I specifically hinted about not picking a single aspect of a language - REPL or whatever. What is so confusing about my wording on "holistic, overall experience working with it"?
Apart from your rant, this is the most pertinent part of what you said:
> There is real, palpable, practical, functional difference between working a Lisp REPL and a REPL in a non-homoiconic languages
And it's false. There's nothing special about a REPL in a homoiconic language. Homoiconicity has nothing to do with a REPL or tools.
I do however think SBCL has a very nice environment with great tools (especially it's compiler) and that's why I've used it a lot over the years, but I really think you underestimate the environments in other languages (including the "holistic, overall experience").
> And it's false. There's nothing special about a REPL
You are wrong. You might be correct about the workflow but still wrong about the semantics. Pedantically, your notion is false, because homoiconism gives the REPL a genuine capability (program-as-manipulable-data across the read/eval boundary) that non-homoiconic REPLs lack.
Can you parse code into an in-memory representation and can you eval?
If yes, then you can do everything that Lisp repls can.
The parser being slightly more complex than parsing s-expressions doesn't materially change anything.
I did not say anything about s-expressions. I said "homoiconicity". S-expressions are not an absolute requirement for homoiconic features. You keep poking me with your "knowledge" as if I'm trying to sell you snake oil and you feel obligated to defend and retaliate. I don't need to prove anything to you, I'm not your ignorant professor attempting to instil you with some defunct truths that exist solely in my head. Do your own research. Or don't, who cares.
Why do you think other people who have fallen in love with, say, Ruby, are less experienced in other languages? They just have less of a cult.
You sweet, summer child, just so you know, I have existed for nearly half a century in this world where the larger part of it I have spent dealing in computing. I have seen and dealt with more programming languages (including Ruby) than you can count with your fingers and toes and that number keeps growing still. I'm not advocating for any particular language, runtime, framework or paradigm - do use whatever your heart desires.
I'm simply pointing out that there is a meaningful difference in REPLs in homoiconic and non-homoiconic PLs. You don't have to listen to me, this is easily verifiable information. Google it, ask LLMs, try it yourself, or ignore the notion entirely - it's completely up to you. But let's not get too intimate and slide into insulting one another - you have no idea who I am and what cults I'm specifically fond of.
Apparently you're not experienced enough to have used a Lisp that doesn't have a REPL, nor a non-Lisp that does have a good one.
You obviously haven't implemented a Lisp either, because then you'd understand what homoiconicity actually gives you. And also how a REPL is implemented (since you wouldn't just get one for free, it's a tool to implement).
You're wrong on every count about my experience. And once again, please: I'm just saying that stages in Read-Eval-Print-Loop does have differences in homoiconic and non-homoiconic languages. THAT'S ALL I SAID. There's zero controversial taste in that statement. None. I'm not bashing on Ruby, or Python, or any language you favor. I'm not telling you to use this or that. I'm just pointing out at specific differences that exist.
So if you implement a non-homoiconic language in SBCL or Racket does the REPL blow up? Is Rhombus language a psy-op?
Implementing C-like language in Lisp runtime - your interpreter reads C-like source, parses it to your own AST structs, and evaluates those. The host being homoiconic says nothing about the guest. You would have to write a guest REPL anyway (if you want one).
Rhombus built on Racket, yes, means it reuses Racket's machinery: the macro expander, the module system, the compiler, the runtime. But it doesn't mean it inherits Racket's parenthesized syntax.
Racket's expander does not operate on text. It operates on syntax objects (shrubbery). It has its own REPL, because the Read stage has to parse shrubbery syntax, not s-expressions. So, my original point stands - REPLs do have differences for homo and and non-homoiconic PLs.
So close yet so far lol...
Please be less of an asshole.
You’re an asshole too for talking like that.
I am not an asshole for pointing it out though, I’m likable.
iLemming: creating a new user just to say that seems a bit over the top.
I don't know who that is. I don't play stupid social games to win stupid social prizes. Why would I create a new account and yet keep replying with another one? I have single accounts pretty much everywhere, except Reddit - they shadowbanned my first one when I posted while connected to a VPN (apparently that's not allowed). You can easily use some OSINT tools to confirm that.
Again though: please don't be an asshole. You're doing a disservice to Lisp.
Da fuk you're talking about? Where the heck am I being an asshole? I have not used a foul language or insult in this thread once, even when someone accused me of being in a cult. Don't patronize me, even if you bought a moral authority on a discount - I'm not your fucking child. No matter how many times you say "please" and call me "asshole" I won't correct anything in my behavior because there's nothing to correct to begin with. What "disservice"? a) I could't care less about Lisp's popularity b) You seriously think someone, anyone would be like "don't look at Lisp... because there's one jerk in this specific HN thread. He said something in 2026. Eight levels nested down..."
High testo, huh.
I would, but apparently I'm in a cult. Truth be told - I'm in multiple ones. Still trying to decide which one is my favorite.
> and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
For what it's worth, hot reload is very widely used in Dart/Flutter. When you are writing UI code using a reactive style framework where rendering the UI appears to be "generate a new UI from scratch on each frame", it's more straightforward to have an intuition about what does and doesn't get reloaded.
It's not perfect, of course. But it works really well for the kind of changes you make when iterating on a user experience.
Largely agreed.
I do think it is worth highlighting how many advanced parts of hot-reloading have already been covered in Common Lisp. Same with highlighting how the REPL is largely not used to directly type into, but is instead a very powerful interface for tools to interact with a running image.
But, again agreed that simply these existing are not that notable today.
Not all Lisp dialects have well-developed hot patching systems. The OOP system has to be carefully designed for it. What if a class definition is superseded by a reload, but there are existing instances? The Common Lisp dialect of Lisp has useful answers to questions like this, but not necessarily every Lisp you come across.
Even Bash supports hot-reloading. I've developed modules that are updated in place by sourcing:
Generally speaking, the pain of handling hot patching depends highly on the structure of your codebase and the reason for hot patching to begin with. For a structure-of-arrays architecture, or for fixing logic errors at run-time, it's really no big deal at all.
It is limited utility, but if you start out from the mindset it's something you'll use a lot, those limitations start disappearing rather quickly.
But with Clojure and immutable by default, hot reload is a real thing, it sounds like not just on JVM but even among lisps.
Clojure barely let you import packages without restarting the JVM recently in its life.
Can you expand on that? I am mildly experienced with Clojure, and much more so with the JVM and I see no reason for that to happen (besides perhaps some kind of module shenanigans or importing stuff that requires special flags?)
But it has this capability and had it for a while. What's your complain?
Ehh, no. In fact, other Lisps are actually slightly better at hot reloading because they're not hampered by the JVM's limits.
Check out stuff like CHANGE-CLASS or whole image loading.
> After getting comfortable reading code with so many parentheses
I never managed to get over the ().
Ruby has a very flexible syntax, compared to many other languages, in that you can omit syntax in many cases. For instance, using () for method calls is largely, for the most part, optional. So when I have the python code:
I find it worse than the ruby code: (Though you can use () in ruby there too; but most people won't do so as there is little point in that.)This is a superficial issue though. Python's biggest mistake is to require implicit self. It always feels as if I need to hand-hold python and trying to explain to it what an object is and what self is. In ruby I don't have that issue. Note that I find both languages fine, but ruby is "more" object oriented than python is, for many reasons.
Lisp is quite different though. I had some exposure to it via scheme and while it can be fun, I feel that lisp is a worse programming language than either ruby or python. The old game haxima/nazghul was given up eventually, primarily due to lack of time, but also because the author was no longer convinced of scheme (the core of the engine is in C if I recall correctly). He tried to switch to python, though, which did not work (but, again, I think it was mostly due to lack of time). The scheme code in haxima/nazghul was quite interesting (https://sourceforge.net/projects/nazghul/ if you want to look at the scheme code there), but I much prefer either ruby or python there. Although it would be interesting to have DSLs that would really focus on the game or project at hand, like in the old Zak McKracken game.
> the ruby code: ``` cat = Cat.new cat.meow ```
As a non-Ruby user, this confuses me. Why can't `Cat.new` be a function reference? Does Ruby explicitly disallow this (i.e. passing around a function)?
You can do this, but there's no syntax. Instead, call the `method` method with `:new` as its argument.
In commercial practice DSLs are an anti-pattern. Someone will create an under-documented DSL that only they understand and move on. The following programmers have to try to decipher this strange language. It is almost always a better idea to use standard language structures and features. Code is read more than it is written.
In Lisp, those following programmers can simply macroexpand. With modern tools they can even do it inside their editor, co-located as replacement text in the same source file, connected to a live Lisp environment, if they wish.
A great example is Common Lisp's own LOOP macro - if someone's usage is difficult to understand, you simply ask Lisp to expand it into the more verbose non-LOOP fundamental calls, and you don't need to understand anything about LOOP. In fact you can replace the LOOP form with the expanded code.
That's a key difference from DSL's in most other languages where the DSL code is really data structures that are interpreted.
Hate to break it to you, but every time you create a function, you're creating your own language, specific to the domain.
I suspect what you're actually objecting to is a lack of referential transparency, which is a problem in many languages (including Lisp, to be fair).
I’m not referring to trivial functions nor classes. I talking about things like overloading operators, or changing the default idioms of the language. This means that a person can’t read the code without fully understanding the entire class structure. For the most part DSLs are highly personal. What one person thinks is a great formalism that make it easy to reason about the problem, to everyone else is spaghetti code.
I thought this was going to be https://web.archive.org/web/20120106121645/http://wiki.alu.o.... Brings back memories of following comp.lang.lisp, not least as a sort of soap opera. Lots of characters and drama, as well as programming wizardry.
This is well-written.
I always thought that the "answer" to programming would be that one day everyone would use Lisp and with awesome tooling and libraries things would be wonderful. In fact, my plan for retirement was to build high-quality libraries for a Lisp language to accelerate this process.
Does the rise of AI bring an end to this dream? Is that, once again, we have solved the problem by adding more cruft? Rather than a superintelligent AI writing in the best programming language available, we're going to just spam lots of Python code until it works?
Does this matter? I don't know, I just wanted a world of elegance.
I've always wondered what would have happened if the first "Interface Builder"
>Jean-Marie Hullot created "SOS Interface" in Lisp for the Macintosh while working at INRIA (1984) which was the first modern "interface builder."
https://denninginstitute.com/itcore/userinterface/GUIHistory...
had become a mainstream Mac product rather than being co-opted for NeXT and use w/ Objective-C.
LISP clicked with me (when taking a comparative language class in college I was the only one who managed to do all of the LISP homework) and learning it was a lot of help in using TeX.
I just wish that there was:
- a nice native or cross-platform GUI toolkit for it which was opensource
- an easy way to distribute projects as stand-alone compiled code
GUI: MCClim, but it's a bit rusty/Motif like/Tk Life.
Deploy: SBCL can build standalone binaries I think.
If SBCL can build stand-alone binaries using MCClim, I know what my next personal project will be.
Get Quicklisp and after that, Ultralisp because the last CLX library it's much faster even on shitty computers las my n270 netbook. Under ECL it almost ran without lag (just a slight one), and with SBCL it was almost as fast as Tk under TCL.
Languages are never the answer. Lisp isn't superior to Visual Basic. They are both Turing complete. Humans can express good ideas in each. It's better to think of languages as tools. An impact drill is not superior to a simple rotary drill. You will have better luck building a deck with an impact drill, but both will do the job.
Personally, I grew beyond this. I tried Common Lisp, Forth and Haskell. I enjoyed books On Lisp and Let Over Lambda.
Now I think the best programming language is Barry Jay's Triage Calculus, which is close to combinatory logic or untyped lambda calculus.
But unlike lambda calculus (which is easily expressed in TC), triage calculus has a built-in quoting and introspection (similar to Lisp's CAR and CDR), which lets you easily add typechecking. So in TC, you can have any syntax you want, just by building the correct abstraction. It's truly an ultimate programming language.
TC shows that there is a false dichotomy between programming language features and syntax on one hand, and the function/API definitions in the standard library or user code on the other hand. Every time you write a piece of code intended to be reusable, you're adding to the language. It can all be expressed as some term in TC.
All the syntactic squibbles are a matter of habit. Although one syntactic feature you really want in TC is the let over lambda abstraction, because it's somewhat annoying to order combinators by hand.
Do you mean Tree Calculus? https://treecalcul.us
Yes, although I wish he would call it properly triage calculus (see https://treecalcul.us/specification/), because although the languages are almost identical in power, the original tree calculus and triage calculus rules lead to different programs.
I must admit - I still don't understand macros. I get that they're code that's generated at compile time. But I don't understand how that's different than a function which evaluates other functions. I guess the latter would actually be evaluated at runtime? I think I get it conceptually but I'm not sure I have the muscle memory to reach for them. Anybody here have an "ah hah!" Moment with macros?
> But I don't understand how that's different than a function which evaluates other functions.
You want a function that takes in arguments, but does not evaluate the arguments when called. So:
Normally, foobar() will be called twice - at the time of the call. With macro expansion, you can ensure that it's not the result of calling foobar that goes into the func, but this expression.A canonical example is if you want to write an if/then/else function:
It's quite possible that the else_path is invalid and will terminate the program if the condition is true. But if you wrote a function this way, it will evaluate both then_path and else_path - not something you want to do in a regular if expression!(I think you meant “if condition is false”)
Technically you can accomplish the same thing in languages with first class functions if the caller wraps their code in a lambda.
No, I meant "if condition is true".
The point is that in most/all languages, if the condition is true, the else branch is not evaluated. And it's usually OK to put code in there that can crash when the condition is true, because we know it won't be evaluated.
But if you make an if function like I did above in, say, Python, both the then_path and the else_path are evaluated before the decision is made.
> Technically you can accomplish the same thing in languages with first class functions if the caller wraps their code in a lambda.
True. For languages that have lambdas (which I guess is most of them nowadays).
Well, as mentioned that's why you create two lambdas as the branches. This is pretty much what thunks are in some languages (like Haskell), so this example doesn't strictly require macros per se.
This is true, but thunks carry overhead that macros don't in this case.
Well, if we have a compiler that sees a constant value, it can completely optimize out one branch. But it of course depends on the semantics of the language.
But yeah you are right in case of a naive compiler.
And I know people will think I'm crazy, but thunks are harder to read.
(And I have trouble reading macros).
Ok yeah looking at it again I understand now. Still learning to read.
Sure, macros are functions that take functions as input, and produce new functions as output. But they take the function's symbols as input and produce a new set of symbols. So a macro can extend the syntax of the language without having to modify the core language system. Anyway, what's unique about Lisp macros vs say, Rust macros, or C style preprocessors, is "homoiconicity". The data structure that a Lisp macro takes as an input, the lisp code, is the same data structure that the language uses normally (S-expressions, lists...), so writing a macro requires few new language skills compared to writing normal lisp (again, compare writing Rust macros, a dark art in comparison).
At the same time, you get a mushy tree as input, instead of a properly typed AST "object" with accessible "function name", "method parameters" everything that you can just refer to.
It will be the third arguments' 2nd arg with s-exprs, following your arbitrary pattern you figured you want to use. So it's very arguable which is easier to use, sure for some "party tricks" lisps will win, for anything more useful my vote would go with rust/Scala macros for sure. So again, you win nothing by homoiconicity, the by-the-compiler implemented parse function is more complex in rust, which is 100% abstracted away.
If you haven't read it, I'd suggest taking a look at Paul Graham's book On Lisp [0]. He says better, and with more examples than I'd provide in a comment block, what I'd write on the subject. Jump to chapter 8 for his discussion on the topic, referring back to chapter 7 if you find the macro definitions difficult to read.
[0] https://www.paulgraham.com/onlisptext.html
I think the best explanation, how Lisp macros are different from most other macro system is, that a Lisp macro is a function executed during compilation. It gets fed the parameters passed to it in literal form and then computes the source to actually compile by the compiler.
The power of this is that this function is also written in plain Lisp and it also sees all definition already existing in your Lisp image. Also, it is completely unlimited in power. So while being run, or as it is called "during macro expansion" the function could google for one of the things passed and use the result to create the output. Of course, this is not a practical example, but shows how flexible the system is.
There's a bit of a mental model flip to make maybe.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
The arguments presented to a macro don't have to be valid code. Your "function evaluating functions" are all individually valid functions you've composed at runtime.
=> actual, compiled Clojure function built up by `foo` parsing the string, producing a Clojure list, and calling `(eval the-list)` and returning it.I passed a string to `foo` in the example, but it could actually be anything the Clojure reader can parse even if it is semantically invalid—arbitrary Clojure data.
function can do what a macro does, but it must run every single time while the macro can run just one time during the final code generation.
Let's say you have some Java-style a + b. It needs to work on native strings (concatenation) along with various kinds of ints and floats. The issue is that each one of these works differently internally, so the system is going to look at the incoming symbols and metasymbols (inferred type data) and is going to dynamically change between different functions like ADD_STR, ADD_INT8, ADD_INT32, ADD_FLOAT32, etc. If it did this at runtime by introspecting the type with some `switch(incomingType)` statement which involves all kinds of extra data and branches that clog the cache and create branches (two of the worst things you can do for performance). Instead, the macro (though they probably don't call it that) looks at everything and hardwires the correct output.
This hardwiring and its associated performance is the difference between a function and a macro. A macro `foo!(a b)` can inline into the current function while the function alternative must save all the registers to the stack and create a new stack frame which is very expensive.
The beauty of lisp macros is how functions and macros look the same. You may think that `(+ a b)` is the same as your favorite languages' `a + b`, but most lisps also allow `(+ a b c)` too. `+` is a macro rather than a function and it knows that `(+ a b c)` needs to convert to something more like `(+ a (+ b c))` and a typed lisp may go further to something like `(+f64 a (+f64 b c))`.
Open notepad and solve your problem in some language you make up on the fly, with macros that's now valid lisp code. A macro is a compiler and nothing more.
You could use them to:
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
If it helps, https://taeric.github.io/CodeAsData.html was my attempt at exploring "code as data" and what makes lisp different here. My specific focus was more to point out that "eval" in lisp doesn't just take in a string. But I think the same general points remain.
Many problems can be solved with either macros or higher-order-functions. The advantage of macros is two-fold (macros also have disadvantages, and Lisp allows you to use either solution; deciding which to use is often a matter of taste):
1. Syntax can be more familiar
2. Performance (to the extent that there is overhead for functions-calling-functions).
For example, let's consider a hypothetical lispy language that doesn't have a short-circuiting "and" operator. Macros would let you implement something to used like this:
Higher order functions would require you to do something more like: More noise, and more work for the optimizer (or in the worst-case, more work at run-time). Languages that rely heavily on higher-order-functions will tend to have terser syntax for anonymous functions. For example in javascript you might do:Macros are conceptually similar to FEXPRs in that they act like functions that don't evaluate their operands and return code. The (only?) difference is that macro are all expanded before execution/runtime.
one way to see it is that it's a function that runs at compile time. for instance instead of dumping magic numbers/tables in a codebase you could put the code and substitute to their value at compile time.
but also it can change your code, so you get to do all the java annotation magic stuff.
[dead]
You do not need macros for anything. They are not the tool to "extend the language", but instructions for the compiler. And if the system does not even have a compiler, macros are useless, often incredibly stupid.
If you have an interpreter only, no compiler, using macros for metaprogramming anwyay at least prepares you for the eventuality that one day there will be a compiler. The macros will Just Work as before, only the expanded code is now processed by compiling.
Suppose you reject the idea that there will ever be a compiler. Macros are still useful for doing "compiler-like things" in the context of interpretation, like transforming code into something that will interpret better.
We can regard the interpreter as a virtual machine acting on a representation of the code; the macro system lets you manipulate the representation in a pre-computed pass, which has no further cost at run time.
If you have a code expansion pass on top of an interpreter, for supporting macros, you can use that as an excuse to perform built-in code transformations that are not macros; those enable you to have more flexibility in how special forms are implemented.
In TXR Lisp case is a macro (family) which performs certain optimizations like recognizing values in a range and emitting a table switch. This works interpreted or compiled:
It is faster to do some checks and interpret the sys:switch special form to dispatch by a numeric index than to do a large number of exhaustive comparisons.Ultimately, the way we optimize interpretation is by compiling, but it can still be worth it to have better interpretation here and there.
You don't want to do this kind of optimization at run-time; you don't want the interpreter to be evaluating the condition "are these cases integers (or characters) in a tight range?". That's a property of the syntax in which the cases are constants; it wants to be pre-computed once.
TLDR, but you are (probably) describing somekind of builtin property.
I am just saying that the end-user do not need macros for much anything, unless she has some specific optimization in mind. Bloody annoying when they start making cryptic macros to "extend the language".
"Need" is a very tenuous word, because most of any tool stack consists of benefits.
End users benefit from things that they don't strictly need.
Once you start talking about what people don't need, it's hard to do it in a way such that they need Lisp but don't need macros.
Sometimes "need" is about dependencies; i.e. things you "need" are provided by upstreams; you stop needing things that you made yourself.
E.g. "I don't need objects. Because, well, I have lambdas and macros, and with that control over syntax and semantics, I made my own objects. So, strictly speaking, I do need objects, just don't need them from you or anyone else."
People disagree over what constitutes a benefit; something someone finds beneficial in their work is crap to someone else, and the easiest way to disparage someone else's beneficial thing is resort to the necessity rhetoric.
"Everything that I found beneficial in conducting my successful XYZ project, which got to a level of complexity and quality in so many years of effort and lines of code, was obviously necessary; all else is unnecessary."
A really sneaky way to do that is to repeatedly acknowledge the benefit of something while emphasizing the lack of necessity. With a sprinkle of FUD about possible harms that outweigh benefits ...
I really do not understand this whining and downvoting.
In so-called real life somebody has to live with the malformed macro, whose only purpose is to beautify the code.
I even made un-macrofier, which deleted all macros and replaced instances with macroexpand, because slight adjustement of code resulted mysterious errors cause by nasty macroes.
And lets not talk about Common Lisp loop macro, which was probably scooped up from latrine of hell by the Devil himself.
Nobody has to live with a malformed macro because they can just observe what code expansion it is performing and write their own which does it better. (Speaking of CL loop, some people have resorted to this solution of actually dragging an implementation of loop into their application, to avoid issues with a vendor's loop, or behavioral differences among different ones---yes, loop has this, unfortunately).
The malformed macro situation is a lot better than the situation of a malformed API function to which you don't have source code.
Anything that is formed can be malformed. DNA can be malformed, leading to malformed fingers someone actually has to live with; so let's not have fingers.
My question always remains. Which variety/implementation of Lisp is it worth it to learn as a hobbyist and which as a professional?
As a professional - assuming you mean “which can I put on my resume to get a job working in that language” - probably (not necessarily!) none, or maybe Clojure.
As a hobbyist, probably all of them. It’s worth at least trying to go through SICP in Scheme (use Racket with the SICP language), it’s worth learning CL (use SBCL) to fully appreciate everything the Lisp world has to offer without compromises, it’s worth learning Clojure because it’s, frankly, a little bit cleaner and more elegant than CL in many respects.
Next week I will try to answer your question with a new post. Stay tuned!
All roads lead to Lisp
What has Lisp ever done for us?
Flight search (Google's ITA Software), underground planning (SISCOG), CAD software (PTC Creo et all), the Gollum face (Mirai), automation of document extraction, renewable energies resource planning (3E), project management (planisware),
games (Kandria),
Maxima,
a faster pgloader (https://tapoueh.org/blog/2014/05/why-is-pgloader-so-much-fas...),
hackernews (was rewritten to SBCL: https://lisp-journey.gitlab.io/blog/hacker-news-now-runs-on-...), first reddit…
https://lisp-screenshots.org/
https://www.lispworks.com/success-stories/
also Emacs (for another dialect).
Yeah but apart from SBCL, Viaweb, Hacker News, Emacs, Clojure, Scheme, Racket, garbage collection, macros, homoiconicity, the REPL, S-expressions, symbolic computation, what has Lisp ever done for us?
...booleans, conditional expressions, first-class functions, lambdas, closures, eval...
Booleans? I think some guy named Boole might have prior art...
(Yeah, OK, he didn't have a programming language...)
booleans as values of course.
arguably redundant since GP did say symbolic communication, but Lisp had T before Algol 60 came out
> What has Lisp ever done for us?
phanitathion, roaphs, aqueduphs, mediphine, wphine...
I'm sorry. Couldn't resist - I watch Python and do Lisp. Or the other way around, sometimes. And, no, I'm not from Barthselona.
"We were not out to win over the Lisp programmers; we were after the C++ programmers. We managed to drag a lot of them about halfway to Lisp."
- Guy Steele, Java spec co-author
"35 years of Lisping at SISCOG"
https://www.siscog.pt/en-us/blog/35-years-of-lisping-at-sisc...
https://en.wikipedia.org/wiki/Emacs
https://help.autodesk.com/view/OARX/2024/ENU/?guid=GUID-A0E9...
https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
I respect Guy Steele immensely, but his view of Java as being halfway to Lisp is sign that he had jumped the shark by that point, and find it hard to take the rest of his work seriously.
It's acted as a road-concentrating hub to simplify logistics.
The aqueducts!
I've been wondering - Is lisp (common lisp, clojure, scheme) easier for iterative work with LLMs?
Yes, to a certain degree, although it does take initial effort to make it happen.
LLMs do somewhat okay job when you use them with Lisp, treating it just like any other PL, which is roughly the Unix/pipe model - batch-style. Agent spawns process -> reads stdout/stderr -> spawns next process. State lives in-between the calls and in files. Each tool invocation is stateless.
Things get far more interesting when you give an LLM a true Lisp REPL. LLM stops guessing and starts empirically analyzing current state of things and produces working solution faster, costing far less tokens. And you get to watch it solve things interactively, e.g. I often let AI poke through our UI (via Playwright-driven Clojurescript REPL), while monitoring situation in k8s - through nrepl port, connected to Clojure REPL. It literally interactively walks the DOM, finds the selectors, clicks buttons, etc. All without restarts, complex state management and all.
Depends very much on your harness and effective use of tools. For vibe-coding, stick to Python.
It can get good with the right setup. I made Codex work with GToolkit (similarly underrepresented, but technically impressive and a seemingly good fit for LLMs), but it took a lot of tweaking of the project structure, extensive instructions in AGENTS.md, and some custom skills (some borrowed from Gt4Llm, the GT's built-in model harness). Out of the box, it burned tokens and took way too long to implement even basic things.
YMMV - as with everything LLM-related - but I think without a similar setup, an agent instructed to write CL/Clj/Scheme/Racket will have the same issues. It might be better in an established, large project - but starting from an empty Git repo, I suspect you'll have to fight your way to productivity.
I've found LLMs to be bad at balancing parenthesis. I've also found them to be less likely to hallucinate library types in dynamic languages, they tend to hallucinate arguments to library functions/methods instead.
> LLMs to be bad at balancing parenthesis
Because you treating Lisp just like any other (non-homoiconic) PL. Give an agent a true Lisp REPL to mess around, and you'd be surprised. Things get very interesting. I still don't understand why more people don't do that - isn't that obvious first thing anyone should figure out? Like I can't even imagine working with Lisp without a REPL and structural editing - I'd immediately fail at balancing parens. Why do you expect a [dumber] machine would do any better?
Yeah. In my GToolkit setup, I started with giving it eval. At first, it tried to "rebuild the world" for each task, but we gradually settled on a set of images for specific tasks that can be rebuilt as a separate step and are otherwise cached. That gave us fast eval, and it kinda snowballed from there. I thought I would need to implement some IPC into a live image, but the startup is fast enough that it doesn't matter too much. The agent now has both the textual source (in Tonel) on disk and can easily query a live image via CLI.
What do you think about making the agent write type annotations? There are built-in forms in CL, and in Smalltalk, I settled for pragmas for now. They are not checked, but since I started using them, I think the rate of one- or two-shotting solutions has gone up.
> I've found LLMs to be bad at balancing parenthesis.
I think that was true last year. In my experience, it’s no longer the case with Claude Code or Codex.
assuming you're using state of the art llms, when they start failing with paren balancing it means they're approaching the limits of their context window and a new session should be started.
I use LLM_based coding agents frequently with Lisp languages and my way of working is different with Lisp languages: 1) if generated code ever has a syntax error, I like to quickly fix the syntax error myself. 2) for some reason I usually prefer to run tests myself in another terminal (with Python, Typescript, etc. I let the coding harness run tests).
I haven’t let coding harnesses run REPLs. When I do let a coding harness run tests I specify in, for examp,e, my Common Lisp skills file to run ‘sbcl -load …” so bash test commands are one liners.
It would be interesting to work on skills and a harness to use REPLs - nothing bad about that idea, I just haven’t tried it.
Current SOTA models work great with Emacs Lisp and Common Lisp. Giving access to the REPL enables them to do an amazing job but they still work well with Unix input/output static code processing paradigm.
yes if you run a repl server like slynk the llms will happily communicate with your running program to develop and test things. It's often quite creepy to witness.
Why is this being downvoted? It's a reasonable question.
It is fashionable to misunderstand Lisp and hate AI.
I spent years (on my free time) coding Lisp, even had the aspiration (but I was younger then) to rewrite P4V (Perforce Client GUI) in it :) -
https://github.com/malkia/p4bee
but haven't touched it in years....
I just read the (cool!) article...didn't see a single CAR or CDR --was using them quite a bit... last century!
The real test of a programming language is its readabilty.
I looked at lisp many times and it just doesn't do it for me.
That might be, on the other side, I consider Lisp highly readable, but quite a few modern languages quite unreadable. For example parsing type signatures with a complex syntax.
Any style of writing you are not familiar with will require a lot of training. Try reading anything written in the Greek alphabet for example.
Fair enough.
If you've not grown up accustomed to C-style syntax, I assure you that C is less readable. Even a for loop is crazy in C.
I'm working on a Lisp dialect which is far from done and changing every day but I thought I would share it: https://github.com/lodenrogue/hith
It uses python for the interpreter.
Neat. What is the primary use case for this dialect?
Its general purpose. For now I'm trying to build out all the basic language features. Once its built I'll use it as my daily driver for whatever I'm coding.
It's a lisp so it can be used for anything since macros have been implemented. You can create your own syntax using macros.
I played a little bit around with LISP but I was astounded when I saw that you run programs using the live REPL. I thought, how can you reliably deploy a finished program if it can get messed with at any time? Why isn't there a way to compile to a binary?
> I was astounded when I saw that you run programs using the live REPL.
That's one way, you stopped too early in your investigation though.
You can produce binaries, though the precise mechanism will vary by your implementation. And there's no reason to use the REPL for it. You can create an executable file with something like this:
And then `sbcl --load program.lisp` (or whatever you name it) and it'll produce a binary for you. Other implementations will have other methods of achieving the same thing.Or, if you don't need a binary, you can have something like this:
And then run `sbcl --load program.lisp`. That will compile and execute it without ever invoking the REPL.(NB: Using a function named main isn't strictly necessary, I named it that for the example. Name it whatever you want.)
Thank you, that's very helpful to know!
I’m guessing there’s some way for a Lisp program to load a DLL and call a function in it so writing a Windows application using the Windows App SDK is likely possible.
The one part I can’t imagine is how do you pass a Lisp function as a callback?
https://www.quicklisp.org/beta/UNOFFICIAL/docs/cffi/doc/Tuto... has some pointers (which might be harder to understand unless you have read the rest of https://www.quicklisp.org/beta/UNOFFICIAL/docs/cffi/doc/Tuto... first). The short version (which I might be mangling in my attempt to shorten it) is, since Lisp is so very good at creating code, you have your Lisp program create a C function that will be passed as the callback to the API. Then the C function that you created will convert the C data structures to Lisp data structures, and call the Lisp function with those Lisp data structures.
If you're really good at Lisp, you can write that automatic-callback-creation function yourself. But if you're really, really good at Lisp, you will just use one of the ones that other people have already written, e.g. `define-alien-callable` from https://www.sbcl.org/manual/#Calling-Lisp-From-C or some other similar FFI.
Yes, Allegro Common Lisp, and Lisp Works support this just fine.
Can I write a game using OpenGL in Lisp?
Yes. Kandria [1] is a great example of someone doing exactly this. There's a bunch of information on the development here [2] and the code is open source [3].
[1] https://kandria.com/
[2] https://reader.tymoon.eu/article/413
[3] https://codeberg.org/shirakumo/kandria
Interesting. I didn't thought it's possible.
I really love the idea of lisp but feels like the learning curve is steep. Sure you can learn basic lisp in an hour or so but then? You need to learn emacs, all of the plugins, module systems.
Maybe I just don't know where to start.
Start here with Common Lisp 8-) https://lispcookbook.github.io/cl-cookbook/getting-started.h... To learn the language, look at the good books out there (Touretzky's etc).
ps: you don't need to learn Emacs. A terminal even is enough. See the Cookbook/ Editors section.
> So why Lisp (or when)
> [...] among its extensibility, its interactive environment, the REPL, and a lot of other features we haven’t touched yet. It is the combination of all of them that makes Lisp programming what it is.
Agreed. However, although the alternatives are few, they do exist. Today, I'd like to convince you (whether you're OP or a commenter) to give one of them a try. I'm talking about GToolkit[1]: Smalltalk/Pharo-based reimagining of Smalltalk as a productive environment for modern system design, analysis, and implementation. It's based on Pharo and its VM, but with GT-specific extensions and replacements, developed on GitHub in both Smalltalk and Rust.
I used both Common Lisp and Smalltalk over the years for some of my side projects. Technically, the environments are comparable: image-based, live, interactive development is central to both. Lisp is easier to fit into modern workflows: it's still just files on disk. If you ignore the REPL and treat the image as a bundler, you can have a Java- or Python-like development workflow. You can gradually adopt the more interactive ways of working with the codebase. Smalltalk was historically more of an "all or nothing" approach, but nowadays it supports Git-based workflows as first-class, with GToolkit providing additional tools directly. GT is tied to a single IDE, but that IDE is genuinely powerful, pragmatic, and easy to customize: the entire IDE is Smalltalk code[2] that lives in the image beside your code, so you can live-edit any part of it at any time.
The languages differ, most obviously in the object model (single inheritance and message sends in Smalltalk, and multiple inheritance with multimethods in CLOS), but I don't think one is strictly better than the other for the vast majority of code. You just need to structure your code differently. Both systems are very dynamic, so neither is like Java or C++. After quite a few projects in both, I am convinced that this difference doesn't matter.
Both languages have very simple syntax. Smalltalk doesn't have macros; instead, it has an extensible/replaceable compiler for method bodies. This capability is used, for example, for compiling grammars into parsers/lexers in SmaCC. It's not as convenient for control-flow abstraction as CL's macros, but Smalltalk tends to make them out of blocks (lexical closures with non-local exits) plus actions on thisContext. In practice, I never found either language too limiting in what I could express, and both have small, regular basic syntax.
TL;DR: Common Lisp and GToolkit Smalltalk offer comparable technical merits, with the most pronounced difference being GT's built-in, high-quality IDE, which open source Lisp lacks.
The IDE (and the capability to build cross-platform, but natively rendered GUI apps) is the major selling point of GToolkit, but the bigger reason to consider it is social. Lisp had a bit of a renaissance from 2005 to 2015, but ~it has since died down~ (EDIT: After checking a bit, it's more like there are similar levels of development now, but it's just varied, and I personally hit a few areas where nothing new happened for a while. Or in other words, the hopes of CL getting catapulted to mainstream by the wave of adoption didn't materialize, and the adoption seems to have largely plateaued, with a slight upward trend since then.) A lot of great things for Lisp appeared in that time: ASDF and Quicklisp being prime examples. Unfortunately, the momentum was lost, and since then, the rate of development has plummeted. It's still a solid proposition because of its stability, which means a lot of old code still works perfectly well, but it's a double-edged sword: it would take a hundred miracles in a row for CL to get a plausible M:N concurrency story, for example. This isn't true for Smalltalk, and doubly so.
There's a small but active community around Pharo, mostly academics and hobbyists (IME; and it's from a decade ago, so YMMV). Pharo itself is already a "Smalltalk-like" language, and it regularly gets features that are not in original Smalltalk (stateful traits and slots (reification of class and instance variables) are good examples). It continues to evolve, and each version brings notable improvements to the language, the VM, and the class library. In CL, since there are many implementations maintained by various groups and the standard is set in stone, adopting extensions to the standard is incredibly hard and time-consuming. You can, of course, commit to a single implementation, but there are only 2 or 3 implementations that are actively developed and trying to do something "new"; they are all experimental and incomplete. And while SBCL has quite a few nice features, its codebase was a bit hard for me to grok (read: days on end of banging my head against a wall, then giving up; I'm much more comfortable messing with Smalltalk bytecode compiler).
Secondly, GToolkit is driven by a group of coders[3] who have established a profitable consultancy. It's being developed around a single vision that the authors swear helps them achieve their projects' goals. Even if you're not sold on moldable development[4] as a methodology, GToolkit gives you all the tools you might need in a modern development and then adds tons of domain-specific tools and utilities that might be useful after just a bit of adjustment. The whole environment is built to make such adjustments as painless as possible, too.
The only problem I experience with GToolkit is the lack of structured, book-like documentation. The GToolkit book is closer to the PHP wiki or a bundle of tutorials than something like the Rust book. It's not that the docs aren't there - they often are, but finding them effectively is a bit challenging. I found that simply cloning all the repos that GT consists of (tens, if not hundreds, but there's automation for that) and pointing an agent to the directory with them is often enough to quickly find what I'm looking for (if the built-in Spotter fails and I'm too lazy to construct an in-image search).
TL;DR: GToolkit/Pharo move much faster than the CL world, and while the number of maintainers might be similar (honestly hard to estimate, though), the effort on the GT side is more centralized and, in my opinion, heads in a better direction. If you want an extensible, live, dynamic environment that shares many of Common Lisp's strengths, but is more polished, more actively maintained, and feels more modern, take a look at GToolkit.
(Disclaimer: no affiliation, just a programmer who likes to explore unpopular languages).
[1] https://gtoolkit.com/
[2] Some parts are implemented as Rust dynamic libraries and called from Smalltalk via FFI.
[3] https://feenk.com/
[4] https://moldabledevelopment.com/
All three major programming environments at Xerox PARC, shared similar concepts.
Interlisp-D, Smalltalk, Mesa (XDE) which evolved into Cedar.
If you read Xerox papers about all of them, there are several quotes on how relevant it was to share the same programming experience across environments.
Which is why, given their linage, JVM and CLR are the closest big mindshare ecosystems that somehow still have traces of those features when using their IDEs and runtimes, even without being a proper Smalltalk or Lisp.
> how relevant it was to share the same programming experience across environments.
Right. If you view the language as just a part of a bigger "programming experience", you can do with a worse language with better tooling (pathological example: (lack of) namespaces in many Smalltalks to this day). If you focus on language design above all else, you end up with either something pretty on paper (but never implemented) or something practical and (kind of) elegant, but it'll take 40 years to get the tooling to where you want it (and still won't cover many important capabilities).
Also, it's easier to swap languages if you focus more on tooling. From what I read, many Lisp and Smalltalk IDEs/environments were (and some still are) polyglot. I sadly didn't have access to them at the time, but I imagine switching from Turbo Pascal to Turbo C++ was easier than going from Turbo C++ to Visual C++? That said, tooling tends to break if the language actively fights it, so it's not like PL design is completely irrelevant.
> Which is why, given their linage, JVM and CLR are the closest big mindshare ecosystems that somehow still have traces of those features when using their IDEs and runtimes
Yes. The only problem is that those concepts/features become niche black magic (somewhat unavoidable for advanced features in "big mindshare ecosystem"). You can definitely compile Java classes dynamically at runtime, install them, and use them normally; however, outside of frameworks/IDEs/JVM languages, the ClassLoader is used mostly for loading static assets, and even simple reflection is viewed with suspicion.
Using Smalltalk or Lisp directly puts those powerful concepts "front and center", which a) is good for education; and more importantly, b) feels good when things click :) It could be an advantage for small groups of programmers, but from what I understand, it becomes a liability in large organizations. Pharo is especially bad at it, and GToolkit doesn't fix everything (they do a good job at providing tools, but don't want to maintain too large a patch set for Pharo, which is a good trade-off given their limited resources). But I'm replying to "Why Lisp (or When)", and in this context, Smalltalk is (IMO) the better direct alternative than JVM/CLR.
I am fascinated by lisp but only had the time to mess with clojure for about a year.
My weird take is that lisp would have a better shot if one would replace the parentheses with square brackets. It might just be me but the round parenthesis make it visually painful to look at code blocks. I feel square brackets would fix that.
Interesting read
Just working a very powerful dynamic workflow feature in my agent product. And guess what, LISP is the one I chose for the generated workflow. LoL
[dead]
It's really fun: I read so many propaganda on Lisp here that by now we should all be coding in it. Don't we get perhaps weekly posts like this?
But ... somehow almost everyone simply ignores this propaganda --- that's actually the way we ought to handle propaganda! Funny that it works with Lisp, but not with many political topics where it can be shown that propaganda really works.
My experience with Lisp is somewhat special-cornered. I only know Emacs-Lisp, and that not great. And for me, Lisp an out, outdated and confusing thing.
I don't know ANY other programming language that elected assembler mnemonics to be "high"-level expression: car and cdr.
While I love Emacs, they claim all the time that is has 100% discovery through build in doc. But that is only 70% true. It doesn't exist in Lisp like it does in other languages. For example, I have an association list. Now, what functions exist that uses them? I can't use the equivalent for (dir alist), alist.dir(), or use some LSP for that. I can't even look in the symbols, because operators are written like assoc or assq with not logic. And then there are either prefixes to them (rassoc) or postfixes (assoc-string).
That there is no namespace is soooo 1970. I mean, even Turbo-Pascal had namespaces.
Typechecking is nonexistant.
At least we have byte-compiling.
And note that I don't croak about the parenthesis. While I find it mildy weird that I as a human program in almost-AST, Emacs has superp support for the parenthesis. So people croaking about them just had never had a good editor IMHO.
Now, one can argue that Emacs Lisp != Common Lisp. But these propaganda articles don't tend to promote one specific Lisp, they only talk in generals. So I can apply this general principle and compare it to the tiny corner of these awful language family.