Wow, I'm not sure I've ever seen this (or if I did, it was 20 years ago).
And I was definitely looking around for this kind of history when I was searching around when writing. Perhaps my google skills have decayed... or google... or both!
Honestly, it feels like something like this should have been put in the standard instead of all the English prose that ended in the section about the preprocessor expansion. Yeah, it's not pretty, but at least it requires way less skill in hermeneutics to understand correctly.
I think the C preprocessor was designed after the GPM clone m6, its successor m4, and Ratfor, so I suspect the difficulty in doing things like this is intentional. I guess I should ask McIlroy, who is responsible for pushing m4 to its absolute limits and was present when the C preprocessor was being designed: https://www.cs.dartmouth.edu/~doug/barem4.m4
_ Pure macros as a programming language
_
_ m4 is Turing complete even when stripped to the bare minimum
_ of one builtin: `define'. This is not news; Christopher
_ Strachey demonstrated it in his ancestral GPM, described in
_ "A general- purpose macrogenerator", The Computer Journal 8
_ (1965) 225-241.
_
_ This m4 program more fully illustrates universality by
_ building familiar programming capabilities: unlimited
_ precision integer arithmetic, boolean algebra, conditional
_ execution, case-switching, and some higher-level operators
_ from functional programming. In support of these normal
_ facilities, however, the program exploits some unusual
_ programming idioms:
_
_ 1. Case-switching via macro names constructed on the fly.
_ 2. Equality testing by redefining macros.
_ 3. Representing data structures by nested parenthesized lists.
_ 4. Using macros as associative memory.
_ 5. Inserting nested parameter symbols on the fly.
_
_ Idioms 2 and 5 are "reflective": the program writes code
_ for itself.
It's very easy to get into enormous amounts of trouble in m4, m6, or GPM. The C preprocessor is not without its problems, but it is rare that I have difficulty in understanding why a given gcc -E invocation produces the output it does.
A simpler explanation of the technique in the article that I believe matches the C standard more closely is the following.
Ideally the result of processing a macro ("macro-replacement") would contain no further macros. But in order to disallow infinite loops during processing of a macro, the C standard specifies that names of macros that would be processed recursively are instead painted blue ("marked") so as to never be processed.
The technique in the article hinges on the fact that macro-replacement, as specified in the C standard, allows not only for the result of macro-replacement to contain these marked do-not-process macros, but also unmarked unprocessed macros. Specifically, such an unmarked unprocessed macro is a functional macro arising after processing a macro with an empty definition that separates the functional macro's name from its arguments.
The technique for achieving recrusive macros is to introduce a functional macro whose definition is the would-be recursive macro, and to replace each occurence of the name of the would-be recursive macro in its own defintion with the functional macro interrupted by an macro with an empty definition.
The result is an unmarked unprocessed recursive macro. Processing such a recursive macro, e.g. by including it as an argument to a macro, corresponds to taking one step of the recursion. Thus any pre-determined finite number of steps of a recurisvely defined macro can be performed.
For example, the would-be recursive macro of the article
There you can find a recursive macro expansion implementation (as a gcc hack) that fits on a slide:
#2""3
#define PRAGMA(...) _Pragma(#__VA_ARGS__)
#define REVIVE(m) PRAGMA(push_macro(#m))PRAGMA(pop_macro(#m))
#define DEC(n,...) (__VA_ARGS__)
#define FX(f,x) REVIVE(FX) f x
#define HOW_MANY_ARGS(...) REVIVE(HOW_MANY_ARGS) \
__VA_OPT__(+1 FX(HOW_MANY_ARGS, DEC(__VA_ARGS__)))
int main () {
printf("%i", HOW_MANY_ARGS(1,2,3,4,5)); // 5
}
It sounds like the one in the article works for more compilers, but there doesn't seem to be a copy-pasteable example anywhere to check for myself. Also, the "Our GitHub Org" link on the site just links to github.com.
Absolutely, the code box under the ascii art is a complete implementation, just paste that in a C file, and then use `H4X0R_VA_COUNT(...)`.
Or, you could follow the link the my typed variadic arguments article (from which this post forked off). The repo there is: https://codeberg.org/h4x0r/vargs
And yes, GCC extensions are often going to be adopted in clang, but generally not the broader world of C and C++ compilers. Everything in my article conforms to the standard.
I played with a lot of preprocessor implementations and did my own (redesigned chibicc's expansion algorithm), not many of them even have paint-blue behavior exactly right (the standard text is vague, to me it was more "matching GCC's" than "conforming to standard").
That's interesting. I agree with you that the standards text is pretty vague. I think that's why other attempts to show how to do this kind of thing don't get deep enough on the semantics, and why I adopted a "try it and see" strategy.
I do try to avoid this kind of thing unless necessary, so I don't have experience as to where the different compilers will fall down on different corner cases. I'd find it very interesting though, so please do share if you kept any record or have any memory!
Examples from this article[1] frequently yield inconsistent result across implementations, particularly the ones that put parenthesis in macros, it is indeed a very corner-case-y thing to do though.
From my read, that article doesn't have anything in it that isn't standards compliant C. My memory is getting worse as I get older, but I'd say by C17 all that was standard, and definitely in C23.
I've noticed many people are still building on systems where the compiler is a bit older, and defaults to C11, even if it has support for C17, so perhaps that's the problem?
It is-- if you click in to their full list, you should see it near the top in their "above the water" section, under `#pragma once`. I suspect it was added after the meme image was produced.
But, if it weren't on the iceberg page, it'd make sense. The semantics of `do { ... } while(0)` are in the standard, and the preprocessor has nothing to do with those semantics.
You are, of course, right, that the construct is used all over the place in macros for good reason, though these days every compiler I care about has the same (I believe not in the standard) syntax for statement expressions, which will work in even more contexts.
One can also (ab)use the build system to run arbitrary preprocessing steps with any language over the "C" input. You can have recursive macros by using M4 or Perl or Python or some other language to expand them, converting your "foo.c.in" into a "foo.c" to hand off to the C preprocessor & compiler. It still feels dirty, but it's often much easier to understand & debug.
Yes, 100%. And since CPP doesn't actually understand C, it's not too hard to do some lightweight preprocessing that requires some real additional parsing.
But while CPP is pretty finicky and not very modern, getting such things working seamlessly with C build systems can be vastly worse (though better than the days where the GNU tools were ubiquitous).
I tend to find meson easy to use compared to all the others, and do this kind of thing, but it's still difficult and brittle.
I wonder if the author is aware of the __VA_TAIL__ proposal[1], it covered similar grounds and IMO very well thought out, but unfortunately not accepted into C2Y (judging from committee meeting minutes).
Yes, I know that it was not accepted, but do not have any color on why not. It's well thought out; but I do not think the semantics are self-evident to the average C programmer who already finds the preprocessor inscrutable.
genuinely remarkable, the altogether perhaps even productive mischief you can get up to, especially with `__VA_OPT__` becoming a proper standard in both C and C++ so you don't have to feel dirty about using it.
i recently made use of plenty of ugly tricks in this vein to take a single authoritative table of macro invocations that defined a bunch of pixel formats, and make them graduate from defining bitfield structs to classes with accessors that performed good old fashioned shifts and masks, all without ever specifying the individual bit offsets of channels, just their individual widths, and macro magic did the rest. no templates, no actual c++, could just as feasibly produce pure c bindings down the line by just changing a few names.
getting really into this stuff makes you stop thinking of c function-like macros as functions of their arguments as such, but rather unary functions of argument lists, where arity roughly becomes the one notion vaguely akin to typing in the whole enterprise, or at least the one place where the compiler exhibits behaviour resembling that of a type checker. this was especially true considering the entries in the table i wound up with were variadic, terminating in variably many (name, width) parenthesised tuples. and i just... had the means to "uncons" them so to speak. fun stuff.
this is worth it, imo, in precisely one context, which is: you want a single source of truth that defines fiddly but formulaic implementations spread across multiple files that must remain coordinated, and this is something you do infrequently enough that you don't consider it worthwhile introducing "real" "big boy" code gen into your build process. mind, you usually do end up having to commit to a little utility header that defines convenient macros (_Ex and such in the article), but hey. c'est la vie. basically x macros (https://en.wikipedia.org/wiki/X_macro) on heart attack quantities of steroids.
Here is something similar: https://godbolt.org/z/Yj61b6GGj
I useed a non-cryptographic PRNG to write a C program that only compiles if you know the correct key.
In many ways being limited ends up being a feature. Even limited as it is, you get some crimes against humanity like the bourne shell source, but at least most people agree it is a bad idea
If it allowed more unlimited metaprogramming, building big complex things as macros might well have become popular
The CPP does allow quite advanced metaprogramming, it's just so obtuse to use and requires insane hacks so almost nobody does. See one of my favorite projects https://github.com/hirrolot/metalang99
Funnily enough, the difference between passing ... and locally-allocated void*[] is basically who has to spill the data to the stack, the caller or the called function.
Well, I've done it that way if I'm willing to limit myself to pointers or ints up to a pointer size, but that doesn't work with floats or doubles, for instance.
Ergonomically, I have tended to start using _Generic for static type checking where possible, and that pushes me more toward to avoiding arrays in types for this kind of thing.
The lack of (easy) recursion in CPP is so frustrating because it was always available in assembly languages with even very old and very simple macro assemblers- with the caveat that the recursion depth was often very limited, and no tail call elimination. For example, if you need to fill memory:
; Fill memory with backward sequence
macro fill n
word n
if n != 0
fill n - 1
endif
endm
So "fill 3" expands to:
word 3
word 2
word 1
word 0
There is no way this was not known about when C was created. They must have been burned by recursive macro abuse and banned it (perhaps from m4 experience as others have said).
The other assembly language feature that I missed is the ability to switch sections. This is useful for building tables in a distributed fashion. Luckily you can do it with gcc.
I've ready the article 4 times already today and I'm still crying. This looks like the solution to a problem I'm having (C++, but I'm doing things that templates and constexpr can't do), but trying to get it all to work is painful. Kudos to the author at making an attempt to explain it.
Not so much that you are not clear - this is an area that is weird enough that I don't think it is possible to be clear, but you gave me some useful hints. I got some of what I want working.
Edit, I started writing and then realized that I need this to work on old gcc that doesn't support these tricks so I have to give up. (I can't wait until this old embedded hardware dies)
What I'm trying to do is #define(A, B) static int A ## _ ## B = register(MKSTRING(A ## _ ## B) take any number of arguments. I can get the register working, but my attempts are creating the variable name fail.
Thanks. Also, per another person in the thread, here are the two annotations he or she was asking for, heavily annotated.
They are both only a couple lines, but it deals with things like the fact that you've got one more argument than you should have commas, and the use of the # and ## operators in these things.
Thanks. I didn't have enough levels of indirection. Once I add enough levels of indirection it works. Well works on my newer systems, I have to build for an embedded system where I'm stuck on an old compiler that doesn't support these tricks.
To be clear, the example I provided for the other person explains the bit you're missing where the names aren't working... if you carefully follow the rules, the # and ## operators don't allow expansion on their arguments, so you have to use a layer of indirection to get them expanded first.
This was back in 80s when you were working on c compilers? That’s an interesting story (80s compiler scene and what you worked on) I’ve picked up bits and pieces over the years, have you written it up anywhere? Would be benefit for many I think.
That has sometimes been true, but for example C++ didn't have variadic macros, maybe still doesn't, because they were added to C after Microsoft decided to stop following C development. https://stackoverflow.com/a/21515839 goes into details of other differences.
Function like macros literally requires name( , i.e name followed directly by open paren, otherwise no macro substitution occurs.
so (name)() will always suppress function like macros (but not non-function ones, i.e regular #define name xxx)
Okay, finally found some time to provide you with a fully annotated example of your original ask here, assuming you wanted to transform the arguments passed to F into IDs, and the arguments passed to G into strings (as seemed to be the case from the rest of the thread).
I've fully annotated it, so it might seem like more than it is. About half the macro code is from the original article (the chunk at the top). And I do implement both transforms for you.
Each one I think is only 6 lines of code by itself, despite the rediculous amount of exposition in the comments.
I'm on my phone, but if you start with the top 8 lines in the code box under the ascii art, you'll get an implementation of `H4X0R_MAP()`; the bottom two lines are an example, and you can just write yourself a body that produces one term. Only thing you need to know beyond that is the stringify operator.
And I should say, if you want to apply the same transformation to arguments twice, but call F() separately from G() per your starting example, you'd just apply your map twice in your top-level macro.
> if you want to apply the same transformation to arguments twice, but call F() > separately from G() per your starting example, you'd just apply your map twice in your top-level macro
From a DoS risk perspective there is no practical difference between an infinite loop, or a finite but arbitrarily large loop, which was always possible.
For example, this doesn't work:
#define DOUBLE(x) DOUBLE(x) DOUBLE(x)
DOUBLE(x)
That would only expand once and then stop because of the rule against repeated expansion. But nothing prevents you from unrolling the first few recursive expansions, e.g.:
#define DOUBLE1(x) x x
#define DOUBLE2(x) DOUBLE1(x) DOUBLE1(x)
#define DOUBLE3(x) DOUBLE2(x) DOUBLE2(x)
#define DOUBLE4(x) DOUBLE3(x) DOUBLE3(x)
DOUBLE4(x)
This will generate 2^4 = 16 copies of x. Add 60 more lines to generate 2^64 copies of x. While 2^64 is technically a finite number, for all practical purposes it might as well be infinite.
Without any specific implementation of a constraint it certainly can happen, although I'm not totally sure that it's something to be concerned about in terms of a DOS as much as a nuisance when writing code with a bug in it; if you're including malicious code, there's probably much worse things it could do if it actually builds properly instead of just spinning indefinitely.
Rust's macros are recursive intentionally, and the compiler implements a recursion limit that IIRC defaults to 64, at which point it will error out and mention that you need to increase it with an attribute in the code if you need it to be higher. This isn't just for macros though, as I've seen it get triggered before with the compiler attempting to resolve deeply nested generics, so it seems plausible to me that C compilers might already have some sort of internal check for this. At the very least, C++ templates certainly can get pretty deeply nested, and given that the major C compilers are pretty closely related to their C++ counterparts, maybe this is something that exists in the shared part of the compiler logic.
C++ also has constexpr functions, which can be recursive.
All code can have bugs, error out and die.
There are lots of good reasons to run code at compile time, most commonly to generate code, especially tedious and error-prone code. If the language doesn't have good built-in facilities to do that, then people will write separate programs as part of the build, which adds system complexity, which is, in my experience, worse for C than for most other languages.
If a language can remove that build complexity, and the semantics are clear enough to the average programmer (For example, Nim's macro system which originally were highly appealing (and easy) to me as a compiler guy, until I saw how other people find even simple examples completely opaque-- worse than C macros.
1. compile time evaluation of functions - meaning you can write ordinary D code and execute it at compile time, including handling strings
2. a "mixin" statement that has a string as an argument, and the string is compiled as if it were D source code, and that code replaces the mixin statement, and is compiled as usual
No. Other modern languages have strong compile-time execution capabilities, including Zig, Rust and C++. And my understanding is that C is looking to move in that direction, though as with C++, macros will not go away.
It's a feature... macros allows people to change the language at will which is great when you're researching programming languages (like in lisp) but less good when you want maintainable and consistent code
C++ has more powerful metaprogramming and look how that turned out
At least you can implement things like vector in C++. I challenge anyone to write the same for C macros. (this is semi-serious - if you get a full generic STL for C that is usable that would prevent a large number of bugs. The implementation can be as bad as you want, so long as the user API is not too difficult. Though note that I'm not a C programmer and so if there are better ways someone else is doing this I'm not aware of it)
Some of the stuff you're doing in the library I've also been doing recently, and has been working well, like using the string of the type name to allow for run-time checking of, what I would called "mixed" data (your variadic type). I've also done the same basic thing as your option type in a way that's closer to your sum type than the maybe type.
But I'd had enough problems trying to get something like your vector actually working that I'd given up, but I think now I'll build something at some point over the holidays. I think as I'm coming up to speed on the history of changes to _Generic that's partially due to the attempts before I had a C23 compiler, but even then, your code there is impressive-- both clear and clever.
I also have enough stuff passed via pointer that the option type for me needed to handle pointers differently-- I basically just have run-time code that does the null check at run time when set. At that point, there doesn't really need to be an 'is_set' type flag.
By the way, the run-time type checking is one of the use cases where I really would like to have a tractable interface for hashing at compile time, preventing unbounded strcmps, and turning them into an integer compare (without having to manage caching such things at runtime).
Same thing would be for making it much easier to statically lay out fixed caches implemented via hashing, instead of inserting into them at start-up, etc.
that is amazing... I don't write C so I didn't dig too deep, but kudos for getting anything to work. Now you use need great documentation so we can figure out how to use it and what it supports without digging into the macros themselves. (tests would be good too, but maybe they are there and I didn't see them).
The behavior of C macros is actually described by a piece of pseudocode from Dave Prosser and it is not in the standard:
* https://www.spinellis.gr/blog/20060626/
* https://www.spinellis.gr/pubs/jrnl/2006-DDJ-Finessing/html/S...
* https://gcc.gnu.org/legacy-ml/gcc-prs/2001-q1/msg00495.html
This is the best explanation I've seen so far:
https://marc.info/?l=boost&m=118835769257658&w=2
This needs to be framed somewhere, very lucid explanation
Wow, I'm not sure I've ever seen this (or if I did, it was 20 years ago).
And I was definitely looking around for this kind of history when I was searching around when writing. Perhaps my google skills have decayed... or google... or both!
Thanks very much.
Oh, are you L33 T.?
If my google fu is that bad, it's pretty much impossible to be.
Oh, I understood you to be saying you'd written this article.
Honestly, it feels like something like this should have been put in the standard instead of all the English prose that ended in the section about the preprocessor expansion. Yeah, it's not pretty, but at least it requires way less skill in hermeneutics to understand correctly.
Thank you very much for providing these links!
I think the C preprocessor was designed after the GPM clone m6, its successor m4, and Ratfor, so I suspect the difficulty in doing things like this is intentional. I guess I should ask McIlroy, who is responsible for pushing m4 to its absolute limits and was present when the C preprocessor was being designed: https://www.cs.dartmouth.edu/~doug/barem4.m4
It's very easy to get into enormous amounts of trouble in m4, m6, or GPM. The C preprocessor is not without its problems, but it is rare that I have difficulty in understanding why a given gcc -E invocation produces the output it does.A simpler explanation of the technique in the article that I believe matches the C standard more closely is the following.
Ideally the result of processing a macro ("macro-replacement") would contain no further macros. But in order to disallow infinite loops during processing of a macro, the C standard specifies that names of macros that would be processed recursively are instead painted blue ("marked") so as to never be processed.
The technique in the article hinges on the fact that macro-replacement, as specified in the C standard, allows not only for the result of macro-replacement to contain these marked do-not-process macros, but also unmarked unprocessed macros. Specifically, such an unmarked unprocessed macro is a functional macro arising after processing a macro with an empty definition that separates the functional macro's name from its arguments.
The technique for achieving recrusive macros is to introduce a functional macro whose definition is the would-be recursive macro, and to replace each occurence of the name of the would-be recursive macro in its own defintion with the functional macro interrupted by an macro with an empty definition.
The result is an unmarked unprocessed recursive macro. Processing such a recursive macro, e.g. by including it as an argument to a macro, corresponds to taking one step of the recursion. Thus any pre-determined finite number of steps of a recurisvely defined macro can be performed.
For example, the would-be recursive macro of the article
#define _COUNT_ONE(x, ...) + 1 _COUNT_TOP(__VA_ARGS__)
#define _COUNT_TOP(...) __VA_OPT__(_COUNT_ONE(__VA_ARGS__))
#define COUNT(...) (_COUNT_TOP(__VA_ARGS__) + 0)
becomes
#define EMPTY
#define _COUNT_INDIRECT() _COUNT_ONE
#define _COUNT_ONE(x, ...) + 1 _COUNT_TOP(__VA_ARGS__)
#define _COUNT_TOP(...) __VA_OPT__(_COUNT_INDIRECT EMPTY()(__VA_ARGS__))
#define COUNT(...) (_COUNT_TOP(__VA_ARGS__) + 0)
To process the COUNT(...) macro 5 times, allowing it to count up to 5 variable arguments, nest it 5 levels deep as an argument to a macro.
#define EVAL1(...) __VA_ARGS__
#define EVAL5(...) EVAL1(EVAL1(EVAL1(EVAL1(EVAL1))))
EVAL5(COUNT(1,2,3))
Related: The Preprocessor Iceberg https://jadlevesque.github.io/PPMP-Iceberg/
There you can find a recursive macro expansion implementation (as a gcc hack) that fits on a slide:
It sounds like the one in the article works for more compilers, but there doesn't seem to be a copy-pasteable example anywhere to check for myself. Also, the "Our GitHub Org" link on the site just links to github.com.Author of the article here.
Absolutely, the code box under the ascii art is a complete implementation, just paste that in a C file, and then use `H4X0R_VA_COUNT(...)`.
Or, you could follow the link the my typed variadic arguments article (from which this post forked off). The repo there is: https://codeberg.org/h4x0r/vargs
And yes, GCC extensions are often going to be adopted in clang, but generally not the broader world of C and C++ compilers. Everything in my article conforms to the standard.
I played with a lot of preprocessor implementations and did my own (redesigned chibicc's expansion algorithm), not many of them even have paint-blue behavior exactly right (the standard text is vague, to me it was more "matching GCC's" than "conforming to standard").
That's interesting. I agree with you that the standards text is pretty vague. I think that's why other attempts to show how to do this kind of thing don't get deep enough on the semantics, and why I adopted a "try it and see" strategy.
I do try to avoid this kind of thing unless necessary, so I don't have experience as to where the different compilers will fall down on different corner cases. I'd find it very interesting though, so please do share if you kept any record or have any memory!
Examples from this article[1] frequently yield inconsistent result across implementations, particularly the ones that put parenthesis in macros, it is indeed a very corner-case-y thing to do though.
[1] https://www.scs.stanford.edu/~dm/blog/va-opt.html#c-macro-ov...
From my read, that article doesn't have anything in it that isn't standards compliant C. My memory is getting worse as I get older, but I'd say by C17 all that was standard, and definitely in C23.
I've noticed many people are still building on systems where the compiler is a bit older, and defaults to C11, even if it has support for C17, so perhaps that's the problem?
Ask and you'll get: https://c.godbolt.org/z/rKsWT5E9T
It seems that MSVC doesn't like those macros, though.
Works with MSVC if you add /Zc:preprocessor (to get a standard compliant preprocessor instead of the legacy one).
You know you're in for a wild ride when the `do { ... } while(0)` hack isn't even on the iceberg.
It is-- if you click in to their full list, you should see it near the top in their "above the water" section, under `#pragma once`. I suspect it was added after the meme image was produced.
But, if it weren't on the iceberg page, it'd make sense. The semantics of `do { ... } while(0)` are in the standard, and the preprocessor has nothing to do with those semantics.
You are, of course, right, that the construct is used all over the place in macros for good reason, though these days every compiler I care about has the same (I believe not in the standard) syntax for statement expressions, which will work in even more contexts.
Ah, I meant that it was in the uppermost, least obscure section, "above the iceberg", not that it wasn't in the picture at all :)
I did the first week of AOC22 in the C preprocessor: https://github.com/camel-cdr/boline/tree/main/aoc22
One can also (ab)use the build system to run arbitrary preprocessing steps with any language over the "C" input. You can have recursive macros by using M4 or Perl or Python or some other language to expand them, converting your "foo.c.in" into a "foo.c" to hand off to the C preprocessor & compiler. It still feels dirty, but it's often much easier to understand & debug.
Yes, 100%. And since CPP doesn't actually understand C, it's not too hard to do some lightweight preprocessing that requires some real additional parsing.
But while CPP is pretty finicky and not very modern, getting such things working seamlessly with C build systems can be vastly worse (though better than the days where the GNU tools were ubiquitous).
I tend to find meson easy to use compared to all the others, and do this kind of thing, but it's still difficult and brittle.
I wonder if the author is aware of the __VA_TAIL__ proposal[1], it covered similar grounds and IMO very well thought out, but unfortunately not accepted into C2Y (judging from committee meeting minutes).
[1] https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3307.htm
Yes, I know that it was not accepted, but do not have any color on why not. It's well thought out; but I do not think the semantics are self-evident to the average C programmer who already finds the preprocessor inscrutable.
genuinely remarkable, the altogether perhaps even productive mischief you can get up to, especially with `__VA_OPT__` becoming a proper standard in both C and C++ so you don't have to feel dirty about using it.
i recently made use of plenty of ugly tricks in this vein to take a single authoritative table of macro invocations that defined a bunch of pixel formats, and make them graduate from defining bitfield structs to classes with accessors that performed good old fashioned shifts and masks, all without ever specifying the individual bit offsets of channels, just their individual widths, and macro magic did the rest. no templates, no actual c++, could just as feasibly produce pure c bindings down the line by just changing a few names.
getting really into this stuff makes you stop thinking of c function-like macros as functions of their arguments as such, but rather unary functions of argument lists, where arity roughly becomes the one notion vaguely akin to typing in the whole enterprise, or at least the one place where the compiler exhibits behaviour resembling that of a type checker. this was especially true considering the entries in the table i wound up with were variadic, terminating in variably many (name, width) parenthesised tuples. and i just... had the means to "uncons" them so to speak. fun stuff.
this is worth it, imo, in precisely one context, which is: you want a single source of truth that defines fiddly but formulaic implementations spread across multiple files that must remain coordinated, and this is something you do infrequently enough that you don't consider it worthwhile introducing "real" "big boy" code gen into your build process. mind, you usually do end up having to commit to a little utility header that defines convenient macros (_Ex and such in the article), but hey. c'est la vie. basically x macros (https://en.wikipedia.org/wiki/X_macro) on heart attack quantities of steroids.
I wept when the author mentioned implementing SHA256 in macros.
Here is something similar: https://godbolt.org/z/Yj61b6GGj I useed a non-cryptographic PRNG to write a C program that only compiles if you know the correct key.
LOL, I suffered so you didn't have to.
In many ways being limited ends up being a feature. Even limited as it is, you get some crimes against humanity like the bourne shell source, but at least most people agree it is a bad idea
If it allowed more unlimited metaprogramming, building big complex things as macros might well have become popular
The CPP does allow quite advanced metaprogramming, it's just so obtuse to use and requires insane hacks so almost nobody does. See one of my favorite projects https://github.com/hirrolot/metalang99
The CPP allows “advanced metaprogramming” like throating a banana can sometimes sound like speech
Also, thanks, now I can finally use
ergonomically: Funnily enough, the difference between passing ... and locally-allocated void*[] is basically who has to spill the data to the stack, the caller or the called function.Well, I've done it that way if I'm willing to limit myself to pointers or ints up to a pointer size, but that doesn't work with floats or doubles, for instance.
Ergonomically, I have tended to start using _Generic for static type checking where possible, and that pushes me more toward to avoiding arrays in types for this kind of thing.
The lack of (easy) recursion in CPP is so frustrating because it was always available in assembly languages with even very old and very simple macro assemblers- with the caveat that the recursion depth was often very limited, and no tail call elimination. For example, if you need to fill memory:
There is no way this was not known about when C was created. They must have been burned by recursive macro abuse and banned it (perhaps from m4 experience as others have said).The other assembly language feature that I missed is the ability to switch sections. This is useful for building tables in a distributed fashion. Luckily you can do it with gcc.
I've ready the article 4 times already today and I'm still crying. This looks like the solution to a problem I'm having (C++, but I'm doing things that templates and constexpr can't do), but trying to get it all to work is painful. Kudos to the author at making an attempt to explain it.
If you can give me specifics on how it's not clear, I'd very much want to improve it. Please DM me about it.
Not so much that you are not clear - this is an area that is weird enough that I don't think it is possible to be clear, but you gave me some useful hints. I got some of what I want working.
Edit, I started writing and then realized that I need this to work on old gcc that doesn't support these tricks so I have to give up. (I can't wait until this old embedded hardware dies)
What I'm trying to do is #define(A, B) static int A ## _ ## B = register(MKSTRING(A ## _ ## B) take any number of arguments. I can get the register working, but my attempts are creating the variable name fail.
Thanks. Also, per another person in the thread, here are the two annotations he or she was asking for, heavily annotated.
They are both only a couple lines, but it deals with things like the fact that you've got one more argument than you should have commas, and the use of the # and ## operators in these things.
https://c.godbolt.org/z/6zqx1dsn3
Thanks. I didn't have enough levels of indirection. Once I add enough levels of indirection it works. Well works on my newer systems, I have to build for an embedded system where I'm stuck on an old compiler that doesn't support these tricks.
To be clear, the example I provided for the other person explains the bit you're missing where the names aren't working... if you carefully follow the rules, the # and ## operators don't allow expansion on their arguments, so you have to use a layer of indirection to get them expanded first.
Imagine trying to implement the C preprocessor. I had to write it from scratch 3 times before it worked 100%.
Wow, you are a braver person than I. Well done.
Thank you. I am actually perversely proud of it.
This was back in 80s when you were working on c compilers? That’s an interesting story (80s compiler scene and what you worked on) I’ve picked up bits and pieces over the years, have you written it up anywhere? Would be benefit for many I think.
https://github.com/DigitalMars/dmpp
It's Boost licensed. Open sourced.
No need for anyone else to struggle to implement a preprocessor.
microsoft fixed their broken c preprocessor implementation just a few years ago
They could have just asked me :-)
They were deliberately leaving their C compiler broken because they considered C++ to be the replacement for C.
C++ uses an identical preprocessor.
That has sometimes been true, but for example C++ didn't have variadic macros, maybe still doesn't, because they were added to C after Microsoft decided to stop following C development. https://stackoverflow.com/a/21515839 goes into details of other differences.
> C has many advantages that have led to its longevity (60 years as perhaps the most important language).
53 years by my count. Did something relevant happen in 1960? Maybe author is alluding to B?
No, should have said '50'. Have fixed that, thanks.
Mildly related, sort of, one can prevent expansion of variadic macros as follows:
I keep on seeing many random code bases just resort to #undef instead...Doesn't this trigger warnings?
Function like macros literally requires name( , i.e name followed directly by open paren, otherwise no macro substitution occurs. so (name)() will always suppress function like macros (but not non-function ones, i.e regular #define name xxx)
Macro is one of the ugliest features available in langs like C/CPP
A C preprocessor implemented in Python: https://github.com/paulross/cpip
I used to write a preprocessor until I noticed those kind of thing...I stopped writing it after that
Can I use this technique to expand MACRO(a,b,c,…) into something like F(a,b,c…); G(a,b,c…)?
Okay, finally found some time to provide you with a fully annotated example of your original ask here, assuming you wanted to transform the arguments passed to F into IDs, and the arguments passed to G into strings (as seemed to be the case from the rest of the thread).
https://c.godbolt.org/z/6zqx1dsn3
I've fully annotated it, so it might seem like more than it is. About half the macro code is from the original article (the chunk at the top). And I do implement both transforms for you.
Each one I think is only 6 lines of code by itself, despite the rediculous amount of exposition in the comments.
If you have any questions about it, let me know.
That's just:
``` #define MACRO(...) F(__VA_ARGS__); G(__VA_ARGS__) ```
The technique in the article is more often used to type check the individual parameters, or wrap a function call around them individually, etc.
Ok. How about into F(a,”a”);F(b,”b”);etc.
The problem being automating enums and their names in one call. Like MACRO(a,b,c) and getting a map from a to “a”.
100%, that's definitely easy to do once you understand the technique.
Please?
I'm on my phone, but if you start with the top 8 lines in the code box under the ascii art, you'll get an implementation of `H4X0R_MAP()`; the bottom two lines are an example, and you can just write yourself a body that produces one term. Only thing you need to know beyond that is the stringify operator.
And I should say, if you want to apply the same transformation to arguments twice, but call F() separately from G() per your starting example, you'd just apply your map twice in your top-level macro.
> if you want to apply the same transformation to arguments twice, but call F() > separately from G() per your starting example, you'd just apply your map twice in your top-level macro
My brain just blew a fuse.
Is this a DoS risk - code that sends your build chain into an infinite loop?
From a DoS risk perspective there is no practical difference between an infinite loop, or a finite but arbitrarily large loop, which was always possible.
For example, this doesn't work:
That would only expand once and then stop because of the rule against repeated expansion. But nothing prevents you from unrolling the first few recursive expansions, e.g.: This will generate 2^4 = 16 copies of x. Add 60 more lines to generate 2^64 copies of x. While 2^64 is technically a finite number, for all practical purposes it might as well be infinite.To do this as efficiently as possible, it's probably worthwhile to use a higher radix and shorter macro names. For example:
Without any specific implementation of a constraint it certainly can happen, although I'm not totally sure that it's something to be concerned about in terms of a DOS as much as a nuisance when writing code with a bug in it; if you're including malicious code, there's probably much worse things it could do if it actually builds properly instead of just spinning indefinitely.
Rust's macros are recursive intentionally, and the compiler implements a recursion limit that IIRC defaults to 64, at which point it will error out and mention that you need to increase it with an attribute in the code if you need it to be higher. This isn't just for macros though, as I've seen it get triggered before with the compiler attempting to resolve deeply nested generics, so it seems plausible to me that C compilers might already have some sort of internal check for this. At the very least, C++ templates certainly can get pretty deeply nested, and given that the major C compilers are pretty closely related to their C++ counterparts, maybe this is something that exists in the shared part of the compiler logic.
C++ also has constexpr functions, which can be recursive.
All code can have bugs, error out and die.
There are lots of good reasons to run code at compile time, most commonly to generate code, especially tedious and error-prone code. If the language doesn't have good built-in facilities to do that, then people will write separate programs as part of the build, which adds system complexity, which is, in my experience, worse for C than for most other languages.
If a language can remove that build complexity, and the semantics are clear enough to the average programmer (For example, Nim's macro system which originally were highly appealing (and easy) to me as a compiler guy, until I saw how other people find even simple examples completely opaque-- worse than C macros.
D doesn't have macros, quite deliberately.
What it does have are two features:
1. compile time evaluation of functions - meaning you can write ordinary D code and execute it at compile time, including handling strings
2. a "mixin" statement that has a string as an argument, and the string is compiled as if it were D source code, and that code replaces the mixin statement, and is compiled as usual
Simple and easy.
No. Other modern languages have strong compile-time execution capabilities, including Zig, Rust and C++. And my understanding is that C is looking to move in that direction, though as with C++, macros will not go away.
The c pre processor. C doesn't have macros. It's fucking miserable. Anyone who uses it is a masochist
It's a feature... macros allows people to change the language at will which is great when you're researching programming languages (like in lisp) but less good when you want maintainable and consistent code
C++ has more powerful metaprogramming and look how that turned out
No, it is not a feature. It prevents C from being parseable and precludes actual metaprocessing. Just use lisp if you want to write maintainable code
Nobody wants to end up like C++ of course, you ain't wrong about that
At least you can implement things like vector in C++. I challenge anyone to write the same for C macros. (this is semi-serious - if you get a full generic STL for C that is usable that would prevent a large number of bugs. The implementation can be as bad as you want, so long as the user API is not too difficult. Though note that I'm not a C programmer and so if there are better ways someone else is doing this I'm not aware of it)
I wonder how you like my vector: https://godbolt.org/z/97YGrbP9s (note, experimental library, but I see no fundamental issue).
Oh, this is very good.
Some of the stuff you're doing in the library I've also been doing recently, and has been working well, like using the string of the type name to allow for run-time checking of, what I would called "mixed" data (your variadic type). I've also done the same basic thing as your option type in a way that's closer to your sum type than the maybe type.
But I'd had enough problems trying to get something like your vector actually working that I'd given up, but I think now I'll build something at some point over the holidays. I think as I'm coming up to speed on the history of changes to _Generic that's partially due to the attempts before I had a C23 compiler, but even then, your code there is impressive-- both clear and clever.
I also have enough stuff passed via pointer that the option type for me needed to handle pointers differently-- I basically just have run-time code that does the null check at run time when set. At that point, there doesn't really need to be an 'is_set' type flag.
By the way, the run-time type checking is one of the use cases where I really would like to have a tractable interface for hashing at compile time, preventing unbounded strcmps, and turning them into an integer compare (without having to manage caching such things at runtime).
Same thing would be for making it much easier to statically lay out fixed caches implemented via hashing, instead of inserting into them at start-up, etc.
that is amazing... I don't write C so I didn't dig too deep, but kudos for getting anything to work. Now you use need great documentation so we can figure out how to use it and what it supports without digging into the macros themselves. (tests would be good too, but maybe they are there and I didn't see them).
yes, unfortunately I do not have a lot of time... maybe I can some funding, but C work this is difficult.