> No memory management is a memory management scheme in 8cc. Memory regions allocated using malloc are never freed until the process terminates. That has greatly simplified the code and the APIs because 1) you can write code as if garbage collector were present, and 2) that design decision has eliminated use-after-free bugs entirely.
>DMD does memory allocation in a bit of a sneaky way. Since compilers are short-lived programs, and speed is of the essence, DMD just mallocs away, and never frees. This eliminates the scaffolding and complexity of figuring out who owns the memory and when it should be released. (It has the downside of consuming all the resources of your machine if the module being compiled is big enough.)
And Rob Landley wants to plug his fork of tcc into qemu code generator http://www.landley.net/qcc/ to do something that I do not completely understand but sounds awesome!
That's why he called it 8cc, on purpose. He works with his fellow Rob Pike 8c author at google, and Go just switched away from 8c to their new internal compiler written in Go.
And 8cc shares the similar KISS principles as 8c, by far simpler than the big and slow c compilers gcc, clang, CompCert.
It's not correct to say that I'm working with Rob Pike. I contributed code to Go project in my spare time but that was a hobby. I'm working for LLVM team at Google and not for Go team. And the naming is, although it may sound like Plan 9-ish, not related to any of the tools. I chose a random character to append before "cc".
Always good to see more relatively small C compilers, even if not in the extremely small size class.
That said, I get a feeling this one has many more files than necessary, as a lot of them only have a few functions in them that could be consolidated into one "utilities" file.
I believe this is mostly a matter of taste. I find this style very consistent. Each file deals with exactly one abstract data type, and all functions around it.
This is common in many languages. Java enforces one (public) class per file, which is also a common coding style in C++, Python, Ruby and PHP, even though those languages don't enforce that.
You also find this in more modern languages like OCaml, where it is recommended to have one "module" per file, and one "module" concentrates on one data type. (and maybe one or two helper types if needed, much like in-class type definitions in C++)
This may be unusual in C projects, but it does have its merits. I read the code and wonder "Oh, what exactly is that 'dict' stuff?". Then, I can blindly guess the file names "dict.h" and "dict.h". I see a full explaination of "dict" and nothing else. I then notice that it is a combination of "map" and "vector", and maybe that's all I need to know. If not, I dig into "map.c" and "vector.c".
Otherwise, I'd have to search, and finally find it in some unrelated file name like "utils", "utililities" or "mapvectorutils". Then I have to search for all dict-relevant code in that file, skipping through a bunch of unrelated code.
To get the "whole picture", I'd prefer to see a simple diagram (autogenerated by IDE, or as part of the docs) that shows which module depends on which other module, maybe somehow arranged into layers. The other strategy "group multiple modules into a file" may convey the highest level quite well, but utterly fails to explain the dependencies of the modules inside a single file.
Another reason why I prefer this style is modularity. Each file does only one thing (in this compiler a file is a compiler pass or defines a data structure) and exposes only a limited set of functions. Other functions are marked as "static" to get a file scope. The shorter a file, the narrower the scope. You of course need to achieve the right balance between number of files and size of each file. The current status feels the right balance for my taste.
Your text search skills or tools are lacking, then. In an IDE, "go to definition" solves this problem. In an editor, incremental text search over the entire project solves this problem (I normally use helm-git-grep in emacs).
I'm a fan of bigger and fewer files. Atomized bits of functionality are tedious to read, when you have to read lots of code to understand the gestalt.
Indeed, this is mostly a matter of taste. Both styles have their merits and downsides. For some projects, I also preferred bigger files up to single-file solutions, which also worked great.
> In an IDE, "go to definition" solves this problem. In an editor, incremental text search over the entire project solves this problem
When using the right tools, it doesn't matter how the code distributed across files. But I believe this is slightly out of scope for the question at hand.
Yes, it's terrible to have a myriad of files each of fifty sparse lines.
"Modern" languages like Java/C# tend to scatter code over many small files. I think it's a cultural thing: since IDEs do not have "split" mode that provides several views to different parts of the same file side-by-side (like Vim does), programmers tend to split code into files small enough to scroll fast and IDE handles switching between files. Plus the dogma "one class, one file" that limits file size even if it's not helpful.
I'm glad that functional programming is on the rise: it divides code units more naturally with modules of functionality, not a one-size-fits-all programming language structure like classes.
P.S. Also, composition of a big source code file conveys some "background" information you'd have to recover from a needlessly fragmented pile of small files.
> I'm glad that functional programming is on the rise: it divides code units more naturally with modules of functionality
I'm not sure this will change anything. For instance, OCaml's modules are usually a bunch of functions surrounding a single abstract type. This is very similar to classes in other languages.
On a complete side note, I was going to tell you that myriad is an adjective and should not be used in the construction, "a myriad of". But then I looked it up; apparently I was wrong, and so were the many other people I had heard that from. Thanks for teaching me.
Compilier design and programming language design are two very diffierent things.
Perhaps someone could even make the argument that a programming language is BEST designed by someone who doesn't know how to make a compiler?
The choices for a programming language should purely be based on the language, and on the user experience. In the design phase no thought will be given to compiler design (initially) and all of that will eventually be hiddden as implementaiton details.
Long compile times are bad user experience, and language design influences this.
Performance is also extremely important for language ux, and there are certainly constructs that will prevent your language from ever being efficient. Someone completely unfamiliar with compiler design could easily add these to the language on features that are simply not worth it, even in his or her own eyes.
Besides, have you ever looked back on something and said 'Gee, I wish I knew less about this when I started'
I understand your point. Perhaps I went a bit too gung-ho with my POV.
I'd note, though, that long compile times shouldn't be too dependant on language design, should it?
I feel that an argument could be made that if a programming language is well defined, then a well written compiler could be well thought out and designed around it.
Of course, I'm sure that there are counterpoints. I'm sure that there are languages for which it is very difficult to write a "fast" compiled.
> Besides, have you ever looked back on something and said 'Gee, I wish I knew less about this when I started'
Actually, I kind of have. Experts (for lack of a better term) sometimes make their solutions more complicated than they need to be, and this is partly because they have the knowledge to do that.
Experts (for lack of a better term) sometimes make their solutions more complicated than they need to be, and this is partly because they have the knowledge to do that.
I think that's caused by a slightly different effect - not considering essential and incidental complexity. It is true that for some programmers, the urge to put in as many things as they know tends to override the more basic goal of solving the problem in the most direct way.
C and C++'s system of macros by textual inclusion and approach to "compilation units" means that virtually all large C++ projects end up having multi-hour compile times.
It's not only compile time that matters: an overcomplicated language grammar/specification hinders development of language infrastructure (static analyses tools, linters, debuggers, parsers/code generators, IDE interaction, etc).
When you design something concrete, you have to make choices on lots of things that you don't really care at that time, but the choice may haunt you later.
I think your point is that too much knowledge may hinder your view. That's certainly something you should watch; but in general, I think risk of making bad decision from ignorance is greater.
I disagree. Programming is creating an efficient man->machine interface. All abstractions are leaky, the best programming languages balance control with convenience. There is a cost and benefit for every design decision. Disregarding implementation often imposes drastic hidden costs that only become apparent as programs grow in complexity and requirements.
Twitter had to change to java, reddit had to change from lisp to python(and still has performance issues constantly), facebook had to write a php compiler, minecraft had to write a C++ version... There's no free lunch.
If a language feature is sufficiently good I'll take a X-fold increase in compile time. There really is no win in placing compile time before ease of use.
Ah, yes, then it becomes a question of real tradeoffs, I guess. There are a lot of domains in which I will happily trade a reasonable amount of performance for nice language feature, though.
That's true. My favourite language is Ruby. Which is an exercise in how to frustrate a compiler writer (in a sign of deep seated masochism, I'm working on a ahead-of-time compiler for Ruby), and much of it certainly is a result of providing nice features.
But many of the worst offenders in such languages also have little to do with "nice features" and more to do with not thinking about compilation. E.g. one of the big issues with compiling Ruby is that classes can generally be re-opened at any time.
It's an awesome feature for a lot of things. But it also makes it incredibly hard to formally reason about Ruby code.
What does "1+1" mean? You can't know without knowing whether or not someone has overridden Fixnum#+. And that can happen anywhere. So even if you know that 1+1 is 2 now, after the next method call, you don't know any more unless you've determined exactly what the intervening method call was, and traced everything it might have called. Oh, and you need to take into account threading. Executing 1+1 in a tight loop, expecting it to mean the same on every iteration? Can't safely do that, as someone might be randomly redefining methods in another thread. Even if you determine that that can't happen now, if anyone ever calls eval() on data you can't statically determine (or its cousins such as "require"), you're back to square one.
Of course 99% of the time nobody has a sane reason to do that, and the 1% (or more like 0.001%) when it happens, it's almost always because someone is proving a point, and nobody would care about breaking the proving-a-point cases. Outlawing overriding methods of the basic system classes, or even "just" specifying a subset of basic methods that the implementations should treat as immutable would make optimising Ruby code immensely much easier with very little practical impact. (Ruby has the woefully underused "freeze" but that applies only to objects; a method-level "freeze" or "final" carefully applied to a few dozen rarely-if-ever methods would make a lot of difference; it'd even be fine to supply a way of overriding it, as at least then you'd be signalling clearly that you're about to do something that requires disabling certain optimisations).
Providing restricted versions of "eval" (so you can provide guarantees about what will still be the same after eval), and the same for "require" would also make a huge difference (e.g. if I had a way of loading new code and being sure that it only defined new classes rather than modifying existing classes, I would be able to aggressively inline a lot of things in situations where I could determine types statically; without it, I can still do inlining but needs to do a lot of work to verify when changes have occurred and potentially fall back on a much slower path, which seriously increase the complexity and reduces the benefit of inlining)
The strength of language designs coming from people who are experienced at writing compilers is that you rarely see those kind of situations, because the first thing a compiler writer would tend to want to do would be to declare a lot of corner cases like the above as "undefined behaviour" at a minimum, and proceed to implement optimisations that would utterly break it.
While Ruby is my favourite language, my favourite influence when it comes to writing compilers is Niklaus Wirth, who from his first language work on successors to ALGOL-60 has consistently been working to reduce the complexity of his languages with each iteration (he has usually removed more than he has added in each new language). One might very well believe Wirth is too much of a purist, but while his languages tended towards the minimal, his overall principle is worth considering: He only wanted to retain functionality in his languages with proven benefit and that he understood how to implement efficiently. A lot of languages could benefit from extensive pruning of functionality on that basis.
(Incidentally, for my Ruby compiler if/when I finally get to the stage when these things matters, I do intend to deviate from Ruby and declare a whole lot of things that people don't generally ever do but that should technically work as "undefined behaviour unless you specify the --generate-horribly-slow-but-compatible-code switch or something to that effect)
Thank you for the interesting read. I do get your point and I don't think I ever really was of another opinion. It's interesting to read a very concrete problem from a contemporary language.
Probably not. The people who design programming languages (at least, those who do it for more than hobby purposes) are almost certainly reasonable skilled and knowledgeable in compilers already.
This might be true for general purpose languages, but as a data point Django's templates suffer immensely by the implementers being unaware of how to write a fast compiler/interpreter.
Those who design programming languages for more than hobby purposes have often "done all the damage" before their language got popular. And many of them may understand compilers now, but as someone who enjoys writing compilers, swearing at language designers is a frequent occurrence.
> No memory management is a memory management scheme in 8cc. Memory regions allocated using malloc are never freed until the process terminates. That has greatly simplified the code and the APIs because 1) you can write code as if garbage collector were present, and 2) that design decision has eliminated use-after-free bugs entirely.