Pimpl greatly reduce build times by removing includes from the interface header at the expense of requiring pointer dereferencing, and it's a tried and true technique.
Another old school technique which still has some impact in build times is to stash build artifacts in RAM drives.
It strikes me as something of a language flaw that without pimpl any addition/removal/ modification of private member functions, or even renaming of private member variables triggers full recompilation of every source file that needs to use a particular class. I understand that changes to the size of the object should do so (even if it's just new private member variables), but if it can be determined by the rules of the language that a particular change to a class definition can't possibly affect other compilation units that happen to use that class, then why can't the compiler automatically determine no recompilation is necessary e.g. by calculating and storing a hash of everything about a class that can potentially have an external effect and comparing that value rather than relying on timestamps...
> strikes me as something of a language flaw that without pimpl any addition/removal/ modification of private member functions, or even renaming of private member variables triggers full recompilation of every source file that needs to use a particular class.
It is not a language flaw. C++ requires types to be complete when defining them because it needs to have access to their internal structure and layout to be in a position to apply all the optimizations that C++ is renowned for. Knowing this, at most it's a design tradeoff, and one where C++ came out winning.
For the rare cases where these irrelevant details are relevant, C++ also offers workarounds. The pimpl family of techniques is one of them, and type erasure techniques are also useful, and protocol classes with final implementations are clearly another one. Nevertheless, the fact that these techniques are far from being widely used demonstrates that there is zero need to implement them at the core language level.
> It is not a language flaw. C++ requires types to be complete when defining them because it needs to have access to their internal structure and layout to be in a position to apply all the optimizations that C++ is renowned for. Knowing this, at most it's a design tradeoff, and one where C++ came out winning.
This statement is incorrect. "Definition resolution" (my made up term for FE Stuff(TM) (not what I work on)) happens during the frontend compilation phase. Optimization is a backend phase, and we don't use source level info on type layout there. The FE does all that layout work and gives the BE an IR which uses explicit offsets.
C++ doesn't allow two phase lookup (at least originally); that's why definitions must precede uses.
There isn't a good reason why private methods should be exposed in the header. This makes refactoring the class implementations much more of a pain in the butt than plain structs + global functions. You can always add internal functions to the implementation without causing a reimplementation of all dependencies, while you can't do the same with private methods.
And private methods aren't exactly "rare cases". The situation is bad to the point that most good codebases make less use of classes, and many average code bases avoid adding private methods and resort to code duplication to a degree.
> There isn't a good reason why private methods should be exposed in the header. This makes refactoring the class implementations much more of a pain in the butt than plain structs + global functions.
Irrelevant. Private member functions aren't mandatory or required, and when developers decide to use them they explicitly state the class needs to export their symbols.
Those developers who somehow feel strongly about private member functions are a multitude of techniques to meet their needs, such as using protocol classes and move private stuff to concrete implementations, or use non-member functions either with internal linkage or stashed in anonymous namespaces.
I don't see the point of complaining that something used wrong is not being used right, while purposely ignoring the myriad of options where things are right based on your arbitrary requirements. I mean, this aspect of C++ is around for at least three decades. Don't you think that if it was a problem someone would already have noticed it?
If you use "class" / private members, you have to use methods to access the members. That, or a friend "class" containing static methods (a namespace won't do). Private methods are annoying because you're forced to expose implementation details, and are forced to duplicate function signatures in the code. Friend classes are annoying because you still have to name them in the class declaration, and having to use them with static methods leads to stylistically inconsistent code and you end up with a weird dummy class that isn't meant to ever be instanced.
AFAIK there isn't a nice way to deal with this other than simply not using private members and coding in a simple C like style. I don't think you've shown a way, either. I don't know what you mean by "protocol classes", but if you mean abstract classes with virtual methods that need to be overridden, those are a bad solution because they overhead of vtables without any technical need or benefits (unless you want runtime polymorphism and vtables are exactly the kind you want).
Private methods do need to be in some sort of privileged "single source of truth" for the class definition though - if you were permitted to have a partial class definition in the "public" header file that defined public/protected methods plus all variables, then a single other partial class definition in a separate header or compilation unit it would go some easy towards solving the issue.
But I still don't quite see why compilers can't make assumptions about what changes to class definitions can possibly require recompilation of other compilation units (even if it only applies at certain optimisation levels).
Private methods, unless they are virtual, do not to my knowledge have any bearing on the ABI of a class, nor need they be known to users of the class, so listing them in the class declaration seems nonsensical.
The reason why they have to be listed anyway could be 1) a vague idea of "consistency" with e.g. public methods and generally the enforcing access control only centrally from the class declaration 2) the idea of overriding the implementation in an inherited class. As far as I'm concerned, both are bad reasons to impose such an annoying limitation to the user of the language.
The whole point of private methods is that they have privileged access to the implementation details of the class (i.e. private member variables and other private methods). So there still needs to be a restricted place they can be defined - what are you proposing?
Mark the function as part of the implementation of the class. Simply prefixing the name with the name of the class (as it's already done) could be enough. Optionally request a keyword.
I don't see what's the big deal, the saying is to guard against Murphy not Machiavelli.
Or go the same route as namespaces -- mark start and end of the class implementation code (can be repeated) and nest functions inside.
There are other options if we ditch the C/C++ compilation model. Though arguably that isn't just bad -- it's an extremely simple way to achieve separate compilation without requiring a separate (probably binary, compiler-specific) representation for compiled interfaces. The latter could probably speed up incremental builds considerably, but it's possibly slower for clean rebuilds because of dependencies.
So you're saying it's perfectly fine for someone to use any class from any library (including the standard library) and define their own "implementation" private methods for such classes, that then access implementation details of that class?
How is that even protection against Murphy? (who sees other examples of such things and assumes that's just the "done thing").
For me, knowing that I can change the implementation details of a class and it having no possible impact on whether other code compiles or how it behaves (assuming I maintain the same "public" behaviour) is absolutely a fundamental language feature. All I'm arguing for is that compilers should be able to make the same assumption - only the private implementation details have changed, so it's unnecessary to recompile other code that happens to include the header file defining some of those implementation details.
If you're worried about these things (I am not), simply restrict the private method to be called only from other class methods. Wait, that's how private methods work already...
Ok I admit to feeling a bit stupid now - you're absolutely right of course. On that basis I don't see why it should be necessary to declare private method signatures inside the class definition at all. I can't really see any reason it shouldn't simply be allowed to define (non-virtual) private class methods anywhere you like - as you say, if someone else other than the original class-author does so, they wouldn't be able to call the function anyway!
So what is a good justification for the current language design? I did find one SO post suggesting if your suggestion were possible, the overloading rules would probably have to change, but that doesn't seem like an insurmountable hurdle.
To be clear, what you're suggesting is that header file (foo.h/hxx/hpp) would have:
class Foo {
public:
Foo();
void doYourThing();
private:
int _privInt;
std::string _privStr;
}
Whether or not some sort of keyword is needed to mark the private functions as such is stylistic I suppose - I'd prefer it were there, but I'm used to C# where the access specifier is part of every member declaration anyway. But it's certainly not necessary - the compiler would just assume "private" if the declaration is not part of the class definition.
And yes, someone else could come along and put
void Foo::anotherPrivateFunction() {
}
in their own code, but they'd never be able to call it anyway, so no harm done (arguably compiler could treat an "unreferenced" private function as an error in itself, but certainly the linker would just strip it out).
Obviously one downside of the above is that if you wanted friend classes to be able to call such private methods, they'd either have to forward declare them, or you'd put them into a separate "foo_private" header file, but again, I'm not seeing why that's a big problem.
If your internal representation is stable, you can put private functions in a private friend class that is only defined in the cpp file: `private struct access; friend struct access;`.
Interesting idea, can't say I've used that pattern used, and obviously the code will end up slightly more verbose (no automatic "this" argument) but in principle could be better than the pimpl technique.
The primary reason for Pimpl is to ensure binary compatibility for C++. QT uses it extensively precisely for this reason. Reduced compilation time is just a nice bonus.
I'll be glad if I never have to see PIMPL ever again. It makes tracking down the actual implementation of some functionality so much harder than it has to be.
> I'll be glad if I never have to see PIMPL ever again. It makes tracking down the actual implementation of some functionality so much harder than it has to be.
Not really. There are two main flavours of pimpls: one where the impl class is a pod that only holds member variables, and one where the impl class holds both member variables and private member functions.
On both, the implementation can and does stay in the very same translation unit. On the former, the code stays in the very same class,without any change.
You only experience the sort problems you describe if you bring them upon yourself. That's hardly the idiom's fault.
I’m curious: do you use a `const std::unique_ptr<Impl>` or just a `std::unique_ptr<T>` or do you have a template that provides value semantics for the `Impl`? If I used PImpl a lot I’d make a `value<T>` that encapsulates ownership with value semantics.
And conversely, if you are using classical polymorphism, you can get essentially the effect of PImpl by having an abstract base class with a static factory function that returns a unique pointer, then implement that factory function by in the cpp file having a concrete class that is defined there and returned as a `std::unique_ptr<IBase>`. That gives you separation of API from implementation without a memory indirection, but you then can’t have it as a value type.
> if you are using classical polymorphism, you can get essentially the effect of PImpl (...)
No, not really. That's only an option for the rare cases where you control the whole class and all it's members, and you can break any ABI whenever you like by converting any class to a protocol class.
In the real world, pimpls are applied to member variables that were encapsulated in the past but you want to remove their definition from the interface, or classes that are generated with code generators at compile time. It makes little sense to replace a class with a protocol+implementation+factory just because you want to get rid of a member variable or you need to consume a auto-generated class.
I feel you're just adding noise to an otherwise interesting discussion.
If all you have to add is handwave over "Pimpl has so many other drawbacks" and to reply "google them" when asked to substantiate your baseless claim, I feel it was preferable if you sat this discussion out.
The noise you add is particularly silly as all your references point is the pointer dereferencing drawback I already pointed out and you claimed there were more.
> all your references point is the pointer dereferencing drawback I already pointed out
I think maybe you didn't read through both the items I linked or the many others that come from a simple google query. One of my links for example points out the memory fragmentation issues, which can also affect performance, as another commenter here has also pointed out. There's more to the story than pointer de-referencing or memory context -- many drawbacks worth knowing about.
There is nothing baseless here; there are pros and cons. But it's not in good form to ask people for details that are easily looked up on a topic as well-known as this one. We are not a reference source for you.
Heap fragmentation is another, pimpl works but it's really papering over limitations in the pre-processor and header model that leaks details to downstream consumers.
For the rare cases where potential pimpl users care about heap allocations, there's a pimpl variant called fast pimpl with replaces the heap allocation with a opaque member variable that holds memory to initialize the impl object. Since C++11 the opaque object can be declared with std::aligned_storage.
> If you don't care about heap allocations then why are you using C++ :)
Because more often than not it is a made-up problem that at best is classified as premature optimization.
Take, for example, Qt and how it uses it's UI form classes. You can include it's autogenerated code by composition, inheritance, and through a pimpl. You are here complaining that everyone should care about heap allocations. Well, in this case they happen only at app start and behind a standard C++ smart pointer, and it's basically only used to instantiate a factory to create UI elements.
Can you tell me exactly any concrete reason why anyone should care about instantiating a few bytes in the heap with a smart pointer at app start?
Sometimes there are real problems,but sometimes there are made-up problems that have no relevance or meaning.
I've had to deal with fragmentation at some point or another across most titles I've shipped or larger programs. Small allocations are actually worse in some ways because they incur a lot of fine-grained fragmentation. My favorite was logic in a fairly well-known game engine that would allocate a generic name for an entity and then immediately rename it to something unique. That 5 byte allocation would be left behind multiplied by every entity cost us ~30mb of a 256mb fixed memory address space.
If you don't deal in those sorts of domains that's fine but don't dismiss them as those are some of the primary use cases for native languages. Pimpl is a hack, it's a clever hack in the context of C++ but it's not something intrinsic to native development. For instance it's a non-concern for languages like Rust(which just handle this whole domain much better due to not having to inherit a bunch of legacy behavior).
There's a bunch of feedback in adjacent threads that you seem to be ignoring so I suspect I won't change your mind here but I would encourage you to be a bit more open and consider that there are use cases where things like this matter rather than dismissing them offhand.
> Exactly this. Pimpl idiom is just a bandaid and typically goes against the main benefits of the language.
No, not really. The pimpl idiom is a basic technique that is employed to "improve the software development lifecycle", which is a description straight from Microsoft.
Even Microsoft's Herb Sutter himself has quite a few articles on the virtues of pimpls and how to implement them.
If you think they are wrong, are you planning on reaching out to them to correct them and to try to educate them on the matter? That would be something.
Herb Sutter's main discussions on this are nearly 15 years old lol. Like I said above, pimpl went out of style maybe a decade ago due to advancements in language design, learning about its flaws, etc.
This is a meaningless statement. As others in this discussion already stated repeatedly, pimpl is a very basic and fundamental C++ idiom that is pervasive in all application, and some frameworks are notorious for using them extensively even right now, as is the case of Qt.
At most, all you can claim is that you developed an irrational dislike of pimpls, but developers are renowned for making poor decisions and even introducing bugs, so it's worth what it's worth. I mean, some developers even in this day and age still complain about the whole notion of design patterns. Are these blends of opinions worth taking into consideration?
You asked "which ones?", I answered with some drawbacks to note, mentioned there are pros and cons, answered your question (with links you didn't fully read).. and you proceeded to go on a defensive tirade accusing me of noise and baseless claims (while you reference deprecated language features), etc. Have a nice day.
What leads you to believe that protocol class + concrete implementations + factory can replace pimpls?
I'm going to make it simple for you: take a Qt widget. You define its widget layout with Qt a designer, save a UI form file, and get Qt's UIC to generate a header file that implements that widget tree. You have three options to include the object in that header: either inherit from it, composition, and pimpl. So you swore off a pimpl. Pray tell, how do you use a Pure virtual interface in this case?
> What leads you to believe that protocol class + concrete implementations + factory can replace pimpls?
What leads you to believe they can’t?
> I'm going to make it simple for you: take a Qt widget. You define its widget layout with Qt a designer, save a UI form file, and get Qt's UIC to generate a header file that implements that widget tree. You have three options to include the object in that header: either inherit from it, composition, and pimpl. So you swore off a pimpl. Pray tell, how do you use a Pure virtual interface in this case?
Never worked with Qt. Can you give me FooBar example where pimpl is superior to pure abstract class, please?
https://en.cppreference.com/w/cpp/language/pimpl
Pimpl greatly reduce build times by removing includes from the interface header at the expense of requiring pointer dereferencing, and it's a tried and true technique.
Another old school technique which still has some impact in build times is to stash build artifacts in RAM drives.