If you say so. I can't count the dollars I've made fixing other people's poorly written multithreaded code in my entire career, including in the last 3 years.
Thread support is [more or less] standardised in all major OS-es now, sure. Doesn't change the fact that it's an extremely bad fit for the human brain to think about parallelism.
Stuff like actors with a message inbox (Erlang/Elixir's preemptive green threads) or parallel iterators transparently multiplexing work on all CPU cores (Rust's `rayon` comes to mind) are much better abstractions for us the poor humans to think in terms with. Golang's goroutines are... okay. Far from amazing. Still a big improvement over multithreaded code as you pointed out though, I fully agree with that.
I might be projecting here and please accept my sincere apologies if so, but it seems to me you are a bit elitistic in your comment. Multithreaded programming is still one of the most problematic activities even for senior programmers, to this day. Multithreading bugs get written and fixed every day.
At this point I believe we should just move to hardware-enabled preemptive actors where message passing is optimised on the hardware level, and just end all parallelism disputes forever since they are an eternal distraction. (The overhead when utilising message passing today is of course not acceptable in no small amount of projects. Hence the hardware suggestion.)
> Doesn't change the fact that it's an extremely bad fit for the human brain to think about parallelism.
I'm curious why you feel this way? Certainly anything with multiple (more than one) thread is a lot harder than single threaded. Given.
But I think the human experience is rich with real world counterparts that can make multithreaded programming "natural" (not necessarily easy, there's a difference). Basically any process in real life you do that involves collaborating with multiple people is a collaborative multi-threaded process. When forced to grapple with multi-threaded solutions, I often ask myself, "if I had a room full of people working on this problem, what would have to be in place to make the operation flow smoothly?" For me, this anthropomorophisation of the process makes it a "good fit" for how my brain is used to solving lots of real world problems.
On the flip side, I haven't had a lot of luck finding real world experiences that I can model coroutines/async/etc on. So they may be ultimately easier, but if our rubrik for "fit for the human brain to think about" is what the wealth of human experience has evolved our brain to handle well, I'm less convinced.
But I like learning new things. Maybe I just haven't seen the lightbulb yet. Help me see your point of view?
> On the flip side, I haven't had a lot of luck finding real world experiences that I can model coroutines/async/etc on.
I feel we might be having too incompatible perceptions of the world but I'll try.
Every communication (human or otherwise) is exactly actors with message inboxes. We the humans can't react to 3 incoming conversations at the same time. Our brain queues up what it hears or reads and then responds serially, one by one (not necessarily in order of arrival). Basically an actor with a message inbox, like in Erlang/Elixir.
The way you assert multithreading mimics the real world to me is not convincing. One possible example in favour of your point of view I could think of is probably a table full of food and 20 people reaching and grabbing from it at the same time. But even then, two people cannot successfully get the same piece of meat. So not a true multithreading in the sense of N readers/writers competing for the same resource; it's more like a shared memory area where parallel writes are okay as long as all writers reference different parts of it.
Sure, a lot of stuff collides in real time without synchronisation out there, that much is true. But you likely noticed we the people don't cope with the Universe's chaos very well and rarely manage to grasp it properly. So is it really a good way to model deterministic and non-chaotic systems with programming? To me it's not.
I am curious why do you even think multithreading mimics the real world at all. Elaborate, if you are willing? (We might think of different definitions of multithreading in this instance.)
Have you ever seen those assembly lines where a bunch of people sit around the conveyer belt and each one grabs a product, makes an addition, and sets it back down? That's fairly close.
Each person on an assembly line operates independent of the rest, there’s no “coordination” as such. A person picks up an incoming object, transforms it and sets it down. For that person, for all practical purposes, the rest of the people working on conveyor belt don’t even exist. I suppose that (not having to coordinate) played a key role in the popularity of conveyor systems. It’s simple enough to debug, address bottlenecks etc.
The closest s/w equivalent I could think of is Unix pipes. When I do “cat foo.csv | sort | uniq -c”, cat/sort/unique aren’t coordinating with each other, each takes the content from stdin, transforms and place it in stdout.
That's what optimum parallelism is. They are collaborating to process N number as many items at the same time in the most optimum way. They also are coordinating to prevent someone from grabbing the same item or placing the same item in the same place. The conveyor belt itself is a ring buffer.
The assembly line principle can also be applied to the concept of threads. Workers collaborate by handing items off to one or more workers(threads) down the line. Thread pools are just micromanagers constantly reassigning the poor workers, but at least the make a lot of friens collaborating.
> "I'm curious why you feel this way? Certainly anything with multiple (more than one) thread is a lot harder than single threaded. Given."
Not the OP, but in my experience most devs are bad enough dealing with state and flow in single-threaded code already. Including myself. But I've at least accepted that and try to mitigate it by avoiding things that make it worse (like threads). Meanwhile, I watch people around me doing code by spaghetti (throwing it on a wall and seeing if it sticks). The better devs don't apply that to their code, but still end up applying it to their "reasoning" about a problem domain, especially a business one.
E.g.: "Screw it, if this variable that I need is null I'll just return a False in this function I'm writing" ... without ever thinking "why" this variable might be null and how that explanation impacts the problem they're solving. Nevermind the fact that they're fully OK with propagating such a broken state further into the program with their False response. Now some poor other dev has to figure out why this function is returning a false result. Odds are they'll do the same thing and now we've doubled the amount of bad states.
Adding in even more complexity in terms of being able to "predict" code state by adding threads is a nightmare. I would most certainly protest and heavily advise against anyone using threads unless it's really really the thing that's needed to solve the problem. Otherwise, it's just adding more a lot more complexity for very little benefit if any at all.
> Basically any process in real life you do that involves collaborating with multiple people is a collaborative multi-threaded process.
I would argue that the code equivalent of people collaborating in the real world is multiple processes with IPC (or even further afield, multiple nodes in a distributed system), not threads. You don’t share memory with other human beings. Imagine how productive you could be if you did!
for certain classes of problems, sure. But how about “embarrassingly parallel” jobs (e.g. weeding a garden) or even “idempotent OLTP” jobs (e.g. studying several subjects at once)?
(Technically these aren’t best expressed as SHM but rather as a tuplespace or greedy worker-pool abstraction, but they can be achieved starting off with “just” an SHM threading primitive, so they count, I think.)
This really depends on where you draw the boundaries. In a computer everything is "memory" of one kind or another, but that doesn't necessarily correspond to human memory. The analogy works if you think of the "actor" as merely the thread's context and dedicated working space (stack), with everything else being part of the surrounding environment. In the real world we don't have anything akin to process separation, except by convention. Even the body itself can be directly manipulated by others. The convention we use for communication between individuals looks more like a message-passing system, but physical manipulation of matter is more like operating on data in shared memory. There is no physical law preventing someone else from coming along and messing with an object I happen to be holding any more than the rules of a computer prevent one thread from messing with a data object held by another thread.
Collaborating humans have nothing like the problems that require memory barriers. We rarely have anything that behaves like shared memory - shared physical objects might come close, but a physical object is inherently de facto protected by a mutex.
Async, actor-like processes are everywhere in the real world - "something comes into my inbox, I do my piece of work on it, then send it to their inbox". Or "I'll send this off, do something else for now, and then go back to working on that when I get their reply".
When you write a sequential program, you are essentially specifying one order of instructions that will produce the correct output. With parallel program, you need to consider every possible orders and add the correct synchronization to prevent unwanted orders from happening. That's the difficult part.
As a programming model, threads don't really help you to achieve this. When there's a bug, your only option is pretty much to think very hard and try to figure out what happened. (Or to use gdb with scheduler locking, which is about as bad.)
There are more than one way to parallelize things as well. For example you could have and assembly line methodology where different threads are working on different stages at the same time. Using a common shared threadpool, as one stage slows down, it receives more workers to speed it up. Using producer/consumer queues to designate when data is being passed from one threads control to the other eliminates a lot of multi-threading bugs if this architecture is appropriate.
I think it's a way of thinking - I was an OS hack - Unix device drivers/etc early in my career, I could think about interrupt races/etc but it was hard, I took a detour into designing chips for a decade where EVERYTHING is parallel, by the time I got back to kernel work (Linux was around by then) I found that stuff that had been hard was now obvious.
In short if it's not a good fit for your brain, change your brain!
H/W components don't share state, which is why H/W parallelism easier. Where they do interact with each other, we have to either design fully async logic or worry about metastability. In contrast all kernels are basically all about sharing state. Even in user code and even when, a language such as Go provides channels, a lot of people seem to use shared state (and mutexes and locks).
I am curious as to how your h/w experience makes it easier hacking on the linux kernel....
There's lots of shared state, every flop is shared state, and mostly here I'm talking about synchronous parallelism in pipelines - yes we do have to deal with metastability too, and that's a whole other level of evil so that people avoid it where possible and spent lots of brainpower mitigating it (metastability is so evil you can't completely make it go away, just make it extremely unlikely).
Go is sort of like saying "we're not going to deal with parallelism directly, just throw everything into fifos to talk to other places" mostly you can't afford to waste gates like that in real hardware (there are places where it makes sense - I've built a lot of display controllers).
I think the big difference is the way you end up thinking about time and timing holes in code (dealing with interrupts), after building gates for a while i think you tend to see the timing holes more easily
Take decade to learn threads? Then expect other devs to do the same? I think that proves the point that threads aren't a good fit, even if we are technically capable of it.
No, it didn't take a decade, that was just how long I did it for (before it got boring) - I just mean that you have to learn to think about parallelism in a different way, we train hardware engineers to do this but not software ones
So it eventually got boring for you. So... threads are a bad mental model for our brains? :P
Not sure what you mean by your statement that hardware engineers are trained to think differently about it. I am not sure there's much overlap between HW and SW engineers in terms of parallelism modelling. Do you have a few examples?
No, chip making got boring - essentially it's a month a year of doing fun creative stuff and 11 months a year making sure that it makes timing and is perfect before you tape out. When you're coding on the other hand you can get something new working every day. I chose to go back to software.
However having said that I find myself designing silicon again, change is good for the brain I guess
Hardware people tend to think in terms of netlists and pipelining so they have to worry about parallism on every clock
I'm not one of the greatest programmers in the world, but I don't find C threads to be particularly hard to deal with in a robust way.
Threads in C are hard if you have a sloppy codebase with global variables everywhere and a general lack of structure and abstraction. Unfortunately, this describes many many C codebases.
I would argue that a moderately talented programmer can write code that does exactly what they think it is supposed to do (whether you call that "bug-free" is a question of semantics) if the have a language and type system (or equivalent) that allows them to express their requirements, and they don't step outside what they can express that way (i.e. they don't write code when they can't express what its semantics should be). Languages/compilers that can do that for multithreaded code are extremely niche.
I'd interpret it to stop using the pthreads model of the parallel coding, in general. Because the pthreads model exists and is used in many programming languages.
But stopping to use C is a good start (for whoever has the choice)!
Maybe the functional programming/managed code fans should step up to the plate and rewrite the operating systems, desktop environments, and hundreds of thousands of command line tools etc that have all been implemented in C or C++.
Not as toy, proof-of-concepts. Fully fledged replacements for all that stuff that can be used in our our daily work. Build distros of the stuff so that we don't have to pick and choose this stuff and replace the bits of our systems in a piecemeal fashion.
They could show us how it's done instead of talking endlessly about it in online forums. So let's face it, it's never going to happen.
I never implied you did, they are merely one of the two major major groups of programmers I see consistently deriding C based infrastructure that presumably enables their paychecks to a large extent.
I'm not defending C, I'm merely sneering at its noisy detractors who spend more time complaining about it than supplanting it.
One could argue as a counterpoint that the Rust community (and all the other language communities) have millions of person-hours (and lines of of C/C++ open source code) to reference as they RIIR, a luxury the C/C++ community didn't really have, much of what they've built since Linux appeared was developed from scratch. Outside of 386BSD, there wasn't a lot of unencumbered UNIX source code available to them.
Perhaps as the Rust community grows in size and momentum this will happen more. Right now it seems like a bit of a manpower problem.
I mean to replace the ones that we use today. Not the ones we used 30 years ago. General purpose operating systems. Perhaps with a distribution we can download for free and install on commodity computing hardware, and actually use today for our primary areas of work/interest.
Note the section 1.7 in your own link ("End of the Lisp machines"). The Lisp software ecosystem was never as rich, broad and varied as the C/C++ software ecosystem is today, before it died.
BTW, I only stumbled across your comment by accident, since you replied to the parent poster instead of me. Parenthesis mismatch?
I respectfully disagree. I have a embedded multi-threaded 'C' program running in over 11k+ retail stores in the USA right now. It's been handling multiple client requests to a Sqlite DB since 2007 without any issues. This product has made my company a lot of revenue. The secret to using threads is all in the design. Don't share resources between threads (I only had one shared resource for 50+ threads guarded by a semaphore).
That is a decent anecdote. Well, with respect, allow me to revise and qualify my read of @baggy_trough's comment:
It's perfectly fine to continue using an existing C codebase for a program, not exposed to the public internet, that's maintained by a focused group of maintainers.
But on the other side of this spectrum, for large exposed projects like OpenSSL, Chromium, or even Linux, C/C++ has become risky.
IMO nobody is arguing about the sometimes part. Many programmers out there can't use Erlang/Elixir or Rust, or any other tech that makes writing parallel code much more deterministic and safe. I am aware of that and I have a deep respect for the programmers in the trenches who fight to produce bug-less parallel code in C.
My argument at least is that, given the choice, there exist much more productive ways to make your employer money than to try and learn nuclear phys... I mean multithreaded programming. :-)
I'm not sure I follow your terse comment. Are you inferring that because skilled programmers have bugs in code with threads, that threads are bad? Isn't that a bit non-sequitur? Skilled programmers have bugs in thread with no threads! Should we therefore infer that if only they added threads, they would be bug free?
I am not arguing in favor (or against) threads. I am not contesting that they lead to more bugs or not. What I was contesting was the assertion was that "threads are not a natural fit." My point is that procedural programming (which nearly all programming is at least small level, minus languages like Prolog) is something human beings have been doing for thousands of years. Look at "recipe" for doing something, and you have procedural program. Look at any recipe for lots of people working together to do something, and you have a pool of cooperating procedures. It's something we get schooled in our whole lives.
I mean, many people have the right idea but for now we have no choice but to emulate the right parallelism approaches on top of the wrong hardware architecture. Maybe this will change one day.
As for FP... I remember a guy saying "imperative/OOP shows you how it does stuff, while FP shows you what is it doing". It was a very eye-opening statement for me.
I'm not saying you are wrong btw, there are multiple efforts put there to create truly parallell processors. But it's not until recently that the technique exist to make them good enough. Good times ahead I would say. I personally know someone part of such effort.
I clearly remember the feeling of just getting it as well. It was a very binary moment :)
Multithreaded programming is still one of the most problematic activities even for senior programmers
I agree wholeheartedly. FWIW I've found higher level abstractions built into the language itself, such as atoms and refs in Clojure, go a long way toward mitigating the problem once and for all. This gives you an actual harpoon to hunt Amdahl's* Great White Whale of Parallelism. Instead of a toothpick (C++) or a pocket knife (Java) to do it.
The trouble is we need to stop hunting whales if we're going to move the needle in a permanent and significant way...
> Multithreaded programming is still one of the most problematic activities even for senior programmers
My second favorite class in college was the elective for distributed computing (which I took instead of compilers, but I wish I'd taken both). It felt good to me and I came out into the job market ready to concurrent all the things.
By the time I'd worked with a disparate group of coworkers, I found out that 1) my enthusiasm was a rare thing, 2) that this was with good reason (many, many burnt fingers), and 3) that just because I love something doesn't mean that it should have a central role in a team project, even if I am the lead.
In a similar way that I 'got dumber' by learning to think like a user (one of the downsides of studying UX) I sort learned to reach for concurrency after I'd tried several other options.
... but here I am, years and years later, debugging other people's async/await code for them. Nobody likes to have to have someone else fix their mistake, but it goes a lot better when I start by telling them this shit is hard. Which it really is. Even when (or maybe especially when) I'm confident my stuff is right, I find a few boneheaded mistakes that I have expressly warned others to avoid in stuff I wrote uncomfortably recently.
I find Go's approach to be in the Goldilocks zone. Its not intrusive enough to slow me down (or guarantee correctness for that matter...) but its enough that I no longer need to save the multi-threading something I know needs to be multi-threaded for a second pass and very rarely have multi-threading issues or trouble debugging.
That being said, developers often have a bit of learning to get there, but that can greatly be accelerated by good team/mentor code review and discussion.
Goroutines are definitely an improvement, no argument from me.
It's just that there exist ways to shoot yourself in the foot quite easily.
Example: I'd find it much more intuitive if writing to a closed channel returned an error value and didn't issue a panic.
But I'm guessing that the Go programmers get used to the assumptions that must be made when working with the language so such things are likely quite fine with them and cause them no grief.
Such things bites everyone. But writing to closed channel is regarded as programming error. You potentially lose values. So logic should be watertight, which is hard with concurrency. Doing simplest approach to locking helps.
Better with panic than silent errors and flawed logic.
I agree that multithreading bugs (eg; race conditions, invalid access, etc.) are common. Thing is, a lot of problems emerge from poor programming practices.
Threads are definitely a valuable tool for performance-oriented work and things like eg; worker pool patterns aren't always a great fit and you end up with more overhead.
There's a lot more lower-hanging fruit to be picked before you have to ditch actors with message inboxes for classic multithreading in order to gain some more oomph. And it has been my experience so far that the overhead of the other parallel coding techniques is peanuts compared to the I/O bottlenecks of the 99% of the apps out there.
I know there exist a group of programmers that really don't have a choice and have to use `pthreads`. But if you do have the choice then IMO sticking to multithreading programming is very backwards and invites a lot of suffering in your future (or that of your future colleagues).
Just use actors with message inboxes. Measure, profile, tweak. If you exhausted all other possibilities and you absolutely positively can't upgrade your server ever then sure, ditch actors and write multithreading code.
Meh, I mean, shared mutable state, to me, is the ultimate culprit of threading complexity. If you don't have that, very often many of your problems go away.
Futures and Promises go a long way in making that better. That's not to say that you can always rely on them, but it is far easier to get them right than it is to get threading right with shared mutable state right.
Actors, Channels, Futures, promises, reactive programming, etc. They all have one thing in common. They kill off shared state in favor of message passing.
> Meh, I mean, shared mutable state, to me, is the ultimate culprit of threading complexity. If you don't have that, very often many of your problems go away.
Yep, exactly.
Everybody can screw up syncing and message passing as well, given enough lack of experience, or schedule pressure, or simply not getting it. But it's much harder to screw up that while conversely, it's extremely easy to screw up multithreaded code with shared state.
"I can't count the dollars I've made fixing other people's poorly written multithreaded code in my entire career, including in the last 3 years."
"Multithreaded programming is still one of the most problematic activities even for senior programmers, to this day. Multithreading bugs get written and fixed every day."
Strongly disagree. I am going to claim that threads / locking are not hard to write if one has at least a bit of common sense and discipline. Problem is that we have hobbyists without any trace of knowledge on how computers work producing commercial software.
Goes like this: I do not want to deal with memory management - it is too hard, I do not want to think about types - my poor brain can not comprehend those, pointers - OMG what are those beasts, threads and locking - gonna commit Seppuku. I will advise those to come to a logical conclusion: programming is frigging hard when one wants to produce a decent product rather then buggy POS. No matter how many concepts the language hides / converts to something else there will be always something else they will not be able to wrap their mind around. So how about trying gardening instead
A lot of things are possible for us the humans. But some your brain gets easier and works better with while others are a struggle even if it gets them right most of the time.
You don't have to make this about implying that others don't have "at least a bit of common sense and discipline".
I'll go the other way around. Some people are capable of comprehending an awful lot of things/concepts and use those productively. And some are not. It is definitely not the concept's inventors fault. Nothing to be ashamed of as we are all different. I am not shedding tears because I can not fathom quantum physics while others can.
If you say so. I can't count the dollars I've made fixing other people's poorly written multithreaded code in my entire career, including in the last 3 years.
Thread support is [more or less] standardised in all major OS-es now, sure. Doesn't change the fact that it's an extremely bad fit for the human brain to think about parallelism.
Stuff like actors with a message inbox (Erlang/Elixir's preemptive green threads) or parallel iterators transparently multiplexing work on all CPU cores (Rust's `rayon` comes to mind) are much better abstractions for us the poor humans to think in terms with. Golang's goroutines are... okay. Far from amazing. Still a big improvement over multithreaded code as you pointed out though, I fully agree with that.
I might be projecting here and please accept my sincere apologies if so, but it seems to me you are a bit elitistic in your comment. Multithreaded programming is still one of the most problematic activities even for senior programmers, to this day. Multithreading bugs get written and fixed every day.
At this point I believe we should just move to hardware-enabled preemptive actors where message passing is optimised on the hardware level, and just end all parallelism disputes forever since they are an eternal distraction. (The overhead when utilising message passing today is of course not acceptable in no small amount of projects. Hence the hardware suggestion.)