I think that’s an oversimplification. You can’t take the “free labor of performing democracy” and put it to equally good use doing anything else I can think of.
I guess you could work in soup kitchens, but that’s horribly inefficient welfare compared to just electing competent leadership, if the ultimate aim is to benefit The People.
I’ve been there. It’s crazy how the smell of spicy hotpot permeates every street.
And it’s also crazy the spice levels they’re used to. It’s the only place in China I’ve seen 微微辣 (“very very little spicy”). And it was still incredibly spicy!
> Despite being clean and readable, I don't tend to do it any more, because it's harder to debug. More often these days, I write things like this:
data = File.readlines("haystack.txt")
data = data.map(&:strip)
data = data.grep(/needle/)
data = data.map { |i| i.gsub('foo', 'bar') }
data = data.map { |i| File.readlines(i).count }
Hard disagree. It's less readable, the intend is unclear (where does it end?), and the variables are rewritten on every step and everything is named "data" (and please don't call them data_1, data_2, ...) so now you have to run a debugger to figure out what even is going on, rather than just... reading the code.
The person you are quoting already conceded that is less readable, but that the ability to set a breakpoint easily (without having to stop the process and modify the code) is more important.
I myself agree, and find myself doing that too, especially in frontend code that executes in a browser. Debuggability is much more important than marginally-better readability, for production code.
> Debuggability is much more important than marginally-better readability, for production code.
I find this take surprising. I guess it depends on how much weight you give to "marginally-better", but IMHO readability is the single most important factor when it comes to writing code in most code-bases. You write code once, it may need to be debugged (by yourself or others) on rare occasions. However anytime anyone needs to understand the code (to update it, debug it, or just make changes in adjacent code) they will have to read it. In a shared code-base your code will be read many more times than it will be updated/debugged.
But, while reading is more common than debugging, debugging a production app is often more important. I guess I am mostly thinking about web apps, because that is the area where I have mainly found the available debuggers lacking. Although they are getting better, I believe, I've frequently seen problems where they can't debug into some standard language feature because it's implemented in C++ native code, or they just don't expose the implicit temporary variables in a useful way.
(I also often see similar-ish problems in languages where the debuggers just aren't that advanced, due to lack of popularity, or whatever.)
Particularly with web apps, though, we often want to attach to the current production app for initial debugging instead of modifying the app and running it locally, usually because somebody has reported a bug that happens in production (but how to reproduce it locally is not yet clear).
Alternatively stated, I guess, I believe readability is important, and maybe the "second most important thing", but nevertheless we should not prefer fancy/elegant code that feels nice to us to write and read, but makes debugging more difficult (with the prevailing debuggers) in any significant way.
In an ideal world, a difference like the above wouldn't be harder to debug, in which case I would also prefer the first version.
(And probably in the real world, the problems would be with async functions less conducive to the pithy hypothetical example. I'm a stalwart opponent of libraries like RxJs for the sole reason that you pay back with interest all of the gains you realized during development, the first time you have to debug something weird.)
It comes across borderline comical when a man gets asked for his sources while he is clearly stating his formed opinion based on his impressions.
It is hard to think of an example without sounding like I am exaggerating. Imagine if you shared on a thread that Rust has a great ecosystem but it is a little bit too overhyped and someone so cleverly asked for you to cite sources.
Do you really rely on academic studies to form any impression of anything? Is your chain of thought full of citations?
I have this screw I want to undo but Schonenberg et. al. has demonstrated that a Philips screw should be used for this situation. Unable to find any citations on the feasibility of a blunt knife in this situation. Further research needed.
There was a thread on HN semi-recently where I observed that if you think about classifying complexities ("big-O of n squared") by polynomial degree (so we think of O(n^2) as "2"), the logarithm function gives you a literally infinitesimal value.
I was then asked about sources for what amounts to an easy homework problem.
My impression it's the us-american weird mix of "think about the kids" and "purism" culture, its fine to give your 12y a gun, but got forbidden it hear a swear word
> But AI has emboldened the North Korean scheme, allowing the IT workers to develop scripts so they can hold down as many as six or seven jobs at a time
How do you get interviewed, hired and payrolled by seven jobs, at once? Are North Koreans in fact 10x developers?
These are two different paradigms. aiopandas is not trying to offload pandas work somewhere else to prevent it from blocking synchronous code, it's trying to let you apply asynchronous functions to pandas operations concurrently while running on the event loop inside of other async code.
That said, this is mostly just going to be helpful if you're running pandas operations that call an external API on each iteration or something, and the actual pandas part of the work is still going to be CPU-bound and block. I am also not a huge fan of the monkey-patching approach. But it's clever and will definitely be useful to folks doing a very specific kind of work
indeed... the longer i write python, the more i just try to solve stuff with a simple ThreadPoolExecutor.
I think doing this is not the best choice for cpu-bound work, which is likely what you're running into with pandas, but nevertheless... I like how you can almost always slap a threadpool onto something and speed things up, with minimal cognitive overhead.
The intended use-case for this is actually very different from what you describe, and one where aiopandas would be much faster than a ThreadPoolExecutor.
Lets say that you have a pandas dataframe and you want to use `pandas.map` to run a function on every element of it where, for some reason, the new value is determined by making an API request with the current value. No matter whether you do this in the main thread or in a threadpool, it's going to run these one at a time, and very slowly. You can make X number of requests at once inside a thread pool where X is the number of workers you set, but this number is not usually very high, and running http requests asynchronously is going to absolutely wipe the floor with your thread pool. You can run hundreds to thousands of concurrent http requests per second on asyncio.
So yes, the actual work that pandas has to do in terms of inserting/modifying the dataframe, that's all CPU-bound, and it's going to block. But 95%+ of the wait time you'd experience doing this synchronously would be just waiting for those http requests to finish. The pandas work is CPU-bound, but each iteration would probably be measured in milliseconds. In this use-case, this library (assuming it works as described) would be far superior, by many multiples if not an order of magnitude.
That said, I have absolutely no idea who is making http requests on each iteration of a pandas map, or what percentage of that group of people didn't solve it some other way.
As a very simple example, here's aiohttp making 10,000 http requests (HEAD requests to a list of different urls) in a single thread but asynchronously vs. ThreadPoolExecutor making them synchronously but across 32 workers (I had to drastically reduce the number of urls in order to make sitting through it bearable): https://asciinema.org/a/MkoOVQBSeBanRRZtsu3xe5FUk
> not the best choice for cpu-bound work, which is likely what you're running into with pandas
I'm not a Python user, why is it not good for cpu-bound work? I see the defaults assume some I/O work, but with `max_workers=~cpu_count` it should be what typical dispatchers for CPU-bound work do in other languages
Python "threads" aren't real threads in the traditional sense because Python's Global Interpreter Lock (GIL) exists, and this means no more than one thread is ever actually running in parallel. They are great for network IO since most IO is just spent waiting for stuff rather than computing anything, but you can't actually run CPU-heavy stuff on multiple Python threads and have the speed multiplier be equal to the number of thread workers. For this, you have to use process pools. (Though this is something that is in the process of finally being alleviated/fixed!)
This seems all a bit misleading to beginners, if you have numerical cpu-bound work in Python what you should be doing is vectorize it, not parallelize.
Would Musk/Tesla theoretically have money enough to just eat the losses themselves for the first 6 months or so, and then secretly jack up the prices with a dumb excuse attached, when Trumpists have forgotten about the possibly that tariffs could backfire?
Tesla isn't a hugely profitable company. I suspect what will happen here is they'll get a waiver on the import duties if they promise to move the production to the US over some number of years.
I guess you could work in soup kitchens, but that’s horribly inefficient welfare compared to just electing competent leadership, if the ultimate aim is to benefit The People.
reply