I agree: it is OK for your open source library to be a bit shitty. However, it also can simultaneously be not OK for you to publish it in a package directory with a generic name and a description talking about how awesome it is as a trap for other people to run into, and that is really the core problem: it isn't that you didn't spend an unreasonable amount of time and money to make a good product that no one paid for (though I do know--all too well, as the sole maintainer of some key projects for a sometimes ungrateful ecosystem--that there are many people who really are just looking to be assholes), it is more often that you overhyped a side project and then put it somewhere obvious for other people to find it, only for them to discover that it was "a bit shitty" long after they deployed it; and sure: it is totally their fault for deploying something without carefully reading your code to figure out that it was a bit shitty beforehand... but should they really have to do that? When you release code that you don't intend to back up with a ton of time and resources, please do the rest of us a favor and use a name that no one is going to mistake for a standard package, put a note on it that says "this is a side project" so expectations are clear, don't write a readme file in the past tense about features you hope to one day implement, or, if you just can't bring yourself to be responsible like this, maybe just reconsider whether you need to publish it at all, as it actually doesn't necessarily make the world a better place just because it is open source. Seriously: if you bring something that you know tastes bad or will make someone sick to the potluck and put a little sign on it saying "gourmet enchiladas: much better than what you get at the store" and put it in the center of the table, it is still a problem even though "it is ok for you to not really know how to cook or not have that much time and participate in the potluck anyway".
"sure: it is totally their fault for deploying something without carefully reading your code to figure out that it was a bit shitty beforehand"
Yes you must read and understand everything or have trust in the publisher. Relying on the package name is going to get you in trouble. Sounds like a great exploit opportunity.
Turns out left-pad.js is on line 8571 of that 14+k line yarn.lock file.
How many React Native developers do you suppose have reviewed every line of code in every library referenced in that file? (Or even how many engineering teams at any React-using Unicorn size startups or FAANG have exhaustively reviewed all those lines of code?)
Is FaceBook a "trusted publisher" of React? Is npm a "trusted publisher" of Node libraries? Was Azer Koçulu a "trusted publisher" of left-pad.js?
(And it's turtles all the way down. Is Microsoft a "trusted publisher" of Windows? Is Dell a "trusted publisher" of bios? Is Intel a "trusted publisher" of cpu micro-code? Is Western Digital a "trusted publisher" of hard drive firmware? Is every unnamed silicon manufacturer and pcb and subcomponent assembly house a "trusted publisher" of the embedded code in their network card/usb hub/battery management chip/whatever?)
I _do_ agree with you, but these days it's very difficult, at least in much/most of the software world, to determine the height of the people/libraries claiming to be "giants" offering their shoulders to stand on. (And like with React, it's more and more difficult to determine which actual giants are standing on bedrock, and which have a foothold with quicksand uderneath it...)
In react's case you put your trust in facebook and because of the size most issues get discovered and someone has an article on how to fix before breakfast.
When you include left-pad in your own project you should audit it. In the end you are the only line of defense. With react it's larger so you can trust others.
I am the sole maintainer of airbnb/lottie-android despite the fact that I left the company almost 2 years ago and it is used by tens of thousands of companies including Google, Samsung, Microsoft, Uber, Netflix, and countless others.
Let's say this doesn't get updated I don't see a problem. It will continue to function as it is until it gets outdated and replaceed. Let's say it changes and does something evil, hopefully Google, Microsoft will catch the issue. If they don't someone will and people will stop trusting react or whatever product pulls these changes in.
Doing it yourself I would just freeze versions.
---------
In this case I find it odd for someone to keep maintanier status for a company repo they no longer work for 2 years after leaving.
I would have forked and introduced new changes in my own repo if I cared about the project. Let others follow if they desire new updates or force airbrb to assign someone to update. or better get airbrb to pay you to keep updating.
Lets just say someone at booking.com or tripping.com or some other hungry and less-that-totally-ethical AirBNB competitor works this out, and offers to buy the library or pay Zac $bignum in return for adding in some "additional analytics" aka industrial espionage allowing them to see searches/bookings made by users of apps with the library embedded.
Now Zac probably isn't the sort of person who'd do that, but once you get to a project like React which has 10,000 or so dependancies - surely _some_ of those dependancy authors would do that, for enough pieces of silver...
What if I have no reasonable way of fully understanding the code nor being able to judge the author on their trustworthiness (even disregarding that this isn't constant).
As a community we need to accept that the above mentioned DOES happen. Pretending otherwise only hurts us all.
> As a community we need to accept that the above mentioned DOES happen. Pretending otherwise only hurts us all.
In fact, it happens very often when people are learning and exploring. So imagine I publish my shitty library "with a generic name and a description talking about how awesome it is". Imagine you're trying to learn $language. You buy into my deception and try to use my library. You get burned, and might be left with an impression that $language or its ecosystem is shitty, and never return, despite the fact that $language would suit the kinds of problems you're solving really well. If it happens enough times with enough libraries, $language may get a bad reputation. Preventing this becomes a matter of protecting the common good.
There are many things short of reading the code in full you can do to lower the chances of this happening. A lot of half-assed libraries have obviously incomplete APIs that you can notice if you think about the interface provided and the nature of what the library claims to be doing. You can also look at repo activity; was it all coded in one week 8 years ago and never touched again? Does it have lots of outstanding issues?
Some things may slip through the cracks, but filtering out most of the chaff can be done without reading every line of source.
This. In my experience, GitHub issues and PRs tell you most everything you need to know about an open source library: if issues are responded to (and how quickly), the attitude of the maintainer with regards to new features, etc. Most of this can be gleamed within 20 minutes of browsing, and when you combine that with a quick 5-minute look at dependencies and tests, you can make a pretty good, educated decision about whether to use the library, and those 25-30 minutes save you hours or days of potential headaches down the road.
Most of time it's just trust. It's not possible to read through node modules, or reason about C compilers, or even inspect the source code of the operating systems.
> However, it also can simultaneously be not OK for you to publish it in a package directory with a generic name and a description talking about how awesome it is as a trap for other people to run into, and that is really the core problem:
As far as I can tell this is a node-specific problem. I would argue it's a byproduct of resume-driven-development, not of open source culture. I think the onus is on npm to guard against obvious click grabbing projects (like is-odd, is-even, leftpad...), but it's also on the JS standards committee to develop a standard library that doesn't need 1 or 2 line dependencies to function the way a reasonable human would expect it to function.
> sure: it is totally their fault for deploying something without carefully reading your code to figure out that it was a bit shitty beforehand... but should they really have to do that?
Yes, you should at least read through the README and take a look at the issues before using any project. Code with 0 issues filed, no forks, and no stars...should not be used in production without proof reading. You can get a benefit of the doubt if it's clearly a popular project (i.e. apple/swift, google/flutter), because at least then you know multiple people before you have tried it out and been satisfied enough to want to help out.
> if you bring something that you know tastes bad or will make someone sick to the potluck and put a little sign on it saying "gourmet enchiladas: much better than what you get at the store" and put it in the center of the table, it is still a problem
The problem is that this is subjective. What does "tastes bad" mean? Are you expected to know what will make someone throw up? Sure, you'd be a dick for serving raw chicken - but the raw chicken projects generally don't have READMEs in the first place, because they were never intended for usage by others. It's your fault if you incorporate raw chicken into your meal and get sick from it.
The problem is that some people view a package manager as a farmer's market while others see it as a restaurant. Should you serve gourmet meals, or are berries OK too? If you sell raw chicken at a farmer's market, it should be obvious to the consumer that they have to cook the chicken first before eating it. The problem is when a customer goes to a farmer's market expecting it to be a restaurant and gets sick from something they didn't understand.
While `left-pad` certainly wasn't the most optimized implementation, it wasn't a "click-grabbing" project of the sort saurik was positing. It solved the problem the name suggested it would solve. The fallout around it really didn't have anything to do with "resume-driven development"
EDIT: Not sure why this was downvoted, maybe people forgot that padStart and friends appeared in ES2015, long after left-pad was created
off-topic: I still don't understand the use case that left-pad solved. I'm not into js at all but why do you need a dependency to add space to text? I'm asking honestly, I just don't get what the use for this was.
Now if you ask "but why?", ultra-small packages like this are popular in JS because of the lack of a standard lib. Lots of devs prefer to install a dependency for each one-liner utility rather than write them themselves.
The only added value of those packages is that they're usually unit-tested against weird edge-cases that you might miss if you wrote them yourself.
the small libraries are best compared to stackoverflow code snippets. Instead of reading and copying a solution that may be out of date, you can use a (theoretically) tested solution that receives bug fixes. In many ways it is superior to the copy and paste of lore
You're misunderstanding, what is the use-case of that specific package? like I've never found myself needing to pad text on the left, just trying to understand why this was such an important package.
Probably for command line tools that need to add some level of hierarchy into their output. You can add 1 to pad length and pass it to a subcomponent to display its output.
I don't have any use for it either, but this package is not important by itself. It happens that this package is (was) at the very bottom of the dependency chain for some huge packages (like Babel, Webpack, popular frameworks, ...).
When its creator removed it, they literally destroyed the foundation of those popular packages. That's what made it suddenly important.
Yes, dates are very complicated. But that doesn't stop pretty much every modern language from having good built-in support for doing this, because it's such a totally common task. JavaScript is such a mess. It doesn't even support integers, for crying out loud, let alone decent date/time/timezone handling. That moment.js is the most popular band-aid to fix this, despite it having a nasty gotcha API that mixes mutable and immutable functions/types in confusing ways just rubs salt into the wound. Heck, you can't even compare two moment dates with inequality operators without either casting them by prepending a plus sign, or calling functions - it just silently does the wrong thing.
Silently? Since when is comparing objects in JS with (in)equality operators the right thing to do? There's no operator overloading nor special symbol you can implement in objects to make this work seamlessly ever. Casting/stringifying or calling functions is the only way for all objects.
The module system in Node.js (not ES6 modules) makes it super easy to use modules because a module is bound to a variable and you can require the module from anywhere and it will be scoped to the function. It's like if all code in the global JavaScript collective is available in global scope.
> However, it also can simultaneously be not OK for you to publish it in a package directory with a generic name and a description talking about how awesome it is as a trap for other people to run into, and that is really the core problem:
I'm observing this regularly. Currently a bit more in the Rust community since I'm more active there, but I think it's a thing that happens everywhere.
I think in most situations it happens is is incidentally: The people who wrote those packages do not actually have the experience to observe design and quality issues in their package. They might still be in university, or have a minimal amount of industry experience, and found working on an open source project to be a nice way to be creative.
And in one metric that they found for themselves, the might even beat very established projects - let it be performance or finding the new "ultimate abstraction". Therefore they are going forward to market the project as the awesome new thing - without knowing about other important metrics that their project does not meet.
I'm not sure how this can be improved. I think it would require experienced reviewers and curators that are willing to look into other peoples open source projects and help them. As well as authors of those projects to be willing to be mentored. Both is not easy. As an experienced engineer you have enough to to that you don't want to look into YetAnotherProject. As someone who wants to hack on a side project as much as possible you probably don't want to waste your time with talking to someone else.
If I've built a small library that does something useful for my set of needs, it's not crap software. It works perfectly well for my needs.
If someone then goes ahead and uses my library, without understanding what needs it satisfies and what it does not, how is that on me?
I release it to the world with the understanding that someone else might have the same requirements that I had when I built it. Why shouldn't I have released it?
Most of the time yes, but it is not so simple. There is a common resource that can be described as "attention of people who are looking for a library for X" and every time you publish a library for X you use up a bit of this resource. Most of the time this effect is small and more than offset by the value of your code, but it is not zero and overusing this common resource is a dick move.
As an extreme example consider content farms like eHow. After all, they simply publish some low-quality how-to guides, no one is obliged to read them. Why then are they so universally reviled?
> If someone then goes ahead and uses my library, without understanding what needs it satisfies and what it does not, how is that on me?
Being upfront and clear about the needs it satisfies is good documentation practice and plain common courtesy. If you did it and they still misunderstood, then yes, it is on them.
I find this standard ridiculous. We dont apply similar standards to blogs nor videos nor art - them being crappy is not "taking away our attention resource".
Publishing them is not using someone elses attention. And I dont go around hating on eHow. I do expect google to put them down in ranking over time and I do structure my queries so that I dont hit them.
Also, small one time open source project is not tweaking SEO like content farms. That comparison does not work either.
I acknowledge the difference, but it is quantitative, not qualitative.
> We dont apply similar standards to blogs nor videos nor art - them being crappy is not "taking away our attention resource".
That's because the attention cost was driven way down by awesome information retrieval systems that we have (google, etc.). But the cost is still there and it is a good idea to compare it against benefits. Again, this doesn't mean that you should not publish anything.
> Publishing them is not using someone elses attention.
If you publish anything, you expect it to be found by someone (otherwise, why publish at all?). Voila, you've used their attention.
> And I dont go around hating on eHow. I do expect google to put them down in ranking over time and I do structure my queries so that I dont hit them.
This just corroborates my point that the cost is real. In this case the cost is paid by google (they have to expend resources to fight SEO spam) and you (you have to reformulate your queries).
> Also, small one time open source project is not tweaking SEO like content farms. That comparison does not work either.
I'm afraid without concrete examples we are talking past each other. Of course a small-time project that is clearly marked as such is totally OK, but the original post by saurik mentioned squatting common names and overpromising on features, which is definitely into the SEO territory.
Ah yes, I certainly have never seen anyone complain about the signal-to-noise level of low-quality blog posts and videos about topics where people are trying to learn about something like COVID-19... oh, wait ;P.
The reality is that when you publish your package someone else who is trying to get something done is going to use their package manager's search feature to look for like "jsonrpc" and find your half-assed for-personal-use-only jsonrpc library as one of the results; great... now, they have to look at your readme file, open issues, and maybe do some kind of code review to verify whether you are taking this seriously enough for them to rely on your package or not. Of course, a lot of times they aren't even skilled enough to do that analysis correctly as they are just getting into software development and that is hard to do. Worst case you end up higher in the search results than an actually-useful package because you sniped a name like "jsonrpc" or "jsonrpc-client".
This is then the point where (and I am not saying you yourself do this, but this is the developer I am most upset with, and this is as someone who long ago mostly gave up using anyone's packages and now mostly deals with this by being the "wet blanket" constantly trying to convince other people about how low quality some random package they shouldn't be using really is) if your readme file talks about how awesome your package is and has a bunch of past tense descriptions of things you didn't get around to finishing and never will--things you might not even accept patches for or feedback on because this was a project you did over the weekend and are now done with and you didn't really sign up to be an open source maintainer--you aren't just using other's attention, you are outright being a jerk and need to really ask yourself what you are doing by publishing your package in the first place :(.
Agreed. People who demand free labor from open-source maintainers get a lot of deserved flak lately, but it is not that project authors are always blameless. Everyone who publishes some code for others to use faces a dilemma: they want their project to be popular (because being associated with a popular project is prestigious), but popularity means that the project demands additional effort to polish it, deal with bug reports and pull requests etc. So there is a temptation to convey the appearance of maturity without putting in the required effort.
So yeah, it is totally OK for your open source library to be shitty, but please state this clearly in README to save everyone's time.
I agree. If you are actively marketing a project, you are declaring yourself open to criticism. You don't get to play the "what do you want for free?" card and say "my thing is totally awesome and you should use it!" at the same time.
Yes you do. I do that all the time. It just depends on who you're talking to. You can usually tell within a very short period of time who is going to get invested and add value, versus who is going to fall behind and demand a disproportionate amount of attention.
I agree with readme needs to specifically mention unimplemented features. But the responsibility for using an unpopular library lies fully on the user, not author. Of course if the library is popular, author needs to maintain it, or delegate auhority to other maintainers.
As user, it's clear that you need a feature which you found on an unpopular library. Now rather than be skeptical and research the library, or wrap it under yours so you can replace it later, or develop your own, you decide to blindly use it and blame the author.
Pip does not need a package to be public in order to install it at all, I’ve installed plenty of packages from tarballs and directories just fine. From the docs[0]
Install packages from:
- PyPI (and other indexes) using requirement specifiers.
- VCS project urls.
- Local project directories.
- Local or remote source archives.
Although I’m not quite sure how you’re using “for public use” in that sentence.
I meant to say private use, not public use. But if I want to install from pip over the internet, which is my most common use case it needs to be publicly accessible, right? Which for me means GitHub, bitbucket, pypi, or some other bespoke solution just to avoid "publishing" something for my own private use. Or do I misunderstand?
I could do lots of things that would be unnecessarily burdensome for the dubious benefit of having my package unpublished. If it's acceptable to install it on a local network I can just do it directly from a normal filesystem.
I know about private repos. The hangup was installing from them. Last time I looked up how to have private python packages the the impression I got was "It's not possible." But now with a single Google search I see that it's very much possible (by using ssh). Not sure what changed other than two years have gone by.
This is why I really liked the approach of using a github username as a namespace for rubygems, it lets you publish things but with some indication that it may just be for personal use.
Yeah the useful thing is namespaces, I agree it would be good to not tie it to any given platform. I remember the cargo (rust) folks discussed including some way to have a namespace. I don't remember the details but I was disappointed they decided not to go that route.
You can install directly from GitHub, bitbucket, etc. with pip, BTW. My point was that if I want to have packages for private use installable over the internet with package management tools for various languages, I have to put them on the public internet, or jump through some hoops.
Package management tools for both Rust and Go and JS do support direct github urls with minimal boilerplate. You have no reason to make something public unless you want to.
Using pip, and possibly other tools, you can try using a .netrc[0] file to manage login credentials for private repositories.
Although it is a cleartext file (so you may need to manage access to it, for instance when using it in CI), at least your credentials will not show up in log files.
> it also can simultaneously be not OK for you to publish it in a package directory with a generic name and a description
Package repositories can be open to all developers or curated by software distribution maintainers. Being able to publish an open source project without dealing with Linux distribution maintainers is a major selling point of programming language package managers.
Why even let people publish packages if they're not meant to?
> it is totally their fault for deploying something without carefully reading your code to figure out that it was a bit shitty beforehand... but should they really have to do that?
Absolutely. Using someone else's code means implicitly trusting them. Everyone should at least check out the package's source code for quality, sanity and signs of maintenance.
The argument was: only packages which are good enough for production use ought to be published on official package repositories. This allows people to assume that all packages on the repository are stable, well maintained, good quality software without manually verifying each dependency.
So why even let random people sign up a developer account and publish packages? Maybe they need a separate curated "serious packages only" repository.
> Being able to publish an open source project without dealing with Linux distribution maintainers is a major selling point of programming language package managers.
This is how disasters like leftpad happen.
The bar for publishing software is extremely low and we pay the price every day in terms of security, bloat and complexity.
Left pad disaster happened because 1.) npm took away another authors project 2.) npm makes it overly easy to unpublish already published projects. That is ridiculous.
It has nothing to do with how easy it is to publish things. Also, adding own left-pad once disaster happened is quite easy in node.js modules system.
I know free software debaters and open source license philosophers love to hate on this one, but the fundamental core concept of "more importantly, it should absolve authors of shame, embarrassment and ridicule for ugly code" in the context of them choosing to release code or not is a great idea I reckon.
(But I agree wholeheartedly with the thrust of your argument, and am of the opinion that if you spent more time on the logo design or website marketing copy for your library than you did writing the code, then "You're doin' it wrong...")
The way I look at it is that releasing an open-source library is like bringing a child into this world: its utmost well-being is your responsibility. If you don't have the bandwidth for that, don't publish it. And, if your circumstances change later, find another maintainer -- after all, you wouldn't just abandon your child on the street, would you?
It's not a child. It's a hammer. Somebody put it in the community shed so others can use it. If it breaks or somebody needs a nailgun, that's not their problem. I'm glad if somebody lent me their hammer so I don't have to buy one, even if it's not the best.
