This is an in-depth discussion of Django's proposed governance changes from a long-time committer (since 2007), release manager, and board member of the Django Software Foundation.
It's also an interesting mini history of the governance of the project and formation of things like the DSF.
Essentially they plan to dissolve the Django Core and make the development of Django more open to new contributors.
If it means anything, reading the first part of it inspired me to want to contribute to it, if at all ever possible, because I believe Django is important enough to sustain and to recognize these issues before they evolve into problems is good idea.
Personally, I find this quite timely, because I recently wanted to get involved in open source and looked into working with Django. Their ticket tracking system looks far easier to follow and get involved with than many other large projects, but still I find zero tickets marked 'easy pickings' and 'unassigned'.
This may be the wrong response, but my takeaway is that the project is too far along for new contributors, so it's better to try and get involved with smaller, less well known projects where I can contribute.
In part, I'm commenting because I would love for this to not be true, and would be quite excited if someone pointed out flaws in my understanding!
Just out of random interest - does anyone know what happened to the Apache Bloodhound project [0], which was based on Trac? I was somewhat optimistic when it started.. let's be real, Trac has many serious shortcomings, but it's still extremely practical for a small team of technical users.
No progress was made on GenericTrac [1] whatsoever for the 5+ years I ran Trac in production and followed the project. Bloodhound made a single release, and then I never heard a word about it again.
It seems not that many people have enough time and long-term interest in contributing significant improvements to such projects.
GenericTrac was mainly a plan by Christian Boos, who worked a lot on Trac in ~2005 - 2013, but has not actively worked on it much after that.
Apache Bloodhound was mainly triggered by one company (WANdisco), which has lost interest in it. Of the few remaining people, Ryan J Ollos has since become a Trac core developer instead, and has been keeping very active there.
Recently Gary Martin (formerly(?) WANdisco) "restarted" Apache Bloodhound as a Django app, not based on Trac anymore.[2]
> Trac has many serious shortcomings
I don't disagree, but would be interested in what you think they are.
First let me say that I haven't looked at Trac since I ditched it from production (prior to the Bloodhound incubator). So don't expect this to be fair criticism today.
> It seems not that many people have enough time and long-term interest in contributing significant improvements to such projects.
I am not convinced that is the only reason. The trac mailing list was (in my opinion) hostile to people that showed up with an idea and willingness to work on it; nobody would give encouragement or guidance and there was usually a "fk off and fork" or two in there. People aren't going to stick around to contribute if the maintainers/community are not interested to include them. That is/was a serious shortcoming.
The data model was not well thought out, with lots of UNION and UNION ALL in the queries, row-modelling things that should be normalized, and so on. That's probably my #1 technical issue, in terms of the code, it was hard to correctly add functionality because things were just read into RAM and manipulated there (expert SQL knowledge is of no use when working with the trac database)
The switch of templating engine did not make the codebase simpler, but instead made everything more convoluted and difficult to work with.
The primary issue though, was the lack of Milestone customization, and the incredibly poor performance. I had to implement custom search, custom indexes, custom reports, and custom everything else. And this was a 5-person team, using only ticketing (no repo, no wiki, nothing).
I would say the mailing list has become much less hostile and more supportive then.
The data model has not changed much, so that criticism is still relevant.
The templating engine has been switched again (ClearSilver -> Genshi -> Jinja). I don't know if you refer to the first or the second switch. I also don't know if this was an improvement or just even more churn.
Lack of Milestone customization: I'm not sure I understand.
> the mailing list has become much less hostile and more supportive then.
Glad to hear that! Running an open project is hard.
> [...] switched again (ClearSilver -> Genshi -> Jinja)
Genshi was current when I used it; I wasn't aware it had switched to jinja (or it slipped my mind by now..)
> Lack of Milestone customization: I'm not sure I understand.
Back then, there were no custom fields for milestones, and the code didn't make it easy to do. Of course, this means you need changes to search facilities, and any other parts of the codebase. I had to write that also, and the target workflow was not at all complex.
As for the performance, we had about 20k tickets and 2k milestones when the system was replaced. By then it had been completely unusable for a long time.
Honestly, the only positives I have to say about Trac concern real-world usage by a hacker. Once you've learned it, and hacked it to your liking, it's very productive. But to get to that point, it's a sixty feet tall stack of ugly hacks, and frankly I would not recommend it to anyone...
Again to be clear, I did not look at the trac repo in many years, nor do I intend to.
No, unfortunately it was nowhere near good enough for that, it was just massacred to solve the problem.. Plus it was built on probably 20 patches from trac-hacks in the first place.
Very hard to find the time to upstream something like that when you are on a limited budget in the first place..
So "It seems not that many people have enough time and long-term interest in contributing significant improvements to such projects" wasn't that wrong. ;)
It seems so !
They suggest others ideas on the contributing page [0] :
- Improve the documentation or write unit tests.
- Triage tickets and review patches created by other users.
I also imagine that reaching committers through irc will bring some ideas to work on.
I see a lot of projects recommend triaging tickets or writing unit tests, but those both seem to be areas that require substantial knowledge about the state of a project, so as a newbie they don't feel accessible.
It would be cool to see lists of how you can triage (e.g. encouraging people to help reproduce) or add tests (e.g. areas that could use more tests).
Possible I'm unusual, but I actually like writing documentation as a way to learn a new codebase.
When I started a new job earlier this year I actually jumped at the chance to grab some documentation tickets, because it meant I had to not just halfheartedly skim through the relevant code but actually read it in detail and understand what was going on.
I have some ideas about how to improve that. But will probably be a little while before I can do anything with those ideas, since right now all my Django-related time is going into the governance proposal.
Big Django fan here. Love the project and framework. I love DRF and other plugins. I am a developer but I haven't ever felt the need to contribute for two reasons - 1) It works 2) The entry barrier is high. For instance, other than some basic documentation, I haven't seen anything more substantial "howto" to contributing including Django developer events. For instance, Carlton Gibson had a DjangoCon 2018 talk about contributing to Django. This was a good start. However, we need more meaningful engagements for example a code walk through for different areas of the Django codebase. Maybe a howto series on hacking on different parts of the codebase. This is more engaging & useful for someone who wants to contribute. It is a step beyond saying "here's a git repo, mailing list and issue tracker. Now, go forth and code!"
I wonder if the continued use of Trac and a mailing list in a modern, Github-centric world is holding back contributions, perhaps even more so than the need to reshuffle responsibilities among existing contributors.
Consider that if a Django user wants to contribute with an account they're already logged into, the only thing they can see is https://github.com/django/django/pulls - a wall of structure and external references, with no clear way to suggest features. To the community's credit, people gently redirect "stray" pull requests to the proper channels, but this still has to feel like a form of rejection to well-meaning developers!
Compare this to https://github.com/facebook/react/issues and https://github.com/rails/rails/issues - inviting colors, a clear way to raise issues one click away from the codebase itself, centralized discussion of those issues, automatic links between PRs and issues. Nobody is left behind; any suggestion can become a discussion with decision-makers in place. This is what the modern developer has come to expect.
I love Django as a framework - it's a set of abstractions that make a ton of sense together, but simultaneously are much more explicit and easier to dive into than Rails. I can command-click my way through the entire stack in VS Code and know exactly what's going on; when there's a customization to make, there's almost always two or three hooks I can use to make it happen. And it pains me that there seems to be a fear of change and a fragmentation of contributor tooling that's holding it back from evolving. I don't know that Trac -> Github is the best path, but I hope that this is the first of numerous innovations that the core and board intend to discuss earnestly over the coming years.
EDIT: As I consider what I've written above, I'm thinking about a concept of TTFK - "Time To First Keystroke" that's intended as part of a contribution. Once you start writing an issue or coding a PR, assuming there's a clear path for it to get accepted, that first keypress is the start of an avalanche that gets obsessed developers deeper and deeper engaged. But if you look at the README on https://github.com/django/django , count how many clicks it takes, and words to read, and new platforms to join, until you can type your first character on a non-documentation feature request. It's not a fun number. Process/triage is great, but you have to have a way that someone's unstructured contribution can gain structure over time without requiring them to retype it or jump through hoops.
I'm extremely skeptical of any "just use platform/service/tool X" type of suggestion, largely because I've been doing this long enough to know that "just" needs to be in scare quotes and that often the claimed benefits don't materialize.
Django does need a place to store the tickets, and GitHub's tracker... is not great. I'd honestly want to look at adding OAuth or other easier signup/account management to Trac before I'd look at moving to GitHub issues.
A fair and valid point, and to Django's credit, they actually do have Github OAuth as the signin procedure for Trac! Unfortunately, it's still a new interface for people to learn and be afraid of - and most importantly, it's a place where activity can be hidden from people who are just browsing the codebase, so the project looks less active than it actually is!
Bots that mirror content/comments between the various platforms might be an interesting interim step that could minimize those scare quotes, and achieve many of the benefits in a minimally invasive way.
Every year or so someone brings up the prospect of moving from Trac to something else (mostly github recently). October was the last time this was brought up[0].
I'm of the opinion that Trac is probably blocking a number of people from contributing, even if that contribution is participating in comments and voting on issues.
But that's exactly the kind of thing the community as a whole can be empowered to do without the old guard of core vetoing the suggestion outright. The python community are considering doing just that now with PEP 581.
I agree with this. I can deal with Trac (although I’d still prefer GitHub) but mailing lists are a big no-no for me.
The funny thing is that I read this post yesterday and thought about contributing to Django... and they lost me when I saw “mailing list” somewhere in there.
1. Lots of self-hostable solutions are available.
2. Unfortunately for mailing lists, those self-hostable solutions very often make it way easier to plow through conversations.
3. Out of sheer curiosity - have you ever had an actual e-mail conversation with multiple persons involved in replying to each other's e-mails? I do it daily and it sucks.
If you know of something which is easy to set up, easy to administer, easy to use and provides better handling of multi-way threaded discussion, I'd be interested to hear about it.
(I know Python is experimenting with a forum-based solution, but I worry about the burden of both yet another account to create and site to check, and finding people to maintain and moderate it)
The mailing list is really only used for decision making and proposals. An issue is usually discussed on the issue tracker until some kind of deadlock occurs.
Then once a PR is developed the discussion happens in PR comments.
The mailing list isn’t used for commenting on patches like some other projects.
Django's stance of not supporting NoSQL and other alternative databases is such an embarrassment. Not saying they need to chase every fancy new technology, but for a "batteries included" library, they sadly miss a whole bunch of features.
But it's worth noting Instagram has been a generous corporate sponsor of the DSF, and hired a Django committer several years ago. I just don't know of any company that currently has employees paid to work on Django, in the same way that, say, several big tech companies pay people to work on various parts of the Linux ecosystem.
Does Eventbrite pay Andrew Godwin to work on Django? He's certainly a prolific contributor but I'm not sure if his Django work is officially sponsored by them.
I definitely did not mean to target Instagram. We have a general problem of open source funding, and something like Django having this problem is disheartening.
It's also an interesting mini history of the governance of the project and formation of things like the DSF.
Essentially they plan to dissolve the Django Core and make the development of Django more open to new contributors.