Well, I thought I wasn’t going to get involved in this, but apparently I am. This is not an OTW elections post, but it is a how-the-archive-project-is-run post, and a what-the-hell-went-wrong-with-that-relea
First up, let me state my background and biases. I’m an open source software developer specialising in complex, database-backed web applications, with 10-15 years experience as such. I first developed a website of that kind in about 1996, and my most recent full-time job working with the "LAMP stack" (Linux, Apache, MySQL, and Perl) was as one of the lead developers on realestate.com.au
, one of the biggest and most highly trafficked websites in Australia. I left there to work on Freebase
, a site organising vast quantities of structured information (including information about TV shows, movies, books, their actors, characters, fictional locations, episodes, release dates, etc) and providing access to it via a free API. Along the way, I've been involved in a great many open source projects, at times in a leadership role. So I have a lot of experience in web development, open source tools and processes, and open/volunteer projects to organise lots of information. Sound like the Archive? Yeah, pretty much.
In 2008 when the OTW and AO3 were kicking off, I briefly considered running for chair of AD&T. However, I was moving countries at the time (Australia to the US) and I heard that Naomi Novik wanted to lead it, so I left it to her. Sometime in 2008-2009, I was briefly on the AD&T mailing list but I got tired of it and left after a while. Ever since then, I’ve occasionally cornered AD&T staffers and let them know that I’m interested in the AO3, would like to code on it, but that I find their processes heavyweight and I’m not interested in going through them just to submit a patch or whatever. I recognise (and have told them) that the processes, meetings, training, structure, etc are useful to many people, especially those who don’t have experience with the tools and need to be taught from the ground up, but that as an experienced coder and open source contributor, I personally found them to be a barrier to my involvement.( How open source projects mostly work, and how AO3 differs )
So, this is why I haven’t been involved in AO3 coding so far. As far as I can tell, the AO3 project either doesn’t have some of the things I need to be able to participate in the way I want, has them but doesn’t provide public access to them, or has them but doesn’t make them findable. All in all, it just makes me sigh and say "too hard".
Hopefully that’s useful background. Now I’d like to talk about version control systems. I’ve mentioned them in passing, above, but they’re really important
so I’m going to go off into a bit of a tangent explaining them. If you know about them already, you might want to skim this bit.( Long explanation of version control systems )
So, that was epic, but if you’ve made it this far, you now understand about version control systems, github, and how they offer collaboration tools for distributed teams, and transparency into the changes that are being made to a codebase.
Now I’d like to actually talk about the AO3 project on github. You can see it at https://github.com/otwcode/otwarchive/
(As an aside, it’s a pity that it uses the name "otwarchive", because it makes it hard to search for it if you know it as "ao3" or "archiveofourown". But I digress.)
Some links of interest:
As an outsider, this actually gives me some pretty interesting insight into the project, including the release that occurred the other day. (If you missed it, you should know that the release was pretty buggy, and that led to a representative of the OTW board apologising for it
and one of the coders who worked on the skins system, lim
.)( What seems to be wrong with the AO3 project, in my opinion )
This does not seem, to me, to be a well managed project. This is a project where the project lead is acting as a gatekeeper, commiting huge swathes of code (sometimes on behalf of third parties) with inadequate documentation, and allowing extremely poor branch hygiene (skins project mixed up with other changes, for example) to infect the main branch, leading to a buggy release. This should not have happened, and, I suspect, would not have happened if the OTW’s technical leaders had had, or had sought the advice of people who had, experience with distributed open source software development projects and the tools they typically use.
(Worst of all, I think that this poor management is directly responsible
for lim resigning
. As a relatively inexperienced coder, she should have been shown how to use a branch for her work. If she wasn't able to for some reason, and needed the project lead to proxy her work, the project lead should have made a branch for it. Without this in place, her code was all mixed up with the stuff that had to be done for this Yuletide release, and so she was forced to rush and work 14 hour days to get it in. That sort of overwork just leads to more errors. None of this would have happened if the project lead had insisted on proper branching for each sub-project -- a simple process that any open source software team lead should do as a matter of course. And lim thinks it's her own fault
, which is the worst thing of all.)
The AO3 software development project seems to desperately need people who have experience with version control systems, with test driven development (jennyst
about this a bit), and with release and deployment techniques for large-scale web apps. It also needs people who understand how to design systems that perform at scale, skilled user experience and UI designers, and a whole slew of other specialised skills.
Now I think it is fantastic
that the AO3 welcomes and trains up new developers. Seriously, I think it is one of the best things about the project, and I would love it if the open source world at large (indeed, the whole tech community/industry) would do more of it. But it needs to be balanced by having people who know what they’re doing. It needs technical leads who have more experience with large-scale web app development than a single web app that was known not to scale. Most of all it needs project managers who know how to manage a project of this scale
, following best practices and so forth.
The thing is, experienced software developers -- I’m one, and I know dozens of others who are probably reading this, many of whom I’ve spoken to about this recently -- don’t want to touch the AO3 project with a bargepole. They can’t figure out how to get involved without being treated like newbs, or how to get their experience and skill recognised. And many of them, who already know how to code and use version control and all of that, would like to contribute just casually, adding a patch here or there, rather than having to commit as an "official" volunteer. And, from what I hear, those who do go through the whole rigmarole are then... not listened to, and leave pretty promptly.
I don’t know whether this is fixable or not, but I hope it is. I have some hope that the move to github, which allows anyone to create a branch to work on, will actually allow the more experienced coders (like me) to get involved at a level we’re comfortable with. And hopefully, if there are more of us around, we’ll get listened to a bit more, and all that experience and skill can actually be put to use by the organisation, rather than brushed off.
Anyway, if you’re an experienced coder, you now know where the AO3 code is, and you know what to do with it. Here’s my fork.