skud: (Default)
Anyone have a tool/thingy for easily switching between two or more DW accounts? Ideally a Firefox plugin, but if you have other things then let me know. Ditto for AO3 accounts. (But not just "use a different browser" please.)
skud: (Default)
Well, I thought I wasn’t going to get involved in this, but apparently I am. This is not an OTW elections post, but it is a how-the-archive-project-is-run post, and a what-the-hell-went-wrong-with-that-release-the-other-day post.

First up, let me state my background and biases. I’m an open source software developer specialising in complex, database-backed web applications, with 10-15 years experience as such. I first developed a website of that kind in about 1996, and my most recent full-time job working with the "LAMP stack" (Linux, Apache, MySQL, and Perl) was as one of the lead developers on, one of the biggest and most highly trafficked websites in Australia. I left there to work on Freebase, a site organising vast quantities of structured information (including information about TV shows, movies, books, their actors, characters, fictional locations, episodes, release dates, etc) and providing access to it via a free API. Along the way, I've been involved in a great many open source projects, at times in a leadership role. So I have a lot of experience in web development, open source tools and processes, and open/volunteer projects to organise lots of information. Sound like the Archive? Yeah, pretty much.

In 2008 when the OTW and AO3 were kicking off, I briefly considered running for chair of AD&T. However, I was moving countries at the time (Australia to the US) and I heard that Naomi Novik wanted to lead it, so I left it to her. Sometime in 2008-2009, I was briefly on the AD&T mailing list but I got tired of it and left after a while. Ever since then, I’ve occasionally cornered AD&T staffers and let them know that I’m interested in the AO3, would like to code on it, but that I find their processes heavyweight and I’m not interested in going through them just to submit a patch or whatever. I recognise (and have told them) that the processes, meetings, training, structure, etc are useful to many people, especially those who don’t have experience with the tools and need to be taught from the ground up, but that as an experienced coder and open source contributor, I personally found them to be a barrier to my involvement.

How open source projects mostly work, and how AO3 differs )

So, this is why I haven’t been involved in AO3 coding so far. As far as I can tell, the AO3 project either doesn’t have some of the things I need to be able to participate in the way I want, has them but doesn’t provide public access to them, or has them but doesn’t make them findable. All in all, it just makes me sigh and say "too hard".

Hopefully that’s useful background. Now I’d like to talk about version control systems. I’ve mentioned them in passing, above, but they’re really important so I’m going to go off into a bit of a tangent explaining them. If you know about them already, you might want to skim this bit.

Long explanation of version control systems )

So, that was epic, but if you’ve made it this far, you now understand about version control systems, github, and how they offer collaboration tools for distributed teams, and transparency into the changes that are being made to a codebase.

Now I’d like to actually talk about the AO3 project on github. You can see it at (As an aside, it’s a pity that it uses the name "otwarchive", because it makes it hard to search for it if you know it as "ao3" or "archiveofourown". But I digress.)

Some links of interest:

As an outsider, this actually gives me some pretty interesting insight into the project, including the release that occurred the other day. (If you missed it, you should know that the release was pretty buggy, and that led to a representative of the OTW board apologising for it and one of the coders who worked on the skins system, [personal profile] lim, resigning.)

What seems to be wrong with the AO3 project, in my opinion )

This does not seem, to me, to be a well managed project. This is a project where the project lead is acting as a gatekeeper, commiting huge swathes of code (sometimes on behalf of third parties) with inadequate documentation, and allowing extremely poor branch hygiene (skins project mixed up with other changes, for example) to infect the main branch, leading to a buggy release. This should not have happened, and, I suspect, would not have happened if the OTW’s technical leaders had had, or had sought the advice of people who had, experience with distributed open source software development projects and the tools they typically use.

(Worst of all, I think that this poor management is directly responsible for [personal profile] lim resigning. As a relatively inexperienced coder, she should have been shown how to use a branch for her work. If she wasn't able to for some reason, and needed the project lead to proxy her work, the project lead should have made a branch for it. Without this in place, her code was all mixed up with the stuff that had to be done for this Yuletide release, and so she was forced to rush and work 14 hour days to get it in. That sort of overwork just leads to more errors. None of this would have happened if the project lead had insisted on proper branching for each sub-project -- a simple process that any open source software team lead should do as a matter of course. And lim thinks it's her own fault, which is the worst thing of all.)

The AO3 software development project seems to desperately need people who have experience with version control systems, with test driven development ([personal profile] jennyst has written about this a bit), and with release and deployment techniques for large-scale web apps. It also needs people who understand how to design systems that perform at scale, skilled user experience and UI designers, and a whole slew of other specialised skills.

Now I think it is fantastic that the AO3 welcomes and trains up new developers. Seriously, I think it is one of the best things about the project, and I would love it if the open source world at large (indeed, the whole tech community/industry) would do more of it. But it needs to be balanced by having people who know what they’re doing. It needs technical leads who have more experience with large-scale web app development than a single web app that was known not to scale. Most of all it needs project managers who know how to manage a project of this scale, following best practices and so forth.

The thing is, experienced software developers -- I’m one, and I know dozens of others who are probably reading this, many of whom I’ve spoken to about this recently -- don’t want to touch the AO3 project with a bargepole. They can’t figure out how to get involved without being treated like newbs, or how to get their experience and skill recognised. And many of them, who already know how to code and use version control and all of that, would like to contribute just casually, adding a patch here or there, rather than having to commit as an "official" volunteer. And, from what I hear, those who do go through the whole rigmarole are then... not listened to, and leave pretty promptly.

I don’t know whether this is fixable or not, but I hope it is. I have some hope that the move to github, which allows anyone to create a branch to work on, will actually allow the more experienced coders (like me) to get involved at a level we’re comfortable with. And hopefully, if there are more of us around, we’ll get listened to a bit more, and all that experience and skill can actually be put to use by the organisation, rather than brushed off.

Anyway, if you’re an experienced coder, you now know where the AO3 code is, and you know what to do with it. Here’s my fork.
skud: (Default)
Without wanting to dive into the difficult stuff that's been going on lately wrt the elections, and now with [personal profile] lim resigning after working herself to the bone on the recent AO3 push... but I said I didn't want to dive into that. Just wanted to mention it as context.

Anyway, I just learned that Archive Of Our Own is now on github which means that anyone can fork a copy of it, code to their heart's delight, then submit a pull request back to the AO3 team saying "here, I did this, want it?" There is also a public issue tracker to see whether the thing you want to work on is already on their radar, or whatever.

This means that you don't have to sign up to the AD&T committee, or go through official OTW processes, or attend meetings, or whatever, to code on AO3. Admittedly it means your code might not be accepted, so don't start with something too heavy, but if you know how to code and use github and whatnot, and you see a bug you can fix, you can go for it.

[personal profile] jennyst (who I was just talking to about it) says they would welcome pull requests, but reiterates what I said about starting small, as they haven't had many yet. She also suggests sending a heads-up to, telling them "I don't want to sign up as an official volunteer right now, but here's what I'm doing and here's my preferred name and email address", just so they are aware of it. (This is also trackable via github, so this is more for the convenience of the existing volunteer wranglers, who may not be familiar with github but may want to know that people are getting involved in this way.)


skud: (Default)

April 2015

   12 34


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags