Irrespective of who does it or where to host it, I’m still a little stuck on my original question: what is the best way for observatories to offer their data to observers? Provide both raw and reduced (via “standard but non-optimal” pipeline) data? Provide only raw data plus reduction pipeline? And what is the correct balance between block-boxy pipeline and loose set of modules, transparent and flexible enough for advanced users to get the precision and insight they want? There seem to be a number of use cases for observatory archive data whose requirements are conflicting.

Astronomy is research. Research is exploration. Exploration requires unprecedented approaches to problem solving – for software systems as much as or more so than any other part of the empirical exercise.

That said, there is a long history of software development in astronomy. Entire programming languages and computing environments have been invented to solve our problems. Modern software development is often more about logistics than algorithms.

The regular ADASS conferences and software sessions at the SPIE are joined in any given year by more than a few more targeted astronomy software related meetings. And then there’s the IVOA umbrella of activities. As in any community, attending the right meetings is a good first step to meeting the prerequisite of comprehending the complete context of a problem.

Many of these meetings result in published volumes or online proceedings that have a shorter turnaround than the refereed literature. The shelf life of software papers is often quite short. Peer reviewed journal articles remain an option for projects amounting to applied computer science research, more than exercises in already established best-practice engineering.

Which is to say that there are numerous opportunities to collaborate on software projects of wide utility to the community without duplicating efforts in progress. More traction might be gained from contacting well-established projects with suggestions for joint projects addressing missing functionality, than from undertaking a DIY program.

Production software development is not as easy as it might seem on the surface. Robust software doesn’t just follow automatically, no matter how good the initial idea is. And unless your career goal is computer programming, somebody other than yourself has to be identified to maintain, fix and grow the software package once it is released into the wild.

In my opinion, much of the really useful public software in the community’s hands has been built by small numbers of scientists who needed to solve a problem they encountered in their own work, for example: DAOphot, the SDSS “unofficial” spectroscopic pipeline (which was essentially assembled by a few postdocs), GADGET; these are observer, instrument, theory activities respectively.

One of the major problems is that cleaning up software for release, documenting it, and/or writing a paper about it take a lot of effort, and since the rewards for it are relatively small (and there is NO way to get funding for it – even for the page charges, unless it falls under the science topic of a grant you have), few people write it up or publish. This is one of the problems I claim that we need to change.

I agree with Kelle’s comments about including software as part of instrument design/proposal. There are still problems we discussed in the paper (even with good intentions, software comes last and the funding gets squeezed). However, there are also people out there who think that because more or less everyone can program, reduction software is properly fobbed off on grad students or postdocs after construction, and this attitude needs to change. Even big instruments for large telescopes in the US often have very little funding for software (Keck did not pay for the scientists who wrote the DEIMOS pipeline, for example). I think ESO has bigger teams and takes more of the hired-programmer approach, which has its own set of drawbacks.

If you read our white paper, at no time did we recommend brewing one’s own repository software. The software underlying at least some of these repositories is open source. Several people seem really energized about this particular question. I think the choice of repository is much less important, and much less difficult, than figuring out how to get the community to reorient its funding and crediting priorities to recognize the effort that goes into writing and releasing software. If you put all your software on sourceforge and your employers and writers of letters of recommendation are fogeys who do not understand what sourceforge is (or any other repository), it is not going to help you in the long run. (Nothing against all you wonderful fogeys out there! I nearly am one myself.)

Who would fund such a repository? What would happen if funding stopped? There is no reason why it would be more fail-safe than say github or sourceforge. In fact, the latter two are so widely used, that I would actually rather trust my code to them than to an astronomy specific repository. The fact that existing solutions are so popular also means that the quality of the websites is much higher than could be achieved by a couple of software developers on an astronomy grant trying to develop something ‘from scratch’.

Developers should just use whatever existing solutions there are for code hosting. I do think it would be nice to have a centralized wiki with links to the different projects on e.g. sourceforge or github, but that would be relatively simple to develop in comparison to an ‘astronomy sourceforge’.

I think a ‘Computational Astronomy & Astrophysics’ journal is long overdue. I want to release a radiation transfer code later this year (on sourceforge), and plan to write a paper, but it’s very hard to figure out what journal would accept such a paper!

I’ve been thinking for a while that this problem needs to be addressed similar to the way NSF took on Broader Impacts. Basically, any proposal to build a new instrument needs to also include a plan for the data reduction pipeline and will not be considered for funding without addressing the issue. It seems common for big, survey instruments to think about the pipeline from the beginning, but this does not seem true for the smaller workhorses.

Some astronomers do write papers about software or algorithms they develop, which in essence makes them citable, and I wonder what makes someone decide whether or not to publish. Size of the effort and the science that’s being produced with it seem obvious – but there must be more subtle reasons too. I’m in the process of putting a lot of my software online and the idea of opening this up to public scrutiny (although I’ve never been at all secretive about it) is a little frightening….

I suppose also our traditional journals, except maybe PASP, are just not suited to a pure-software paper. Do we need a new peer-reviewed journal purely for software/methods/algorithms in astronomy? It could even be designed as a front end to a repository.