How research is done?

Sean at Cosmic Variance is starting a three part series called Anatomy of a paper, on the mysterious process of doing research and publishing it. The first post in the series is called Inspiration and describes how a crazy idea is born (and refined, and modified, and metamorphosed):

Speaking of which — the answer to the inflation-with-a-preferred-direction question wasn’t obvious, so Lotty asked Mark about it. (Who knows where I was — off traveling, probably.) He didn’t know either, but it sounded like an interesting question. So (as one will do) he started scribbling down some models of inflation that might behave that way. Basically, trying to invent a way to allow the negative pressure associated with the inflaton field (the hypothetical field whose energy drives the hypothetical accelerated expansion) to be direction-dependent. We have some general pre-existing ideas about how inflation might conceivably work, and a good field theorist has a bag full of models that can be shaped into different forms depending on the problem under consideration, so it was a matter of asking how easy it would be to tweak those models to give them a preferred direction.

When I did eventually drop by my office, Mark mentioned the idea to me. It sounded interesting, but I didn’t have anything insightful to add off the top of my head. But that afternoon there was a physics colloquium, during which my mind wandered, and I started thinking of different ways the inflaton might get a direction-dependent pressure. After the talk, I went to Mark’s office to say “Your idea is crazy, but here’s an idea that might work.” The next day, Mark gathered Lotty and me into his office to explain why my idea was crazy, but he had a new idea that might work. That process continued for a while, back and forth between the three of us; suggesting models, finding reasons why they should be discarded, realizing that a previously-discarded model might be able to sidestep the previous objections, and so on.

A must-read post, and I cannot wait for the other two parts!

PS:- Here are some of my own thoughts on getting ideas of research, and a link to Highly Allochthonus‘ thoughts on the same (he also calls it inspiration, by the way!).

Update 1: Here is Part II: Calculations:

I should mention that, while working on the vector-field idea, I found myself in another bar — this one across the puddle, a neighborhood pub in London. Guinness this time, not a martini. And wouldn’t you know it, the bartender sees my equations spread out there and asks what it is I’m doing. (By the time I retire, every bartender in the Western hemisphere is going to have at least a passing acquaintance with the basics of contemporary cosmology.) This guy was really into it, and wanted to write down not just the title but also the ISBN number of the book I was reading. Since it was Dodelson’s cosmology text, which is a gripping read but full of equations, I scribbled a short list of more accessible books he could check out, about which he seemed truly excited. Now if only the London pubs would stay open past ten p.m., we’d have an excellent situation all around.

Now, that reminded me of a T-shirt slogan I saw in IISc once:

Don’t drink and derive.

Looks like Sean does not believe in that advice!

Update 2: All good things must come to an end; series of blog posts are no different. Here is the final section called Culmination:

They had, in fact, derived a few of the equations of which we were justifiably proud.

But not all of them! We had, in other words, been partially scooped, although not entirely so. This is a remarkably frequent occurrence — you think you’re working on some project for esoteric reasons that are of importance only to you, only to find that similar tendencies had been floating around in the air, either recently or some number of years prior. Occasionally the scoopage is so dramatic that you really have nothing new to add; in that case the only respectable thing is to suck it up and move on to another project. Very often, the overlap is noticeable but far from complete, and you still have something interesting to contribute; that turned out to be the case this time. So we soldiered on, giving credit in our paper to those who blazed trails before us, and highlighting those roads which we had traversed all by ourselves.

In this post, Sean also summarises the paper writing process in a single sentence:

At the end of the process — from meandering speculation, focusing in on an interesting question, gathering the necessary technical tools, performing the relevant calculation, comparing with the existing literature, and finally writing up the useful results — you have a paper.

A series worth printing out for meditations and contemplations!