Comparing Planck's noise and dust to BICEP2

In case anyone reading this doesn't recall, back in March an experiment known as BICEP2 made a detection of something known as B-mode polarisation in the cosmic microwave background (CMB). This was big news, mostly because this B-mode polarisation signal would be a characteristic signal of primordial gravitational waves. The detection of the effects of primordial gravitational waves would itself be a wonderful discovery, but this potential discovery went even further in the wonderfulness because the likely origin of primordial gravitational waves would be a process known as inflation which is postulated to have occurred in the very, very early universe.

The B-mode polarisation in the CMB as seen by BICEP2. Seen here for the first time in blog format without the arrows. Is it dust, or is it ripples in space-time? Don't let Occam's razor decide!

I said at the time, and would stand by this now, that if BICEP2 has detected the effects of primordial gravitational waves, then this would be the greatest discovery of the 21st century.

However, about a month after BICEP2's big announcement a large crack developed in the hope that they had detected the effects of primordial gravitational waves and obtained strong evidence for inflation. The problem is that light scattering of dust in the Milky Way Galaxy can also produce this B-mode polarisation signal. Of course BICEP2 knew this and had estimated the amplitude of such a signal and found it to be much too small to explain their signal. The crack was that it seemed they had potentially under-estimated this signal. Or, more precisely, it was unclear how big the signal actually is. It might be as big as the BICEP2 signal, or it might be smaller.

Either way, the situation a few months ago was that the argument BICEP2 made for why this dust signal should be small was no longer convincing and more evidence was needed to determine whether the signal was due to dust, or primordial stuff.

Planck

The best measurement of the dust signal comes from the Planck satellite. Planck doesn't measure the CMB with the same sensitivity as BICEP2, but it has measured the CMB over the whole sky and at many different frequencies. The fortunate situation is that the amplitude of a dust B-mode signal would increase at larger frequencies. Therefore, the hope is that, if this signal is due to dust, then Planck will be able to see it at the larger frequencies. In fact, it was estimates from unreleased Planck data that indicated that perhaps the dust signal is of the same amplitude as the BICEP2 signal.

The problem is that the expected amplitude of signal in Planck's larger frequency measurements, if the BICEP2 signal is dust, is right on the verge of Planck's sensitivity. Therefore, even though Planck can tell us something about the likelihood that this is or isn't dust, noise is still a big issue. In the long run we need to wait for BICEP2 level of sensitivity at multiple frequencies at which point it will be easy to tell dust from primordial stuff. In the medium run, Planck and BICEP2 are now, apparently, collaborating and will be looking, carefully, to see whether Planck's high frequency measurements look like BICEP2's low frequency measurement. If they do, that's bad news, because within BICEP2's field of vision Planck high frequency measurements are only sensitive to dust. If they don't look similar, this doesn't necessarily mean that BICEP2 haven't measured dust, because Planck could just be noise dominated. All of these tricky subtleties are being worked out and hopefully, before the end of 2014, some sort of quantitative (though perhaps still not conclusive) statement about the probability that BICEP2 has seen dust will arise.

In the meantime, in the "short run", cosmologists are going to be impatient and will try to extract as much information as they can from any available data. I like this attitude. I think it's a sign of a healthy curiosity and passion for knowledge. However, one should be careful about what confidence one places in any results obtained. The reason Planck and BICEP2 are taking a long time to say anything is not just because Planck is a large group and getting agreement takes many meetings, conference calls and emails. It is also because there are many effects that need taken into account and understanding each of them takes time. If one doesn't take that time, one might miss something.

With that set of caveats out of the way I'll discuss this interesting paper from a few days ago.

Digitising pdfs, the new way to do cosmology

The B-mode polarisation in the CMB as seen by Planck. Are the features the same as in the BICEP2 plot? That's the crucial question. Well, that and whether this actually corresponds to the CMB as seen by Planck, given that it was digitised from a slide presentation in 2013. Still, not long now for the real thing (i.e. a paper from which to digitise images!)

Neither Planck, nor BICEP2 have released their B-mode polarisation data (i.e no file was released giving the B-mode polarisation signal associated with each line of sight analysed on the sky). Instead, they've released images of the signal on the sky, mostly in pdf format, with a colour bar indicating the signal.

The sneaky thing various groups have been doing, while waiting for actual data, is to digitise these images. That is, to use the colour scale in the image and convert this to a set of signal amplitudes at the various lines of sight being analysed. In fact, even BICEP2 did this, to a Planck image, in their first manuscript. Today another group has analysed a digitised version of Planck's maps, as well as BICEP2's map.

What this group did is conceptually similar to what Planck and BICEP2 are (so the rumours say) doing behind the scenes. That is, to essentially look at the two maps and measure how similar they are.

I won't go into any additional details regarding how the digitising was done. It is described in the paper. The main obstacles come from a bunch of arrows on the BICEP2 image that need to be removed and replaced with estimates of the signal, and from removing the small and large scale fluctuations from the Planck image (because BICEP2 did this to their image and one needs to compare like for like). This process is a little messy and we shouldn't forget that the Planck map being used is the same one from 2013 that has been used in the past and only ever appeared in a slide during a conference talk! However, without the data itself, it's the best people can do, so why not? It's better than nothing (or so I think).

What they saw

With these digitised images they performed a number of tests. The first test basically amounts to counting the numbers of hot spots in the image that pass a certain hotness threshold and subtracting the number of cold spots colder than the equivalent threshold (the "genus statistic"). One can compare this result as a function of the threshold to what is expected from Gaussian statistics. BICEP2 (or, at least, the digitised data from BICEP2's images) appears consistent with Gaussianity under this test. The Planck data does too. At least, this is true after removing the large scales and the small scales from the image. It is worth noting that without this removal, Planck's data seems highly non-Gaussian by this test, not surprising for dust.

The dotted line is the expectation for something that has a Gaussian distribution. The solid line is the BICEP2 data. It seems BICEP2 passes this Gaussianity test. I wonder if this rules out any inflationary models that predict a freaky strong amount of tensor non-Gaussianity? Someone should write a paper!

They then compare the amplitude of the genus statistic for each experiment. Here they find that BICEP2's value is larger than Planck. The interpretation of this is that the fluctuations BICEP2 see are more prevalent on small scales and less prevalent on large scales, compared to Planck. This is actually what one would expect if Planck was seeing noise+dust (i.e. a more flat spectrum) and BICEP2 was seeing the effects of primordial gravitational waves (i.e., a spectrum that, over the considered scales, is growing larger towards smaller scales). However, as they point out, this isn't new. One can already see this from plots of the angular power spectrum in BICEP2's own paper (i.e. the fluctuations are larger on smaller scales). Also, in an earlier paper it was found that primordial gravitational waves are a marginally better fit to BICEP2 alone than dust is, if the amplitude of each is allowed to be free.

The cross-correlation of the previous two images. The blue spots are where both images were blue, the red spots are where both were red, the green is where one image was blue and the other red. The white haze is where either plots was not particularly blue or red (i.e. no positive or negative correlation). I can definitely see more red/blue than green, I think. Is this enough correlation to explain all of BICEP2's measurement? Is this consistent with randomness?

The next test they did, that I'll discuss, is a cross-correlation of the two data sets. This essentially amounts to statistically examining whether Planck's data is showing positive and negative B-modes along the same line of sight as BICEP2's. A large cross-correlation would indicate that when Planck is positive, so is BICEP2 and when Planck negative, so is BICEP2. A value close to zero would indicate that there is no relation, when Planck is positive, BICEP2 is just as likely to be positive as negative. A negative value would be incredibly surprising and would indicate that when Planck is seeing a positive signal, BICEP2 is more often than not seeing negative (and vice versa).

They do see a small, positive, cross-correlation. Now, remember, that what Planck is seeing is likely some combination of dust and noise. Their noise couldn't possibly correlate with BICEP2's (completely different instruments at different locations). Therefore, if there is some correlation, it will be coming from the dust. This positive correlation therefore indicates that at least some of BICEP2's signal is probably coming from dust. The crucial question is how much?

The answer in the paper is probably not all. They estimate the amount of correlation between Planck and BICEP2 that would be needed to fully account for BICEP2's signal and it is more than what they observe. This, also, isn't really particularly new. In fact, BICEP2 did a similar analysis in their original submission, using the same conference talk Planck data, and came to a similar conclusion.

Anyway, after accounting for this correlation, and estimating the remaining signal in BICEP2, the obtained value for "\(r\)" (which essentially measures the amplitude of primordial gravitational waves) is \(0.1 \pm 0.04\). This is not the "\(5\sigma\)" initially claimed by BICEP2 (i.e. \(r\simeq 0.2\)), but, if everything else that led to this value can be trusted, it is still non-zero evidence for primordial gravitational waves. Curiously, this smaller value for \(r\) is actually much easier to align with Planck's temperature data and inflation (for example it would alleviate what I called a second "cosmological coincidence problem").

Where now?

Now, we continue waiting. This paper hasn't really said anything that hasn't already been said or wasn't already known. It has just said and shown these known things in different ways. Any day now we are to expect Planck's paper revealing the non-conference-talk maps of the high frequency polarisation signal along BICEP2's line of sight. These will just be images though, not raw data. The word on the street/corridor is that a fully written draft exists and has clearance to be submitted and nobody I've spoken to knows why it hasn't been. The sort of phrases I've heard about what to expect from this is that "it will clarify a lot of things", but "it won't be conclusive". The safe bet is that it will show that Planck has seen some dust along this line of sight and some noise and that some of BICEP2's signal is almost certainly dust, but that, for now, precisely how much isn't certain.

When mentioning things like \(r=0.1 \pm 0.04\) to Planck people in the past they've essentially shrugged their shoulders and said something like "yeah, that's probably possible"; however, one should keep in mind that a \(2.5\sigma\) deviation of noise alone in BICEP2 would "probably be fine", so that doesn't really say much.

What we really crave is a cross-correlation analysis, similar in spirit to the one in the paper discussed above, but using the actual data. With the data not being public, only BICEP2 and Planck can do this, and they are. Results from this are expected "before the end of the year" (though which year is unclear).

What we really, really crave is more data, at more frequencies, with BICEP2 or better level of precision. This will also come in time.

Twitter: @just_shaun