Sunday, October 30, 2011

Cheating at jigsaw puzzles

Recent guest poster and blog follower, Matt informed me of the following interesting example of science helping art. Or more precisely, helping recover art.

"In 1944, a bombing raid almost completely destroyed an enormous Padua church fresco that dated back to the Renaissance and had once been admired by Goethe. Some 88,000 tiny pieces of plaster were rescued from the rubble, and a mathematician has managed to piece some of the masterpiece back together."
Link to the original article.

I recommend going to the original article to properly understand what this clever mathematician has done. If I try to summarise I expect that my chances of accurate description are relatively small, given that I will be summarising a news story that is already a summary of the actual algorithm used by the mathematician.

It's a rare example of the overlapping of art, history and mathematics. The application won't have a profound impact on any of the disciplines, but it is interesting nonetheless.

Of course if the art historians had just given me all the pieces 13 or so years ago I'm sure I could have put them all together on one of the many afternoons I spent in the summer holidays watching cricket and piecing together progressively more complicated jigsaw puzzles. But, I'm sure they had their reasons to wait.

[With apologies to Shaun for editing this post, here are a couple of mathematical explanations of this research, beginning with a summary. -MM]

Massimo Fornasier, "Mathematics enters the picture." Mathematics and Statistics, 2009, Volume 3, 217-228.

Massimo Fornasier, "Faithful Recovery of Vector Valued Functions from Incomplete Data: Recolorization and Art Restoration," Scale Space and Variational Methods in Computer Vision: Lecture Notes in Computer Science, 2007, Volume 4485/2007, 116-127.

Massimo Fornasier, Domenico Toniolo, "Fast, robust and efficient 2D pattern recognition for re-assembling fragmented images." Pattern Recognition, Volume 38, Issue 11, November 2005, 2074-2087.


  1. "progressively more complicated" should be loosely interpreted.

    The best I ever managed was:

    - A 1000 piece puzzle that didn't have a picture on the box to work off.
    - A 1000 piece puzzle that had a picture, but was an almost featureless desert plus sky.
    - A 2000 piece puzzle of a painting of an old dusty library. Man I felt like God when I finally finished that monster.

    I have since seen 5000 piece puzzles in shops. I feel for my poor future grandchildren.

  2. Very interesting! I ran a quick search for articles about the maths side of the research. Don't think I can attach them via the comments so I'll append in the post.

    I've never thought about it like this, but it's true that the conservators/restorers in museums are the most scientific people in the building.

  3. An aside - this article and its sources are a good example of how the current model of disseminating academic research can actually block access to that research.

    Anyone who might have read the newspaper article and wanted to know more about the technicalities is dependent on access to pay-wall protected journals. My own access is via a university library that pays untold thousands to research journals annually for precisely this privilege. But I'd bet that the mathematicians, who actually did the work, gave it to the museum, and wrote it up to make it comprehensible to a broader audience, would email you their work without a fee on request.

  4. Funny you should bring up this particular problem. Barnabas, our other illustrious guest poster, actually emailed me yesterday the following two links:

    suggesting I write a brief post about open access. It is a very important and relevant issue to the current academic world. I'll collect all of these links into a brief post later in the week.

    In physics and mathematics we have a wonderful resource known as the arXiv where everyone universally uploads their articles either as they submit them to a journal or once they have been published. Some of the more technical papers that appear as references in the links you added to the main blogspot can be found here, which is a list of the papers Massimo Fornasier has uploaded to the arXiv.

    We physicists find the arXiv indispensable and wonder how other disciplines survive without it. We still need journals to tick the "this has been peer reviewed box"... we really don't need them for anything more than that though.

    In fact, completely unrelated to both this blogpost and Barnabas' email, the topic of journals came up at lunch today. This is less surprising than a general reader might think. This topic probably comes up weekly in academic institutes around the world.

  5. I'm not bashing peer-review, even in its more rigid aspects. Some kinds of fact-checking can only be conducted by other specialists and with specialist intuition, and disciplinary rigor matters a lot. Like the whole problem of deep inter-disciplinarity that this blog (in our own local way) is all about, there isn't an easy solution to this literal kind of access either.

    But things can only start to change with an acknowledgment of a problem. Critical Inquiry's new approach to the theory-archive is a good example from the Humanities & Social Sciences for your open-access post, I think. Check out particularly the first link, which is a digital manifesto/ clarification of intent.

    "We physicists find the arXiv indispensable and wonder how other disciplines survive without it." Good question. Poorly? There are other ways to share, of course: the art world has socializing and schmoozing; humanists have conferences, panels, round-tables, and teaching. But from my observations the sciences are just fundamentally better at collaborative work than “we” are, with artists being a half-exception to this rule.

    Anyway, all this is indeed an aside until open-access gets its own post: I think the original topic of algorithmic maths/mural restoration is really interesting & I hope to take a closer look at all the articles.

  6. BTW, did you ever encounter Julie Maxton at Auckland? She is now the head of the Royal Society.

  7. No, not personally. The closest connection I have is that a good friend of mine did his law honours thesis with her back in the Auckland days.

    It seems like the secret NZ plan to take over the world is proceeding nicely.

  8. Opening now in Paris: a show curated by David Lynch called "Mathematics: A Beautiful Elsewhere"

    The press description is quite suggestive - I might perhaps write it up as a post in the next week, but for the moment here's an excerpt:

    Mathematics: A Beautiful Elsewhere is a unique exhibition ... offering visitors, to use the mathematician Alexandre Grothendieck’s expression, “a sudden change of scenery.”

    A large number of mathematicians and scientists contributed to the creation of this exhibition, and eight of them acted as its overseers: SIR MICHAEL ATIYAH, JEAN-PIERRE BOURGUIGNON, ALAIN CONNES, NICOLE EL KAROUI, MISHA GROMOV, GIANCARLO LUCCHINI, CÉDRIC VILLANI and DON ZAGIER. Representing a wide range of geographical backgrounds and mathematical disciplines, they work in areas such as number theory, algebraic geometry, differential geometry, topology, partial differential equations, probability, mathematics applied to biology…

    They were accompanied by nine artists chosen for their exceptional ability to listen, as well as for their great sense of curiosity and wonder. All of these artists have exhibited at the Fondation Cartier in the past: JEAN-MICHEL ALBEROLA, RAYMOND DEPARDON AND CLAUDINE NOUGARET, TAKESHI KITANO, DAVID LYNCH, BEATRIZ MILHAZES, PATTI SMITH, HIROSHI SUGIMOTO and TADANORI YOKOO, as well as Pierre Buffin and his crew (BUF). They worked together to transform the abstract thinking of mathematics into a stimulating experience for the mind and the senses, an experience accessible to everyone.

    Two exceptional contributions will make it possible to observe the real-time evolution of two major projects of contemporary science: experiments with matter being conducted by the CERN inside THE LARGE HADRON COLLIDER (LHC), and the mapping of the early universe as recorded by THE PLANCK SPACECRAFT OF THE ESA whose data is being studied by European astronomers, notably at the IAP. In addition, Pierre-Yves Oudeyer and his colleagues at INRIA and Bordeaux University will display the latest results from their work, which concerns a SOCIETY OF ROBOTS ENDOWED WITH ARTIFICIAL CURIOSITY, the ERGO-ROBOTS. Their active presence in the exhibition is in itself an experiment that will allow these scientists to advance even further in their revolutionary research program.

    Mathematics: A Beautiful Elsewhere is a geometric, algebraic, artistic and cinematographic mosaic that gives everyone a chance to experience fragments of mathematical beauty.

  9. Hmmm, it opened in mid-late-October. I've been toying with writing something about it myself for a while, but I've found it difficult to find any concrete information of what any of the exhibitions actually are. I need a password it seems to see them online.

    Can I see any of the exhibitions anywhere?

    In any case, it really is an interesting art exhibit, very much in line with this blog and its motivation. I would be very curious to see your views on the exhibit. In fact perhaps we could both commit to writing one post, giving the artist and the scientist's views on the exhibit. Given that ESA's Planck satellite is, of all of humanity's experiments, the one with the most relevance to my research I should be able to find something to say.

  10. Yes, though it's open 'till March which is a good run of time. It does seems like a perfect object for a collaborative post. And we would be responding to the idea of the show in any case, since we can't see the exhibition itself. I'll look around and see what reviews I can find and what they say.