MIT Algorithm Finds Previously Unseen Connections Between Pieces of Art
MIT researchers developed a new system called ‘MosAIc’. MosAic has found some rather difficult to notice commonalities between various artworks located in Amsterdam’s Rijksmuseum and the Metropolitan Museum of Art. The algorithm uses deep leaning networks to scan images for similarities in pieces of work that are otherwise totally unrelated.
All you have to do to use MosAIc is input an image. That’s it. MosAIc does the rest of the work, finding similar art. In one instance, MosAIc linked Jan Asselijn’s ‘The Threatened Swan’ and Francisco de Zurbaran’s ‘The Martyrdom of Saint Serapion.’ MIT CSAIL PhD student Mark Hamilton, explained: ‘These two artists did not have a correspondence or meet each other during their lives, yet their paintings hinted at a rich, latent structure that underlies both of their works.’
One of the hardest parts about writing MosAIc was the task of creating an algorithm that not only could find similarities in style or color, but also in meaning. The developers studied a deep network of ‘activations,’ (properties) of each image in the museums. The node distance between the ‘activations’ was the metric researchers used to judge similarity.
The software’s creators also used a relatively new image search method called a ‘KNN Tree.’ This way of searching is able to group images into a tree-type topology. To find a match, the algorithm begins at the base of the group-structure. The software then follows the ‘branch’ most likely to have a match until it’s found the closest image in that branch. The data structure is then able to improve itself by more or less ‘self pruning’ the ‘branches.’
Hamilton hopes the work started with MosAIc can be later expanded upon. This algorithm could be useful for many fields such as, social sciences, humanities, and medicine. Hamilton stated, ‘These fields are rich with information that has never been processed with these techniques and can be a source for great inspiration for both computer scientists and domain experts.’
Featured Image Credit: [MIT/CSAIL]
