On a December night in 1973, thieves overpowered a security guard at Harvard’s Fogg Museum, broke into the Coin Room, and made off with somewhere between 8,000 and 12,000 pieces of silver and bronze currency from ancient Greece and Rome—a haul worth $5 million or more (approximately $37 million today). Authorities caught the group of small-time local criminals responsible for the heist in late 1974; in the intervening years, most of the coins were recovered. But not all of them.
Now, museum staff hope to leverage artificial intelligence to track down some of the silver stolen in “the biggest art theft in the U.S. at that time,” says Laure Marest, the Damarete associate curator of ancient coins at the Harvard Art Museums. In collaboration with computer scientist Yifei Bao, Marest is developing a program that works from digitized, archival photographs to scour millions of online auction records and identify pieces of the unaccounted loot that may pop up on the web for resale.
When she started her job in 2023, Marest went through archival records and discovered that 10 to 15 percent of the collection taken during the theft was still missing. “That was probably close to 1,000 coins, and some of them [were] really valuable,” she says.
So she enlisted Bao—who was then a doctoral student at Boston University and a Hao family intern at the Harvard Art Museums—to create an algorithm that used ChatGPT to assist with the search. “What we want is, we find a coin online, we put it into the machine, and the machine can tell us if the coin is a match to our missing coin,” said Bao, at a recent seminar at the museum.
The first step was to identify a portion of Harvard’s inventory that could serve as a training source for the algorithm, cross-check for potential matches, and identify what, exactly, was still missing. Because the museum’s coin collection had grown gradually since 1895, and cataloging methods have evolved over time, this last question was still a bit of a mystery.
“When the coins were stolen,” Marest says, “there was no comprehensive system to just push a button [that could] tell you, ‘How many coins do I have?’”
With help from undergraduate interns, Marest built a database of 3,000 coins that had been photographed and documented in a paper-based cataloging system, identifying about 200 that were still missing. Most were silver, from ancient Greece.
Bao then trained a model to recognize those missing coins, using image matching and a deep learning function that translates a coin’s specific visual properties—a small indentation, a space between the hairline on a deity’s profile and the coin’s outer edge—into numerical vectors that the computer could analyze mathematically.
“We want computers to understand the image by transforming it to a mathematical fingerprint,” Bao said in the seminar. From there, the algorithm can determine the likelihood, in percentage terms, that a coin from the database matches one that appears online.
Ancient coins have unique qualities that can both help and hinder a machine making such determinations, Marest notes. Most were stamped by hand and serialized: like snowflakes, no two are exactly alike. But they are small and made of metal, which makes them tricky to photograph; it’s hard to train a computer to account for fuzzy resolution or variations in lighting conditions.
Still, for an idea in the proof-of-concept phase, the project’s measure of success isn’t necessarily to recover all, or even any, of the coins Harvard lost. That would be “a long shot,” Marest acknowledges. Online auction records “only go back 15 years, while those coins probably went on the market in the ’80s and ’90s,” she says. “So, unless they went back on the market more recently, they would not show up.”
The real value of the project is broader, she thinks. Provenance is a top-of-mind issue in the museum world. Determining where a priceless artifact originated—and whether it was stolen, looted from an archaeological site, or taken during war—can involve a lot of digging through archives and assembling pieces of far-flung information.
“If we can have the help of AI, matching some of those records more expediently, obviously that’s great,” Marest says.
A project that bridges the museum and computer science worlds, she says, can also help budding computer scientists like Bao realize the potential of their skills in spheres of knowledge they may not have considered.
“When you think of AI projects these days,” Marest adds, “you hear about finance, information technology, marketing, healthcare. Museums are not at the top of the list.” But the more proficient people become with the technology, she says, “the better choices we can make about what we want this technology to do for us.”
