Exploring Leaf Patterns with Machine Learning and Computer Vision

Fig1-no-labelsPenn State University’s Peter Wilf and colleagues have created computer software that uses machine learning algorithms and computer vision to identify patterns in leaves. Prior to the creation of this computer program, botanists had to painstakingly look up formations in a reference book. This technology—which is 72% accurate—makes this process a whole lot easier. The software creates a heat map on the leaf images which then highlights the different characteristics with tiny red dots. The dots then show which family the leaf can belong to. Read more about this exciting new technology in Wired’s article: A Computer With a Great Eye Is About to Transform Botany.

Here is an excerpt from the story:

‘“I’ve looked at tens of thousands of living and fossil leaves,” says that paleobotanist, Peter Wilf of Penn State’s College of Earth and Mineral Sciences. “No one can remember what they all look like. It’s impossible—there’s tens of thousands of vein intersections.” There’s also patterns in vein spacing, different tooth shapes, and a whole host of other features that distinguish one leaf from the next. Unable to commit all these details to memory, botanists rely instead on a manual method of identification developed in the 1800s. That method—called leaf architecture—hasn’t changed much since. It relies on a fat reference book filled with “an unambiguous and standard set of terms for describing leaf form and venation,” and it’s a painstaking process; Wilf says correctly identifying a single leaf’s taxonomy can take two hours.

That’s why, for the past nine years, Wilf has worked with a computational neuroscientist from Brown University to program computer software to do what the human eye cannot: identify families of leaves, in mere milliseconds. The software, which Wilf and his colleagues describe in detail in a recent issue of Proceedings of the National Academy of Sciences, combines computer vision and machine learning algorithms to identify patterns in leaves, linking them to families of leaves they potentially evolved from with 72 percent accuracy. In doing so, Wilf has designed a user-friendly solution to a once-laborious aspect of paleobotany. The program, he says, “is going to really change how we understand plant evolution.”’

Click here to read the Penn State press release: Leaf mysteries revealed through the computer’s eye.

Photo credit: Shengping Zhang

VIEW FELLOW’S PROFILE