How to Talk to Animals: The Earth Species Project

Introduction

Have you ever wondered what animals are saying to each other? Do they have languages, cultures, and emotions like us? Can we communicate with them and learn from their perspectives?

These are some of the questions that drive the Earth Species Project (ESP), a non-profit organization dedicated to using artificial intelligence (AI) to decode non-human communication. ESP believes that an understanding of non-human languages will transform our relationship with the rest of nature and support conservation efforts.

In this blog post, I will introduce you to the people behind ESP, their achievements and challenges, and their vision for the future of interspecies communication.

Who are the people behind The Earth Species Project?

Earth Species Project was founded in 2018 Britt Selvitelle, a member of the founding team at Twitter, and Aza Raskin, who helped found Mozilla Labs(website has been archived) and is also the co-founder of the Center for Human Technology. They were inspired the work of ethologists and bioacousticians who study animal vocalizations, such as the gelada monkeys who produce sounds similar to human speech.

In 2020, they were joined Katie Zacarian, an early member of the team at Facebook, who has been working on applying technology to solve conservation challenges since 2015. Katie has led the development of relationships with biologist partners, driven the thinking about impact, and became the CEO of ESP in 2022.

ESP has also grown to include a dynamic AI research team of five, with deep expertise in mathematics, neuroscience, deep learning, and natural language processing. They are:

• Jen-Yu Liu, who is exploring how to generate new vocalizations that carry meaning and enable two-way communication with animals.

• Benjamin Hoffman, who is developing self-supervised foundation models that can learn from large amounts of unlabeled data.

• Maddie Cusimano, who is working on solving the cocktail party problem, which is how to separate and identify individual voices in a noisy environment.
  
• Marius Miron, who is applying music information retrieval techniques to analyze animal sounds.

• Masato Hagiwara, who is investigating how to use transformers and attention mechanisms to model sequential data.

ESP also has a board of advisors that includes experts from various fields, such as Kay Firth Butterfield, the head of artificial intelligence at the World Economic Forum, Wendy Hanamura, the director of partnerships at the Internet Archive, Brewster Kahle, the founder and digital librarian of the Internet Archive, and Sean White, an advisor, and investor.

What have they achieved so far?

• Earth Species Project has been working on several projects that aim to advance the state-of-the-art in artificial intelligence and animal communication. Some of their notable achievements are:

• Creating unified benchmarks and data sets that can be used to validate their experiments and accelerate the field. These include acoustic and multimodal data from various species, such as sperm whales, zebra finches, carrion crows, humpback whales, and chiff-chaffs.

• Turning motion into meaning automatically discovering behaviors from large-scale animal-borne tag data. This can help us understand how animals use body language and gestures to communicate.

• Establishing semantic generation and editing of communication training AI models to produce new vocalizations that can be used for interactive playback experiments with captive animals. This can help test hypotheses about the meaning and function of animal signals.

• Publishing their first scientific paper on the cocktail party problem in bioacoustics. This paper presents a novel method for separating overlapping animal vocalizations using deep neural networks and shows promising results on real-world data from sperm whales and zebra finches.

What are their challenges and opportunities?

Earth Species Project faces many challenges and opportunities in their quest to decode animal communication. Some of them are:

• Ethical considerations: ESP is aware of the potential risks and harms of interfering with animals and their cultures. They follow strict ethical protocols and work only with captive populations or non-invasive methods. They also seek to promote animal welfare and conservation through their work.

• Technical limitations: ESP acknowledges that current AI techniques are not sufficient to fully capture the complexity and diversity of animal communication. They are constantly exploring new methods and paradigms that can overcome these limitations and improve their models.

• Collaborative opportunities: ESP welcomes collaborations with other researchers and organizations that share their vision and values. They are open to sharing their data sets, tools, and experiences with anyone who is interested in joining their mission.

What is their vision for the future?

Earth Species Project envisions a future where humans can communicate with other species and learn from their perspectives and experiences. They believe that this will foster a deeper appreciation and respect for the rest of nature and lead to a more harmonious and sustainable coexistence.

ESP is also interested in exploring the implications and applications of their work for other domains, such as education, art, health, and entertainment. They imagine that their technology could enable new forms of creativity, expression, and learning that would enrich our lives and culture.

If you are curious and inspired ESP’s work, you can visit their website to learn more, follow their progress, and support their cause. You can also watch their talks and lectures to hear them share their insights and stories.

Conclusion

ESP is a remarkable example of how AI can be used for good and how humans can connect with other species. They are on a fascinating and challenging journey that could change the world as we know it. As Sherlock Holmes would say, “The game is afoot!”

What do you think? Leave a comment below: