If you were diving in the waters off what’s now southern Peru about 36 million years ago, you may have come across a creature about 4 meters long.

Its name is Mystacodon and it’s the earliest known mysticete, the group that, today, we call the baleen whales.

But Mystacodon probably didn’t have any baleen,-- the sieve-like structure inside of whale’s mouths that lets them feed by taking in a mouthful of water and sifting out their prey.

Instead, Mystacodon had teeth.

So, if this was a baleen whale, where was its baleen?

Where did baleen come from?

And how did it live without it?

For a while, we assumed that the story was pretty straightforward: ancient whales developed baleen early and rode that evolutionary wave to success.

But some newly discovered fossils have suggested that what really happened might be more complicated, with global changes to the ocean playing an important role.

And, in the end, it looks like one of the key changes in whale evolution may have kind of… sucked.

The earliest whales were predatory land animals who first went into the water about 52 million years ago, during the Eocene epoch.

While they started out looking kind of wolf-like, over time, they adapted to the water, becoming sleeker and more streamlined.

By around 38 million years ago, whales had diversified into forms like the now-extinct basilosaurids, as well as into the ancestors of the two major whale groups around today.

One of these groups is the odontoceti or ‘toothed whales,’ which include animals like sperm whales, orcas, and dolphins, and the other group is the mysticeti, or baleen whales.

Modern baleen whales are among the biggest creatures to ever exist, and some researchers have assumed that filter feeding showed up early in their evolution, explaining their size and success.

And this may be true – it’s still being debated.

But, there’s also a problem with this assumption: we don’t actually know for sure when baleen evolved.

Part of the problem is that, while the first fossil mysticetes appeared about 36 million years ago with Mystacodon, the oldest direct evidence of baleen doesn’t show up until much, much later.

It comes from around 11 million years ago, from a site in Peru with about three dozen well-preserved whales.

Baleen, it turns out, doesn’t fossilize well.

Teeth, on the other hand, do fossilize well, but baleen is not a whale’s teeth.

Instead, baleen is made up of keratin — the same substance that’s found in hair and fingernails — and grows directly out of the whale’s gums.

It’s basically like a big mustache inside a whale’s mouth.

And being made of keratin means it’s less resistant to decay than teeth are, and it also tends to detach from a whale’s mouth soon after death – so it’s very rare for baleen to fossilize.

Instead, scientists look for other anatomical clues that are associated with it, like small holes in the upper jaw of ancient whales that would have brought nerves and blood vessels down to the baleen plates., But these features can be pretty variable and hard to interpret.

So, over the past few years, paleontologists have taken another look at species once thought to have had baleen and have also found new mysticete fossils with distinctive feeding strategies.

Take Mystacodon.

In 2019, researchers suggested that its remains held clues about how the creature ate — and it wasn’t filter feeding.

For one thing, while it had relatively large back teeth, its front teeth were small.

Its teeth also showed signs of abrasion, as if they were regularly scratched up by something rough, like sand or sediment.

And it doesn’t have the grooves on the roof of the mouth we’d expect to see if it had baleen, too.

It also had a relatively short snout compared to other whales at the time, but with a bigger mouth overall by volume and powerful throat muscles.

This suggests that it could have been capable of what’s known as suction feeding — capturing prey by essentially sucking it up out of the water.

And its forelimbs were odd, with powerful shoulders and robust digits that may have helped it forage for prey on the bottom of the ocean.

All this suggests that Mystacodon actually hunted around the seafloor, sometimes eating creatures that lived in the sediment, sucking them - and some grit - up as it ate.

And fossils have been found of other species that may have eaten larger prey or possibly used their teeth, rather than baleen, to filter feed, similar to today’s crab-eater seals.

This means that early toothed mysticetes may have had a variety of morphologies probably related to diverse niches.

They also varied in body size, from relatively tiny, like Fucaia, which was only about 2 meters long, to large, like Llanocetus, which could be 12 meters long.

If this is all true, and none of these whales had baleen at this point, it suggests that bulk-filter feeding wasn’t the adaptation behind the diversity and gigantism we see in baleen whales.

So what else might explain it?

Well, the answer might be bigger even than baleen whales: changes in the ocean.

Because roughly 34 million years ago at the start of the Oligocene, Earth began cooling down and the precursor to the modern Antarctic Circumpolar Current appeared as Antarctica separated from South America and Australia.

The Antarctic Circumpolar Current is a powerful current that circles the pole, churning the ocean and pulling nutrients up from the bottom to the surface.

This would have made the ocean more productive, potentially opening up new resources and niches, possibly leading to the explosion in whale diversity.

But how does the evolution of baleen fit into all of this?

One idea that’s been around since Charles Darwin proposed it, is that early mysticetes may have used their teeth as a sieve, like Llanocetus might have done .

.

.

Another line of thinking is that ancient whales may have had both teeth and baleen at the same time.

But it’s not clear how the two different feeding strategies would have worked together in one animal.

A third hypothesis that’s gained attention recently actually touches on what Mystacodon may have been doing — suction feeding.

In 2018, scientists described a completely new species of 33-million-year-old whale — about 3 million years younger than Mystacodon — found in Oregon.

They called it Maiabalaena and it also didn't have the holes in its jaws that scientists would’ve expected to see if it had baleen, but didn’t have any teeth, either.

What it had instead was a large and robust hyoid apparatus.

This is a set of bones in an animal's throat with muscles attached.

It’s thought that Maiabalaena could have used this to create strong suction forces as it fed.

And a third fossil – the 27-million-year-old whale Mammalodon – also seems to have adaptations for suction feeding.

So it’s possible that suction feeding may represent the transition between a bite-and-hold kind of feeding strategy and the bulk filter-feeding we see later with modern baleen whales.

This would’ve meant that teeth were no longer under selective pressure.

And based on genetic analyses of modern whales, we can see that the genes for teeth are still present, but they’re non-functional.

Eventually, these suction feeders developed baleen as an outgrowth of their gums.

It may have started as individual structures that later merged to form plates of baleen or it may have come from keratinized gums like what we see today in Dall’s porpoise.

These look like bumpy, rigid growths in between their tiny teeth, which help the porpoise grab onto slippery squid.

As for why they went from suction feeding to filter-feeding, it might come back to those ocean changes.

A highly productive ocean may have pushed whales towards bulk feeding, since the increase in nutrients would’ve caused things like patchy blooms of plankton.

In these conditions it would be advantageous for animals to bulk feed rather than try to pick and choose individual prey.

By around 28 million years ago, this explosion of diversity seems to have leveled off, with both bulk filter-feeding mysticetes, as well as toothed ones with their own feeding strategies, sharing the oceans.

But by about 23 million years ago, we see a shift towards filter-feeding being the dominant strategy.

We’re not sure why, but it may have to do with the toothed mysticetes facing increasing pressure from their cousins, the odontocetes, who could echolocate.

New competitors like seals and sea lions had also appeared on the scene.

Then, around 3 million years ago, many of the smaller mysticetes that lived close to the shore disappeared, probably due to changes in the climate related to glaciation.

The near-shore environment would have become less stable over time, while changes in the poles made longer migrations more important to the life strategy of whales.

And larger, ocean-going species would’ve been better able to complete these migrations.

So the picture these hypotheses paint is of a group of animals that didn’t become big and successful off of a single evolutionary innovation.

Instead, they may have taken advantage of changes in the ocean and new food resources to diversify, with a few losing their teeth in favor of suction feeding and then filter feeding.

Then, over time, competition and changes in the Earth’s climate winnowed the group down to just the ocean-going, baleen giants we know today.

Baleen whales show how tricky it can be to trace evolutionary transitions, even when we know the beginning and the endpoint.

And this is still an area of active research.

They also remind us that when we see animals with certain adaptations — like no teeth and baleen — it’s easy to assume that they’re related or developed at the same time, but this isn’t always the case.

Sometimes, the history of life on Earth is weirder than we expect – and that’s what