The basal penguins lived around the time of the Cretaceous–Tertiary extinction event somewhere in the general area of (southern) New Zealand and Byrd Land, Antarctica. Due to plate tectonics, these areas were at that time less than 1,500 kilometers (932 mi) apart rather than the 4,000 kilometers (2,485 mi) of today.

The most recent common ancestor of penguins and their sister clade can be roughly dated to the Campanian–Maastrichtian boundary, around 70–68 million years ago. What can be said as certainly as possible in the absence of direct (i.e., fossil) evidence is that by the end of the Cretaceous, the penguin lineage must have been evolutionarily well distinct, though much less so morphologically; it is fairly likely that they were not yet entirely flightless at that time, as flightless birds have generally low resilience to the breakdown of trophic webs that follows the initial phase of mass extinctions because of their below-average dispersal capabilities (see also Flightless Cormorant).

The basal fossils
The oldest known fossil penguin species is Waimanu manneringi, which lived in the early Paleocene epoch of New Zealand, or about 62 million years ago. While they were not as well-adapted to aquatic life as modern penguins, Waimanu were generally loon-like birds but already flightless, with short wings adapted for deep diving. They swam on the surface using mainly their feet, but the wings were – as opposed to most other diving birds, living and extinct – already adapting to underwater locomotion.

Perudyptes from northern Peru was dated to 42 million years ago. An unnamed fossil from Argentina proves that by the Bartonian (Middle Eocene), some 39–38 million years ago, primitive penguins had spread to South America and were in the process of expanding into Atlantic waters.

Palaeeudyptines
During the Late Eocene and the Early Oligocene (40–30 million years ago), some lineages of gigantic penguins existed.

Nordenskjoeld's Giant Penguin was the tallest, growing nearly 1.80 meters (6 ft) tall. The New Zealand Giant Penguin was probably the heaviest, weighing 80 kg or more. Both were found on New Zealand, the former also in the Antarctic farther eastwards.

Traditionally, most extinct species of penguins, giant or small, had been placed in the paraphyletic subfamily called Palaeeudyptinae.

Size plasticity seems to have been great at this initial stage of penguin radiation: on Seymour Island, Antarctica, for example, around 10 known species of penguins ranging in size from medium to huge apparently coexisted some 35 million years ago during the Priabonian (Late Eocene). It is not even known whether the gigantic palaeeudyptines constitute a monophyletic lineage, or whether gigantism was evolved independently in a much restricted Palaeeudyptinae and the Anthropornithinae – whether they were considered valid, or whether there was a wide size range present in the Palaeeudyptinae as delimited as usually done these days (i.e., including Anthropornis nordenskjoeldi). The oldest well-described giant penguin, the 5-foot-tall Icadyptes salasi, actually occurred as far north as northern Peru about 36 million years ago.

In any case, the gigantic penguins had disappeared by the end of the Paleogene, around 25 million years ago. Their decline and disappearance coincided with the spread of the Squalodontoidea and other primitive, fish-eating toothed whales, which certainly competed with them for food, and were ultimately more successful. A new lineage, the Paraptenodytes, which includes smaller but decidedly stout-legged forms, had already arisen in southernmost South America by that time. The early Neogene saw the emergence of yet another morphotype in the same area, the similarly sized but more gracile Palaeospheniscinae, as well as the radiation that gave rise to the penguin biodiversity of our time.

Main source: Wikipedia