Tombs, waterworks and a Roman gold coin are traces of the existence of a Roman settlement on Port-Cros Island. In the fifth century, the monastery of Lerins had a significant branch on the island of Levant, this branch had an annex in Port-Cros in the Vallon de Notre Dame. This monastery was destroyed by Barbarian pirates who swept through the islands between the tenth and the sixteenth centuries.
En 1531, François 1er, lors d'une visite à Hyères, fut alerté par les habitants sur les risques que faisaient courir les pirates à cette portion de côte. Il érigea en marquisat les trois îles de Bagaud, de Port-Cros et du Levant et confia, le 13 février 1532, cette charge à Bertrand d'Ornezan avec obligation de construire et d'entretenir des fortifications.
In 1549, Christophe Rocquendorf became governor. To attract settlers he gave the right to asylum to convicted criminals. The island was quickly invaded by individuals whose behavior caused more problems than that of the pirates they were expected to fight.
In 1617, the forts that were supposed to defend Port-Cros were still not built. Under the influence of Cardinal Richelieu, the Tour de l'Eminence, the Fort de l'Estissac and the Fort of Port Man were built. King Louis XIV fought the acts of piracy and banditry commited by islanders and pirates but there never was any efficient garrison on the island, which made it possible for the English to pillage and plunder it in 1700 and to invade it in 1742, before the Earl of Maurepas fought them to regain control of the island.
The last Marquis de Port-Cros was Louis Colvet, Mirabeau's father-in-law. He sold the three islands to Jean Joseph Barthélémy Simon of Savornin in 1783. In 1793, the British devastated Port-Cros Island again. Then Emperor Napoleon 1st became interested in Port-Cros, to which he referred to as a strategically important anchoring place. He restored the defenses in 1811 and installed a garrison. In 1812 it had 1 000 to 2 000 men. After the fall of the First Empire, it was replaced by a company of discharged or unfit for service soldiers.
Bagaud Island was purchased by the State in 1815. Port-Cros Island and Le Levant Island belonged to Mr. Gazzino and Rolland, and then to the Earl of Retz. He sold Port-Cros to Mr. Bourgarel. Over the centuries the island had different owners, and in the early twentieth century it was run by Dr. Crott who tried in vain to convert it into a rich American tourist resort. The island was purchased by Marcel Henry,a sollicitor, and Claude Balyne in March 1921. These owners developped tourism: a hotel was built on the island and and shipping links with the mainland developed. Between the two world wars, artisits and writers visited the island: André Malraux, André Gide, Saint-John Perse, Paul Valéry and Jules Supervielle who made the Fort du Moulin his family home. Eugène-Melchior , Viscounte de Vogue, a French writer born in 1848, also stayed in Port-Cros (he introduced to the French the great Russian novelists, especially Tolstoy, Turgenev, Dostoevsky and joined the French Academy for writers in 1889).
On 15 August 1944, during the allied troops landing in Provence, the island saw intensive fighting between the German garrison (150 men) and French commandos. The last bastion would fall on 17 August.
Mrs. Henry, who died in 1966, bequeathed the island to the State, with the exception of the hotel (The Manor), which her great-nephew Peter Buffet inherited,. In 1963, the creation of the national park of Port-Cros was the latest turning point in a rich history. Yann Arthus-Bertrand, a photographer, has signed a long lease (Emphyteutic lease) with the National Park regarding the Fort of Port-Man which he is restoring.