Defeating Cooney

On June 11, 1982, Holmes defended his title against Gerry Cooney, the undefeated #1 contender and an Irish-American. The lead-up to the fight had many racial overtones, with promoter Don King and others hyping Cooney as the "Great White Hope." Holmes said that if Cooney wasn't white, he would not be getting the same purse as the champion (both boxers received $10 million for the bout). Although Cooney tried to deflect questions about race, members of his camp wore shirts that said "Not the White Man, but the Right Man." In their fight previews, Sports Illustrated and Time put Cooney on the cover, not Holmes. Boxing tradition dictates that the champion be introduced last, but the challenger Cooney, was introduced last.

The bout was held in a 32,00 seat stadium erected in a Caesar's Palace Parking lot, with millions more watching around the world. After an uneventful first round, Holmes dropped Cooney with a right in the second. Cooney came back well in the next two rounds, jarring Holmes with his powerful left hook. Holmes later said that Cooney "hit me so damned hard, I felt it, boom, in my bones." Cooney was tiring by the ninth, a round in which he had two points deducted for low blows. In the tenth, they traded punches relentlessly. At the end of the round, the two nodded to each other in respect. Cooney lost another point because of low blows in the eleventh. By then, Holmes was landing with ease. In the thirteenth, a barrage of punches sent Cooney down. He got up, but his trainer, Victor Valle, stepped into the ring and stopped the fight.

After the fight, Holmes and Cooney became close friends.

Trouble with the WBC

Holmes's next two fights were one-sided decision wins over Randall "Tex" Cobb and ex-European champion Lucien Rodriguez. On May 20, 1983, Holmes defended his title against Tim Witherspoon, the future WBC and WBA heavyweight champion. Witherspoon, a six to one underdog and with only 15 professional bouts to his name, surprised many by giving Holmes a difficult fight. After twelve rounds, Holmes retained the title by a disputed split decision.

On September 10, 1983, Holmes successfully defended the WBC title for the sixteenth time, knocking out Scott Frank in five rounds. Holmes then signed to fight Marvis Frazier, son of Joe Frazier, on November 25, 1983. The WBC refused to sanction the fight against the unranked Frazier. They ordered Holmes to fight Greg Page, the #1 contender, or be stripped of the title. Promoter Don King offered Holmes $2.55 million to fight Page, but the champion didn't think that was enough. He was making $3.1 million to fight Frazier and felt he should get as much as $5 million to fight Page.

Holmes had an easy time with Frazier, knocking him out in the first round. The following month, Holmes relinquished the WBC championship.

IBF heavyweight champion

Despite his no longer being recognized by the WBC as champion, Holmes was still regarded as the lineal champion as well as being recognized as world champion by The Ring. On December 11, 1983, the newly formed International Boxing Federation extended recognition to Holmes, and he accepted.

As 1984 began, Holmes and Gerrie Coetzee, the WBA champion, were signed to unify the titles on June 15, 1984, at Caesars Palace. The fight was being promoted by JPD In., but it was canceled when Caesars Palace said the promoters failed to meet the financial conditions of the contract. Holmes was promised $13 million and Coetzee was promised $8 million. Even after cutting the purses dramatically, they still couldn't come up with enough financial backing to stage the fight. Don King then planned to promote the fight but Holmes lost a lawsuit filed by Virginia attorney Richard Hirschfeld, who said he had a contract with Holmes that gave him right of first refusal on a Holmes-Coetzee bout. Holmes then decided to move on and fight someone else.