According to Johnson's 1927 autobiography, he married Marry Austin, a black girl from Galveston, Texas, in 1898. No record exists of this marriage, and the 1900 census shows him still living at home with his parents and siblings. Although they were probably never legally married, Johnson introduced Austin as his wife wherever he went. She was the first "Mrs. Jack Johnson," but far from the last. Austin came with Johnson when he moved to California in 1901, but stayed behind after he returned from a training visit to Colorado in 1902.
While living in Philadelphia during the summer of 1903, Johnson met a black prostitute named Clara Kerr at a North Philadelphia whorehouse. In September of that year, she traveled with him to Bakersfield, California, and was his companion for much of the next three years. In October of 1906, she ran off with an old friend of Johnson's, a racehorse trainer named William Bryant. She took most of Johnson's jewelry and clothing as well. Johnson tracked the couple down to Tucson and had Kerr arrested on burglary charges, but Johnson and Kerr reconciled for a time. They moved to California, but Johnson was having trouble finding moneymaking fights, and when the money ran out Kerr left him again this time for good.
During a three-month tour of Australia in early 1907, Johnson had a brief romance with Alma "Lola" Toy, a 20-year-old white woman from Sydney. The Sydney sporting world was scandalized, and Toy later won a 500 pound libel judgment from The Sunday Times Newspaper Company for an article published in its sporting paper, The Referee.
After returning from Australia, Johnson began traveling with white women exclusively, writing in his 1927 autobiography that "the heartaches which Mary Austin and Clara Kerr caused me led me to forswear colored women and to determine that my lot henceforth would be cast only with white women." In actuality he forswore no one he continued to pursue both black and white women for the rest of his life. But from that point on only white women would be "Mrs. Jack Johnson."
He met the next "Mrs. Jack Johnson" on August 8, 1907. Hattie McClay, whose real name was Anna Peterson, was a prostitute working in a "call house" in Manhattan. She quickly became Johnson's most frequent traveling companion, but soon learned that the world was not ready for a white "Mrs. Jack Johnson." Upon returning to North America after the Tommy Burns fight in Sydney, Australia, the finest hotel in Victoria, British Columbia, refused the couple a room. And Johnson's hometown of Galveston cancelled a planned homecoming parade after it was learned that he was traveling with a white woman. McClay eventually lost "Mrs. Jack Johnson" status to Belle Schreiber, and later Etta Duryea, but Johnson and McClay continued to see each other off and on. When they finally parted company in February 1911, he paid her $500 for the return of all his letters and telegrams.
In March 1909, Johnson visited the Everleigh Club, an exclusive all-white bordello in Chicago's Levee District, with his manager George Little, who sidelined by collecting protection money from houses in the district for two corrupt city aldermen. Johnson, who had desired entry into the club for months, didn't have a chance to do more than just look around. But he convinced five of the girls to go for a ride with him in his car. One of them was named Belle Schreiber, the 23-year-old daughter of a Milwaukee policeman who preferred the "sporting life" to the secretarial work for which she'd come to Chicago three years earlier. The madams at the Everleigh Club fired her (and at least four other girls) after she slept with Johnson, but she was soon Johnson's new favorite, traveling with all expenses paid, plus "a little over to have in my pocketbook." But after Etta Duryea entered the picture in the fall of 1909, Schreiber was soon relegated to second-tier status. She left him twice after violent quarrels, but she had trouble finding other work. Madams fired her, or refused to hire her in the first place, because she'd been Johnson's lover. In October 1910, Johnson helped her open her own place of business wiring her money for train fare from Pittsburgh to Chicago, then paying the first month's rent and buying all of the furniture for her own brothel. But two years later federal investigators tracked her down to a whorehouse in Washington D.C., and she became the prosecution's chief witness in Johnson's Mann Act trial.
In October 1909, while attending the Vanderbilt Cup car race on Long Island, Johnson had a chance meeting with a 28-year-old Brooklyn society woman named Etta Duryea. She was glamorous and well-educated, and played the piano and sang, but was prone to depression. According to the New York World Duryea was "elegantly dressed and slender, with dark hair, large dark eyes, and a lovely sad smile." She'd quickly become a favorite among white "sporting men" in New York since separating from her husband two years earlier. Before long, she was Jack Johnson's favorite. But Belle Schreiber and Hattie McClay resented her, fearful that they would soon be displaced. For her part, Duryea expected better treatment than the other women in Johnson's life and expected faithfulness. She could not handle Johnson's continued infidelity, his abusive behavior and the hostile reaction of the public, and her bouts of depression gradually deepened.
In December of 1910, Johnson began to suspect Duryea of having an affair with his chauffeur, a Frenchman named Gaston Le Fort, and hired a private investigator to have her followed. On Christmas day, Johnson confronted her and beat her so badly that she was hospitalized. Somehow the couple reconciled, and they blamed her injuries on a fall from a streetcar. They were quietly married in Pittsburgh less than a month later. Things seemed stable for a while. Johnson had helped Belle Schreiber go into business for herself, and he'd paid off Hattie McClay to keep her quiet and out of his life. But in January 1912, Duryea's father died, and shortly afterwards news of her marriage to Johnson made it back to Brooklyn. Her isolation and depression deepened even further over the following months, and she attempted suicide twice. On September 11, Johnson tried to send her on a vacation to Las Vegas with a friend. At the last minute, she said that she wasn't feeling well enough to go. Johnson ran to the train station to change the tickets and returned to his Chicago nightclub, the Café de Champion, to find Duryea dying of a self-inflicted gunshot wound in their upstairs apartment.
In the summer of 1912, Jack Johnson met Lucille Cameron, an 18-year-old prostitute from Milwaukee who visited the Café de Champion with a friend. He soon hired her as his "stenographer," but less than a month after Etta Duryea's funeral she was seen in public on Johnson's arm. In October, Cameron's mother went to the police and charged Johnson with kidnapping her daughter. She told the press, "Jack Johnson has hypnotic powers, and he has exercised them on my little girl. I would rather see my daughter spend the rest of her life in an insane asylum than see her the plaything of a nigger." On October 18, Johnson was arrested for violating the Mann Act, but Cameron refused to cooperate and the case fell apart. Less than a month later, Johnson was arrested again on Mann Act charges. On December 4 less than three months after Duryea's suicide Johnson and Cameron were married, an act that outraged the public.
After his Mann Act conviction in June 1913, Johnson skipped bail. He appeared in Montreal on June 25, where Lucille was waiting for him. For the next seven years, they lived in exile in Europe, South America and Mexico, until Johnson surrendered to the authorities on July 20, 1920. When Johnson was released from the federal prison at Leavenworth, Kansas, on July 9, 1920, Lucille was waiting. Four years later, she filed for divorce on the uncontested charge of infidelity.
The fall after divorcing Lucille Cameron, Johnson met two middle-aged women, Irene Pineau and Helen Matthews, at the race track in Aurora, Illinois. The following February, Pineau divorced her husband, and she and Matthews began seeing Johnson together. Pineau emerged as Johnson's favorite, and the two were married in Waukegan in August 1925. She remained at Johnson's side for the rest of his life. In December of 1932, they sailed for Europe. Johnson staged a series of exhibitions in Paris, but his plans to open a boxing school in Berlin were scuttled by Hitler's rise to power in January 1933. When Johnson was killed in an automobile accident in June of 1946, Irene buried him in Chicago, in a grave next to Etta Duryea. At the funeral, a reporter asked her what she had loved about Johnson. "I loved him because of his courage," she replied. "He faced the world unafraid. There wasn't anybody or anything he feared."