The much anticipated ruling, which will likely be appealed to the state's highest court, would make California the second state in the United States to legalize gay marriage after Massachusetts.
San Francisco Mayor Gavin Newsom ordered city officials to provide marriage licenses to same-sex couples last year in defiance of state laws banning the practice. The California Supreme Court ruled that Newsom had exceeded his authority in allowing the licenses, but asked a lower court to consider the broader issue.
"The denial of marriage to same-sex couples appears impermissibly arbitrary," Kramer said in his ruling. "Simply put, same-sex marriage cannot be prohibited solely because California has always done so before."
The judge wrote that creating benefits for same-sex couples without allowing marriage was not sufficient and referred to the landmark 1952 civil rights decision of Brown v. Board of Education, Reuters reported.
"The idea that marriage-like rights without marriage is adequate smacks of a concept long rejected by the courts: separate but equal," he wrote.
Robert Tyler, an attorney with the conservative Alliance Defense Fund, said the group would appeal Kramer's ruling, according to the Associated Press.
A pair of bills the California Legislature is considering would put a constitutional amendment banning same-sex marriage on the November ballot. Thirteen other states approved such an amendment last year, essentially taking the issue out of the control of lawmakers and the courts.
California voters approved a measure in 2000 defining marriage as a union of a man and a woman.