Gerson Fuentes, 27, who was arrested on Tuesday, appeared in Franklin County, Ohio, municipal court for an arraignment on Wednesday. A police investigator testified at the hearing that Fuentes confessed to raping the girl at least twice.
The victim in the case underwent an abortion in neighboring Indiana on June 30, the officer testified.
A statewide ban on abortions in Ohio after the sixth week of pregnancy – before many women are aware they are pregnant – went into effect just hours after the Supreme Court's June 24 ruling that struck down Roe v. Wade, the 1973 decision that had established a constitutional right to abortion.
The incident gained attention this month when the Indianapolis Star newspaper reported she had to travel to Indiana to secure an abortion because she was three days past Ohio's six-week limit, which does not include exceptions for rape or incest.
Abortions after six weeks remain legal in Indiana, though the Republican-controlled state legislature is expected to consider new restrictions later this month.
Biden, a Democrat, spoke of the girl in remarks to reporters about abortion access.
"Just imagine being that little girl – 10 years old," he said angrily at the White House on Friday.
Fuentes is being held on $2 million bond and is scheduled for a preliminary hearing on July 22.
A court-appointed defense attorney for Fuentes did not immediately respond to a request for comment.
In all, 26 U.S. states have either severely restricted abortions in the wake of the ruling or are expected to do so in the coming weeks and months.
Some conservatives had questioned the veracity of the Indianapolis Star story.
Republican Ohio Attorney General Dave Yost told the USA TODAY Network Ohio bureau on Tuesday that the details were "more likely than not a fabrication" because there had been no arrest and no evidence uncovered, Ohio's Columbus Dispatch reported.
But in a statement on Wednesday after the arrest, Yost, who defended the state's six-week ban in state court after the Supreme Court decision, said, "My heart aches for the pain suffered by this young child. I am grateful for the diligent work of the Columbus Police Department in securing a confession and getting a rapist off the street."