The dramatic move came before Andersson was fully in office as she had not yet had counsel with the King, a spokesperson told CNN.
Her resignation follows a budget defeat in parliament Wednesday, Sweden's Twitter account added, with lawmakers supporting the opposition's bill.
The Green Party has also decided to leave the minority coalition government with Andersson's Social Democrats, it said. "The current government will remain as an interim government until a new government is in place," it added.
Andersson, 54, said she had told the speaker of Parliament she hoped to be reappointed as prime minister at the head of a "single-party, Social Democrat government," Reuters reported.
She replaced Stefan Löfven, who recently stepped down as both the country's premier and as leader of the Social Democrats party.
All the other Nordic countries -- Finland, Denmark, Norway and Iceland -- have previously elected female national leaders.
As prime minister, Andersson was preceded by 33 men. She previously worked as Deputy Director General of the Swedish Tax Agency, according to her CV on the Swedish government website.
She holds a master's degree in Economics from the Stockholm School of Economics and has served as Sweden's finance minister since 2014.
She is also the second woman to head up the center-left Social Democrats party, according to Sweden's Twitter account.
Andersson won parliamentary approval after reaching a last-minute deal with the former communist Left Party, but her grip on power was tenuous because of the Nordic country's fragmented political landscape.
Her predecessor Löfven governed by performing a complex juggling act to secure support from both the Left and Center parties in parliament, even though they are not part of the coalition government.
But the Center Party was worried by the deal with the Left Party and had said it would not back Andersson's government in a vote on a finance bill proposed by three opposition parties.
"We cannot support a budget from a government which is moving far to the left, which we think the incoming government is doing," Center Party leader Annie Loof told reporters.
Löfven, who stepped down earlier this month to give Andersson a chance to boost support for the party before a general election in September next year, said he would not continue if he lost the budget vote.
Whoever leads Sweden will face significant challenges. Gang violence and shootings blight life in many suburbs of the capital, Stockholm, and other major cities.
The Covid-19 pandemic exposed gaps in the much-vaunted welfare state, with the death rate in Sweden much higher than in neighboring Nordic countries.
The government also needs to speed up the shift to a "green" economy if it is to meet its climate change goals.