His centre-right VVD Party won 31 out of 150 seats, polls suggest.
His party came far ahead of the next three parties, including Geert Wilders’ anti-immigration Freedom Party (PVV), the Christian Democrats and the D66 Party, which each got 19 seats.
Mr Wilders’ party had been leading in opinion polls but support for the party appeared to slip in recent days.
Voter participation in the general election was high – with an estimated turnout of at least 81%.
Analysts say a high turnout may have benefited pro-EU and liberal parties.
“Today was a celebration of democracy,” Mr Rutte said, adding that the Netherlands had said no to the “wrong kind of populism”.
Although the VVD had lost several seats since the last election, many had expected the party to lose much more ground to the Freedom Party.
Many had been watching the vote in the Netherlands closely, as an indication for how populist parties may fare in other elections in EU countries.
France goes to the polls next month to elect a new president, while Germany is due to hold a general election in September.
German Chancellor Angela Merkel has called Mr Rutte to congratulate him, while Luxembourg Prime Minister Xavier Bettel has also tweeted his congratulations.
Martin Schulz, president of the European Parliament until earlier this year, said he was “relieved” Mr Wilders’ party had lost.
“We must continue to fight for an open and free Europe!” he added on Twitter (in German).
However, Mr Wilders warned that Mr Rutte “has not seen the last of me”.
He previously said that the “patriotic revolution” would continue to take place, and “the genie will not go back into the bottle”.
Pharrell Williams’ song Happy pumped out across a conference hall converted to host the victorious VVD.
“Of course he’ll still be prime minister,” a loyal party member with black rimmed glasses told us. “He’s the best man for the job.”
Entry to the gathering was invitation only. Most of the foreign press were contained in a side room. The champagne was flowing but there wasn’t much fizz. The mild-mannered, measured Mark Rutte appears to have been given a mandate.
He will say he stopped the “dominos of populism” from falling, but to do that he shifted himself to occupy the populists’ territory, talking tough on immigration and integration.
As parliamentary seats are allocated in exact proportion to a party’s vote share, the VVD party will need to go into coalition with other parties.
The VVD had ruled out a coalition with the Freedom Party – but not the other two runners-up, the Christian Democratic Appeal (CDA) party, and the Democrats 66 (D66) party, which are both pro-EU.
The CDA said it was delighted with its election result and looked forward to helping form a coalition.
The VVD will need at least three other parties before it can secure a majority.
Hence, the other smaller parties will be seen as potential power-brokers.
Exit polls suggest the Green-Left party performed strongly, winning a total of 16 seats, compared to four in the last parliament.
Meanwhile, the Socialist Party took 14 seats, while the VVD’s previous coalition partner, the Labour Party, saw its number of seats plunge from 38 to nine.
Analysts said it appeared to have been punished for its role in the coalition government, where it helped pass austerity measures.
Party leader Lodewijk Asscher called it “a bitter evening for labour – unbelievably disappointing”.
“Rebuilding the party begins today,” he said.