Sharif's surgery comes as his government prepares to present its annual budget on Friday. It also remains under pressure over allegations of corruption linked to the so-called Panama Papers.
Sharif "was in high spirits" when he went in to the operating theatre at about 8:00 am, his daughter, Maryam, said on her Twitter account. She did not say how long the surgery was expected to take.
The operation was for a "perforation of the heart", a complication from a 2011 procedure, Maryam said in a Twitter post last week.
Sharif, 66, was prime minister for two terms in the 1990s before being overthrown in a 1999 military coup.
After years in exile, he returned to Pakistan in 2007 and led his party to a victory in a 2013 election.
Sharif has been accompanied to London by his brother, Shahbaz Sharif, who is chief minister of Punjab province, and several other family members and aides.
He has travelled to London for medical treatment several times over the past year.
On Monday, Sharif telephoned his Indian counterpart, Narendra Modi, thanking him for his wishes for a quick recovery, the Pakistani Foreign Office said in a statement.
Sharif made a bid to improve ties with old rival India a main policy in his 2013 election campaign, though progress has been slow.
Sharif has been overseeing state affairs in the days leading up to the surgery, and on Monday addressed an economic meeting, signing off on budget proposals that include a target of 5.7% growth in the year beginning in July.
Pakistan missed its gross domestic product growth target of 5.5% for the year ending in June, hitting only 4.7%.