Junior foreign minister VK Singh has sought to deflect criticism of the government.

AFP/New Delhi

One of Prime Minister Narendra Modi's ministers was accused of stoking caste tensions in India on Thursday after two Dalit children were burnt alive in a case that has triggered street protests.

Junior foreign minister VK Singh sought to deflect criticism of the government over the sensitive case by saying the administration cannot be blamed for everything, even somebody stoning a dog.

"On everything, say even a dog getting hit by a stone, you can't blame the government. That is not the way," Singh told reporters early on Thursday.

Opposition political parties seized on the comments, accusing Singh of likening India's lowest social caste of Dalits, formerly known as untouchables, to dogs.

The comments came after a nine-month-old boy and his toddler sister were burnt alive in an arson attack on Tuesday in Faridabad district, around 40 kilometres outside New Delhi. His mother is in critical condition in hospital.

Authorities say caste violence is not the motive for the attack, but the district has since been tense, with Dalits blocking a highway in protest on Wednesday and police pouring into the area.

Caste violence has long blighted India where the ancient and rigid social hierarchy still holds sway especially in rural areas.

Facing a mounting backlash, Singh, a former army general, late Thursday rejected the criticism, saying his comments about dogs and the tragic deaths were unrelated.

"What I said was very clear that please do not blame the government for everything local that happens," Singh told India Today TV.

"If by any stretch of imagination, (though) it was not intended...if someone has felt hurt because someone has maliciously twisted my sentences then I have no qualms in apologising," he added.  

The opposition Congress party demanded his resignation, branding the comments shameful and immoral.

Congress leader Rahul Gandhi has accused the government of not doing enough to protect the poor and those from the lowest castes who suffer prejudice and violence.

Three people have been arrested over the attack in which the family's home was doused with petrol and set alight.  

Local authorities have said the case was related to a long-running feud between the victims' family and members of the higher Rajput caste, which also claimed the lives of three people a year ago.