Two Australian journalists were detained overnight and have been barred from leaving Malaysia after they tried to "aggressively" question Prime Minister Najib Razak about a corruption scandal, police said on Sunday.

The pair were detained after they crossed a "security line and aggressively tried to approach the prime minister" who was visiting a mosque in Kuching on Borneo island, according to a police statement.

"Both of them were subsequently arrested for failing to comply with police instructions not to cross the security line," it said.

The journalists work for the Australian Broadcasting Corp's Four Corners investigative programme.

"ABC 4Corners team arrested in Malaysia last night after trying to question PM Najib Razak over corruption scandal," the programme's executive producer Sally Neighbour tweeted on Sunday.

Reporter Linton Besser and camera operator Louie Eroglu had approached Najib on the street before their arrest, the broadcaster added.

The pair were detained on Saturday night but released on Sunday without charge.

Neighbour said their passports, which were initially seized, had been returned to them but they "can't leave Malaysia".

"We will discuss with the Attorney General's Chambers (whether) to charge them," national police chief Khalid Abu Bakar was quoted as saying by Malaysian news agency Bernama.

"Police are responsible for the prime minister's security. So we do not want anything untoward happening to him," he said, adding that the journalists were barred from leaving Sarawak state while investigations were underway.

Deputy Prime Minister Ahmad Zahid Hamidi said local and foreign media would not be sanctioned for covering events but "must perform their duties according to the journalism ethics".

Scandal-hit premier  

But veteran opposition MP Lim Kit Siang said he was "horrified by the very clumsy and ham-fisted manner" in which the affair was handled.

Footage posted online by The Star showed Besser asking questions at a tense press conference in Kuala Lumpur earlier on Saturday relating to the still-murky 2006 murder of a Mongolian woman.

Two of Najib's bodyguards were convicted of the murder and sentenced to death.

Najib, who was defence minister at the time, has strongly denied any involvement in the murder and has said he did not know the woman.

But government critics have long alleged that the two bodyguards, members of an elite unit that guards top ministers, were scapegoats in the killing of Altantuya Shaariibuu, who was at the centre of allegations of massive kickbacks in the $1.1bn 2002 purchase of French Scorpene submarines.

Najib, 62, has also been under fire over allegations that billions of dollars were stolen from a state firm he founded, and over his own acceptance of a murky $681mn overseas payment.

Najib and the state firm have vehemently denied any wrongdoing but he has curbed investigations into the scandals and purged his ruling United Malays National Organisation of critics, essentially shutting off internal party challenges.

Whistle-blowers have been arrested while media outlets reporting on the allegations have been muzzled, raising concerns over rights and freedom of speech.

Leaders from across Malaysia's political spectrum joined forces on March 4 to call for the removal of the scandal-hit premier.