ACT's caucus is today considering whether any of its MPs should be ministers in John Key's Government, but whatever happens former leader Rodney Hide is set to be stripped of his roles.
Dr Brash will meet Mr Key at 4pm today to discuss who if anyone will take on the roles.
Mr Hide stepped down as ACT leader on Thursday to allow former National leader Don Brash to take the reins with the aim of getting a better result in November's general election.
Mr Hide told NZPA then that Dr Brash was OK with him continuing to hold the local government, regulatory responsibility and associate education portfolios.
Dr Brash has since indicated he would like Mr Hide to leave Parliament altogether, describing his brand as "toxic".
Prime Minister John Key told reporters this morning he hadn't given any thought to the idea Dr Brash might decide to have no ministers in his caucus.
If that happened he would look at allocating the roles to existing ministers.
Dr Brash isn't an MP but is effectively picking ministers for the Government, but Mr Key said his influence only went so far.
"He's not dictating to me. What he is doing is dictating to his own caucus and he's the elected leader now of his caucus."
While he worked constructively with minority party leaders, Mr Key said he made decisions about ministerial appointments.
"They don't have carte blanche I ultimately decide if I have confidence in someone or I don't."
ACT deputy leader John Boscawen holds consumer affairs and associate commerce. The other three MPs are Heather Roy, Sir Roger Douglas and Hilary Calvert.
Mr Key reiterated he wouldn't have Sir Roger as a minister and said he didn't expect Dr Brash would recommend Ms Calvert because she was a new MP.
"I know that he's considering his options between the appointment of either John Boscawen, Heather Roy or Rodney Hide or potentially having no ministers at all," Mr Key said.
"He's made it quite clear he intends to support us in confidence and supply so at the end of the day from the Government's perspective that's the ultimate test about whether we can pass our budget or not... We're happy either way."
Dr Brash ran a racially divisive election campaign in 2005 and has right-wing policies which Mr Key has described extreme.
Mr Key is not planning to be photographed with Dr Brash after today's meeting.
"This is a working meeting, not a photo op."
Today's caucus will also decide if Mr Boscawen remains deputy leader - the deputy is likely to head the party in Parliament.
Labour leader Phil Goff said it was strange for Dr Brash to be a leader outside Parliament.
"Well I think New Zealanders will raise their eyebrows about this. Here's a man who took over the leadership of the ACT Party when he was still the member of another party.
"It doesn't look good -- people outside of Parliament pulling the strings to the extent of determining the composition of the Government's Cabinet. That's kind of unbelievable, but hey, that's what we're getting from the Key-Brash leadership."
Mr Goff said Dr Brash would force National to the right and that could be good for Labour.
"I think there's a lag effect in these things (polls), but I think people will look at the prospect of what a Brash-Key government would bring, and they'll think twice about whether that's what they want from New Zealand."
NZPA