Duff and Phelps were appointed on Tuesday after Her Majesty's Revenue and Customs (HMRC) forced the issue in the Court of Session in a bid to secure payment of £9million in PAYE and VAT, a debt accrued during Whyte's tenure.
In a press conference on Thursday, joint administrators Paul Clark and David Whitehouse also revealed that a £24 million loan from Ticketus against season ticket sales did not go through the company's accounts.
On the issue of Ticketus money and HMRC, Whitehouse admitted: "That's not common practice."
Asked if he was concerned, he replied: "Yes. All financing of the business will be subject to proper investigation."
The administrators were quizzed on why £ 9million in taxes have not been paid since Whyte purchased Sir David Murray's shareholding in the club.
Clark said: "It's a difficult question to answer at this stage.
"What one needs to understand is that the monies don't get put into a separate account and therefore they are the tax man's monies.
"This is part of the funds that are involved in the club and the payments just weren't made.
"Exactly why and whose decision that was, that's not something we are able to confirm."
Whitehouse added: "They are deductions that were made at source in relation to PAYE that weren't then paid over so it has effectively been used as a funding tool for the club and, ultimately, unpaid taxes are not an appropriate long-term funding tool for the club."
Regarding the Ticketus money, Whitehouse said: "Our understanding is the funds from Ticketus didn't come through the company's accounts so we haven't got visibility on that and that's what we are trying to get."
Rangers are also awaiting the outcome of a tax tribunal, relating to payments made during Murray's ownership, that could leave the club with a bill of £75 million, according to Whyte.
A company, MCR, who was asked to provide consultancy advice in relation to Whyte's takeover, was acquired by Duff and Phelps late last year.
However, Clark did not feel that being part of a takeover and then acting as the club's administrators was unusual.
He said: "It's not unusual in my view."
He added: "There were always difficulties with the club. Of course the 'big tax case' was the big unknown and remains the unknown in terms of the final, significant item on a liability level.
"I should stress, we have never been involved with the management of the club or handling the bank account or finding out who was being paid and why they were being paid.
"The work that we were doing then is different to the work that we will need to do now."
One a day when a lot of questions appeared to be left unanswered, there was no real clarity on who has the security on the club's assets, such as Ibrox and Murray Park, with the administrators stating the sale of both was possible but unlikely.
Clark said: "On the face of the registered documents at Companies House, there is a company that Craig Whyte has an interest in that appears to be the secured creditor.
"The issue there will be whether that security is valid and, of so, for what amount."
Good news for Rangers fans concerned about the future of the Scottish champions was the administrator's view that HMRC do not want to see a situation where the club is liquidated.
Whitehouse said: "They are supportive of the process of trying to rescue the club so by definition, yes, that's correct.
"From an HMRC position, it is wrong to continue to fund a loss-making business from unpaid taxation. That position was unsustainable."
Clark added: "We are confident that, based on what we have seen, that we will be able achieve a successful result, which is that Rangers Football Club will continue.
"That's what we are working towards. Our first priority was to make sure that Saturday's match went ahead and we have been able to achieve that."
Former Rangers chairman Alastair Johnston wants the takeover of the club last May to be investigated.
He said to the BBC: "I have today written to the Crown office asking for an investigation into the background surrounding the acquisition of Rangers Football Club by Craig Whyte, and in particular whether there is evidence of fraud."
A Strathclyde Police spokesman said: "We can confirm that we have been passed information regarding the ongoing situation at Rangers Football Club.
"This is currently being examined. It would be inappropriate to comment further."