A military spokesman said troops had repelled repeated attacks by the militants, led by the jihadist Islamic State in Iraq and the Levant.
Forty assailants were killed overnight and on Wednesday, he added, denying the facility was close to being overrun.
But some workers said the militants had seized most of it and that troops were concentrated around the control room.
It comes as US President Barak Obama considers a formal request from Iraq’s government for air strikes against the militants, who have seized several northern cities and towns since last week.
ISIS-led militants launched an assault on the Baiji refinery, about 200km (130 miles) north of the capital Baghdad, early on Wednesday.
Workers trapped inside said the attackers had breached the perimeter and destroyed one of its fuel tanks. Some said they had taken control of most of the facility. Video footage showed smoke billowing from the refinery and the black flag used by ISIS flying from a building.
However, officials insisted that troops had resisted the advance.
Nearly all the 15,000 workers at the refinery and 100 foreign experts left on Tuesday when the plant was shut down in anticipation of the attack.
On Thursday morning, the remaining 250 to 300 workers were evacuated under an agreement brokered by local tribal leaders, one of the workers told the Reuters news agency.
The battle over Baiji, which supplies much of the country’s domestic fuel, has sparked fears that Iraqis will soon experience long lines at petrol pumps and electricity shortages.
BBC Arabic’s Reda El Mawy reported that there were already queues at petrol stations in the northern city of Irbil and that food prices were rising.
Meanwhile, police told the BBC that ISIS-led militants had killed 13 police and Kurdish Peshmerga militiamen while capturing the village of al-Bashir, south of Kirkuk, over the past two days.
The BBC’s Richard Galpin says it seems the jihadists and their allies may be pushing towards the strategically important northern city and its nearby oil fields.
On Wednesday there were fierce clashes in the Baquba, about 60km (37 miles) north of Baghdad, as jihadists again tried to enter the city centre and seize its prison.