Victorian health authorities say it's still too early to say if the worst of the pandemic is behind us. But the state Oppositions says the time has come to ease restrictions on elective surgery.