I'm curious how you define "damage to the ecology". How do you decide that one ecosystem is "better" than another, and that transforming one into the other constitutes "damage"?
Well, I am not sure that you can make such a decision in absolutist terms, however I'll give you some ideas.
Oz has immensely old and fragile soils.
One of the major effects of the kind of damage feral grazing animals do (human over-grazing and over-cropping has done terrible damage too) is to remove the vegetation that anchors this fragile soil. They are not adapted to Oz plants, and they tend to do stuff like wallow, which destroys the vegetation, eat to the roots, and pull roots out, eat young bushes and trees so that none can grow to replace them in the system. Their hooves also destroy the vegetation, leaving soil bare.
What this means is that Oz just keeps losing what was already an incredibly thin topsoil. This means land gradually becomes unable to support life.
The loss of trees (again, humans were the main culprits) means that the salt table rises.
Without vegetation to slow its progress, when it rains, enormous amounts of soil are lost.
Along with global warming, this is affecting the ability of Oz to produce food, (not taking into account that some folk regard the life of the eco-system as a good in itself, and that destroying countless millions of living creatures is not a great thing)
Also....again this is no absolute....if we take it as a given that driving species to extinction is not a great thing to do:
Cats have decimated the biodiversity where they have spread.
Destroying the capacity of the land to support vegetation obviously destroys plant species...but it also means the loss of many other forms of life.
These impact upon each other...so that the loss of one seemingly unimportant insect, for instance, can mean the loss of a whole group of plants, which then again means that a range of beasties die out, too.