@Cyracuz,
You are equivocating on the word "evolving".
The evolution of our bodies is an empirical fact posited to explain what we experience. It is a conclusion of natural science, is well established, and can be used to explain predictions that are confirmed by observation. This "evolving" is not the same as "evolving" of the physical world from consciousness. In fact the use of the term "evolution" in the latter context is a misuse even though I get the matter to which I believe you intend to refer.
The positing of a world containing objects that obey physical law and evolve, and the association of our consciousness with one of these objects is not the emergence of the physical world from perception. They are not the same thing and to use the same word for them and then set up a false dilemma is a logical error.
Not: The physical world evolved from consciousness *or* consciousness evolved from the physical world. But: The physical world evolved from consciousness *and* consciousness evolved from the physical world.
For you the latter occurred sometime after your conception and probably before you were born. The former is ongoing as you read this. Basically the physical world evolved from consciousness and when we observe what we observe has the property of being an objective world in which our perception is associated with a physical object and in which that physical object is the result of evolution in the biological sense.
In the last case the word evolved is used in two senses. The first sense is a structure of being to use a loose term, the second sense is an accidental property of nature that in fact exists and is well established using the evidence collected by biologists. These facts do not contradict each other. Neither is unreal.
Material causality requires causality ex-nihlo not the other way around. If you prefer to use the term "it just is" then go ahead but understand its meaning. To put it in a way a scientist might like it to be heard "The physical theory of evolution assumes that there is a physical world and that we can make observations of it."
The fact not often acknowledged is that we do so by positioning our bodies correctly. There is no observation of Galileo's moons by you except when your eye is placed at one end of the telescope and the other end of the telescope is pointed toward Jupiter. No physical law is specified to enforce this requirement. It is assumed in physics. The telescope must point at two objects not one. This establishes the basis for the association of your consciousness with your body.
This association is purely empirical and not a necessary part of the experience of being. It is an accident of being in the medieval sense of accident. In fact, it is logically possible that it change in the future that is there is a logical possibility for the world to dematerialize and for us to remain conscious and for our conscious experience to still have content.
This logical possibility is restricted by physical law and by the association of our consciousness with our biology - it is therefore logically possible but not, we think, physically possible. (Here strictly I am assuming something beyond physical law but which is at the basis of all experimental physical science - the assumption that we can observe based on the location and functioning of our bodies.) If it (de-materialization) were to occur then the conclusion that the world is physical would have to be revised as the basis for all of the conclusions of natural science, including the material basis of our consciousness, lie upon our experiencing.