There a lot of things that can happen. Your main city will be on a river because of the ability to transport goods. A 500 year rainfall can flood it and destroy the businesses and homes.
Things happen that you won't be able to predict. They always do. Someone could try to corner the market on silver and drive it up which when it crashes hurts investors and drastically changes your tax revenues. The number of things that happen are far to numerous to control.
How do you propose to keep people from *******? (The obvious answer is to invest in an education system which has a proven track record of lowering birth rates, but you obviously won't borrow the money needed to do this)
Do I detect a touch of defeatism in you guys?..
Firstly, my island nation will be rolling in money from taxes, and even after paying for infrastructure there'll be ample left in my Treasury to cover any emergency that might come along.
Remember, my one million population will generate literally millions of dollars in taxes each week, and if I mishandle that, I deserve being put up against a wall and shot!
As for shady speculators who try to corner the market in silver or whatever, i'll bust their asses!
Secondly, as ruler, I'll issue a decree limiting couples to a maximum of 2 kids each (I think China also did something like that) to ensure the population never exceeds the lands ability to produce enough food to feed it, (and i'll possibly even limit it to 1 kid later)