@Moghyss,
This is a classic negotiation. If this goes to court, there is a cost for both you and your neighbor (in risk, time and possible legal costs).
It is in both of your interests to keep this out of court. So negotiate in good faith and welcome your neighbor to do the same.
Rule #1 of negotiation is that you need to be willing to walk away. In this case, walking away means letting your neighbor take you to court. Of course, if this happens, you want to document that you are trying to be reasonable because that will go before the judge.
What I would do...
1. Ask my neighbor to come up with a figure (in negotiation, you always want the other person to make the first offer).
2. If the neighbor is being ridiculous. I would say "that's ridiculous" and ask them to name a reasonable figure (from #1).
3. If the neighbors number is in the ballpark of reasonable, I would make a counter offer, with a polite argument of why this is reasonable.
4. If the neighbors number is reasonable. I would accept it. Case closed.
5. If the neighbor is being unreasonable, I would do nothing. Let him make the effort to take it to court. The only thing I would do is make sure I have in writing my attempts to reach a reasonable settlement.
6. Once the neighbor took me to court, I would seek legal help. If it is small claims court (i.e. the number is less then a couple thousand dollars) the legal fees would likely be low. You need a lawyers advice (paid for by the hour). You might not need a lawyer actually present.