@Kyleah,
Personally I think it best to end it without meeting with him. Normally I feel a face to face break up is best, to be considerate of the other person, but this is an affair not a dedicated relationship. Also ask yourself - are you ready to face him? How hard would it be?
And considering the fact that this is an affair to me doesn't warrant a face to face - it might be best if you are up front and say it is best that we just drop this now and then when we are both free to focus on a real relationship we could meet up and discuss.
I think you know the answer