You shouldn't remove the bandage to look at it. That will disturb the clot and it will bleed again. Leave it in place for at least an hour or until you go to the doctor.
As far as stitches in a finger, while working in food service in my younger years, I had several serious knife cuts on my fingers and between my fingers. It should not impede your guitaring ability unless you've cut muscle or nerve. It doesn't sound like you have done so. Getting it properly attended to will lessen the degree of permanent scarring and injury.