Starting qb rules

Also whatever the answer may be is there an nfl rule that supports this?

Yes, he is still the starting quarterback. And the starting running back is still the starting running back even if he doesn't run on the first play. And the starting receiver is still the starting receiver even if he doesn't receive...

Do you know they have a starting kicker?

I don't think "starting quarterback" is an officially designated NFL position nor does it have any real significance other than to let people know who is currently the top dog. It's just something the team announces.