If you do ask/invite, something might happen.
If you don't ask, I can guarantee nothing will happen.
Be nice and casual about it.
Hey, next time you see me go by with the dog, come on out for a bit of a walk with us
don't pursue it
he already did his bit by letting you know he'd seen you in the area. you missed the natural opportunity to invite him to join you (he was pretty much asking for an invite) so you have to drop it in the next time you see him
flirting doesn't mean giggling or staring. it's about responding naturally and casually
now keep in mind, flirting isn't normally meant to lead to anything so you need to think if you really want to flirt or if you want to get to know him