tenant won't allow landlord to show house to prospective tenants

Did they sign an agreement allowing you to enter?

Spot on.

Every good lease agreement should have reasonable access built into it.
I lease two flats, and had the very same problem with one of them a few months ago. I reminded the tenant that he risked losing part of his deposit if he denied me reasonable access to show a prospective tenant around, and he finally agreed.
Begrudgingly, but he agreed all the same, and all because of the lease condition.

Check the written agreement.

An important inclusion if you want to show the place after notifying tenant of impending eviction.

Where I live, I have to give the tenant advance written notice to enter the premises. There is nothing for them to sign. I also have to have a valid reason as to why I want to enter the premises.

How specific does that notice have to be in regard to time. Wednesday at 2:30 p.m., or just sometime in a two or three day span?

I believe it has to be 72 hours.

I'm new to owning a rental.

We have several rentals here in Florida and State laws very. Here you have to give a 24-hour notice in cases where a tenant gives notice to vacate the property. At least that’s what the property management company we use has told us.

Check to see what your legal rights are in your state. I know (at least at one point) when I was renting, that legally I had to allow my landlord reasonable access for him to show propective future renters once I gave him my notice I was no longer going to rent. At that point I was no longer in a lease (the lease had expired and was simply renting month to month).

There was certain stipulations in it - like 24 hours notice and things like - I do not remember all the details as it was so long ago, but I would imagine there would be some legal protection allowing the landlord access for such reasons. Usually there are laws protecting both the renter and landlord.

I did a quick search and found this (although this is CA - it also provides some general info on all states):

2. What are examples of situations when a landlord may enter, but only after giving the tenant reasonable notice?
Typically, a landlord has the right to enter rented premises after giving tenants reasonable notice in order to make needed repairs (or assess the need for them) and to show the property to prospective new tenants or purchasers. In addition, a landlord may enter rented premises in instances of abandonment (that is, when the tenant moves out without notifying the landlord) or by court order. A landlord may not enter just to check up on the tenant.

3. Assuming it is not an emergency, but the landlord has a valid reason to enter -- for example, to make repairs -- what kind of notice is required?
States typically require landlords to provide a specific amount of notice (usually 24 hours) before entering a rental unit. In some states, such as California, landlords must provide a reasonable amount of notice, legally presumed to be 24 hours. Landlords can usually enter on shorter notice if it is impracticable to provide the required amount of notice.

5. What are the landlord's options if a tenant refuses to allow entry even when a landlord has given adequate notice and has a valid reason to enter?
A landlord should not force entry except when there is a true emergency, such as a fire or gas leak. However, if a tenant is repeatedly unreasonable in denying the landlord access, the landlord can legally enter anyway, during reasonable times, provided he does so in a peaceful manner. However, in no case should the landlord enter if the tenant is present and saying "stay out."

7. How can I find out the specific laws on privacy in my state?
Find your state's statutes at a law library or large public library. If possible, look for the larger annotated version which will also contain brief notes as to key court decisions. Look in the index under Landlord-Tenant and then for the subheading Privacy. You may also be able to get information from a local apartment association or tenants' rights group. Your state Attorney General's Office or Consumer Protection Agency can also provide advice. Nolo Press publishes two books on the subject for California: The Landlord's Law Book, by Brown and Warner and Tenants' Rights, by Moskovitz and Warner.

http://www.tenant.net/Other_Areas/Calif/misc/nffland.html#examples

I can give my tenants written notice to vacate. If they refuse, then I have to go through the courts and start an eviction process. I have no right to force them out.

You can pretty much sue anybody over anything. It doesn't mean you will win.