Circuit means a loop, which is what lawyers, particularly in the old West, used to do. They would travel from court to court in their state, arguing cases for a few days in one town, and then in another. This was called riding the circuit. So that explains it in the state sense (at least I hope that explains it). President Lincoln used to ride the circuit in Illinois (before he became President, of course). See: http://www.papersofabrahamlincoln.org/narrative_overview.htm
For Federal court, so far as I am aware, the circuits just refer to areas of the country. So the Third Circuit, for example, contains Delaware, New Jersey, Pennsylvania and the US Virgin Islands. See: http://www.legalengine.com/fedcaselaw/caselawfederal3.htm
There are twelve Federal circuits. Eleven are numbered one to eleven, plus there is a DC Circuit that only covers the District of Columbia.
The lowest Federal court is the District Court. Each state has at least one District Court and a lot of larger states have several. But if you lose in the District Court, and you want to appeal to a higher authority, you appeal to that area's Circuit Court. So you see that in Pennsylvania you might start off in the Eastern District Court, then appeal to the Circuit Court if you lost at the lower level. Above that is the US Supreme Court. Above that is nothing.
There is a difference between Federal and State court. State courts handle what are (to use a very vague term) more local matters. Car accidents that happen within one state, where all of the parties are from that state, are state matters. A construction accident at a factory incorporated in the same state where the accident occurred is most likely to be a state matter. A criminal case, so long as there are no Constitutional questions, and so long as everything happened in one state, is a state matter. There are exceptions and more details and a lot more types of cases are solely state matters, but that's the general gist of it.
Federal cases are less local and often involve incidents that can be connected to more than one state. For example, I live in Massachusetts. If I drive to nearby New Hampshire and get into a car accident there, if the amount of damages is high enough, the case can be brought in Federal court. Federal courts also handle cases brought under the US Constitution (such as civil rights cases) or under Federal laws, such as antitrust matters.
Cases that have exhausted their state remedies aren't automatically transferred to Federal court. When a criminal case ends up in Federal court, it's because there was a Federal law violated (such as the Anti-Racketeering laws) or a Constitutional right was violated (such as the prohibition against unlawful search and seizure). But most purely state cases cannot be heard in Federal court (an exception is if there is a significant conflict between states' decisions, this doesn't happen often). So, when people say they're going to "take a case all the way to the Supreme Court", they are often mistaken. It just doesn't work that way. If every case could go to the US Supreme Court, they'd be hearing cases around the clock.
I hope this explanation didn't raise more questions than it answered. This was a very good question; thank you for asking it.