Banks are required by law to hold reserves equal to a fraction of the deposits held by customers at their banks. Banks generally try to keep reserves to the legal minimum, because reserves (kept either as vault cash or as a ledger entry with the Federal Reserve) are idle funds, as opposed to loaned or otherwise invested funds, which make money for the bank. Obviously, money sitting in a vault doesn't do this.
So, banks are more or less treading a fine line on a daily basis. It may be that at the end of the day they need more reserves than they have. Other banks have more reserves than they need (excess reserves). Banks with excess reserves can make overnight loans to banks which are short of reserves. The interest rate on such loans is called the federal funds rate. (Strictly speaking, this is the average of the interest rates charged for such loans, which may vary from institution to institution and transaction to transaction.)
Obviously, the more excess reserves there are available to loan, the easier it is to negotiate such a loan under better terms (lower interest rate), since banks compete with each other to make such loans. Conversely, if excess reserves are in short supply, then banks can charge more for this rare commodity.
The Federal Reserve influences the federal funds rate by influencing the amount of reserves, hence the amount of excess reserves. It does this by selling or buying Treasury securities on the open market (hence the name Open Market operations).
When the Fed buys Treasury securities from a private seller (generally an institution), it does two things: (1) transfers the purchase payment into their bank account; (2) credits their bank with an equivalent amount of reserves.
Lets say that the Fed buys a hundred million dollars worth of Treasury securities. The seller gets that paid into their account. The Fed credits their bank with the same amount in reserves. But under a fractional reserve system, the bank may only need to keep reserves equal to 10 percent of the new deposits. Since the new deposit equals a hundred million dollars, the bank is only required to hold ten million in reserves to cover it. So the bank now has 90 million dollars in excess reserves, which it can loan. This drives down the federal funds rate.
If the Fed instead buys Treasury securities, it debits the bank's reserves. Now the bank has lost reseves and total systemwide reserves have decreased by a hundred million dollars. That makes reserves more scarce and drives up the federal funds rate.
The federal funds rate is the rate at which banks obtain overnight loans from other banks. That affects the banks's bottom line. The bank passes on its costs, plus a profit margin, to individuals and businesses which in turn borrow money from it. Thus, the federal funds rate influences the prime lending rate. The prime rate in turn influences such things as mortgage rates. So the Fed can more or less set the federal funds rate, and from there its influence spreads, but less predictably, since factors other than reserves influence the other lending rates.
It is important to understand that in controlling the federal funds rate, the Fed is controlling excess reserves in the banking system as a whole. It is not controlling the money supply directly. The amount of Fed Open Market operations needed to set the federal funds rate will depend on many factors, including existing reserve levels, inflation, the velocity of the money supply (which is how fast money circulates), foreign trade, and other factors. So the Fed these days doesn't target money supply measures, but merely the federal funds rate.
The Fed can also loan reserves to banks, directly, through its Discount Window. The interest rate of these loans is called the discount rate. This is generally higher than the federal funds rate. The distinction is important: the federal funds rate involves private commercial banks loaning money to each other overnight; the discount rate involves direct loans by the Fed to banks. The Fed can set the discount rate by fiat. It sets the federal funds rate by Open Market operations which determine systemic excess reserves in the private commercial banking system.