The way that the secondary mortgage market works, through companies like Fannie Mae and Freddie Mac, is when government founded agencies buy loans from lenders. They will either hold these loans in portfolios or put them with other loans into mortgage backed securities. The securities are sold to Wall Street firms, mutual funds, or other types of financial investors.
Although you may think bankers and mortgage brokers are in control of the rates that you will pay, this is really not the case. In fact, investors are the ones who are in control. When it is predicted that the economy is doing well, investors will demand higher yields from lenders. The reasoning behind this is due to the fact that they do not want to buy low yield bonds, in case the federal government raises rates in order to calm the economy. This will mean that they will make higher yield bonds later.
In this situation, the only way that it will be possible for a lender to get their sold is to raise the yields which are offered to investors. For consumers, the rates will go up.
When the economy starts slowing down, investors will start investing in bonds because they assume that the Federal government will have to cut interest rates to get the economy back and running. If they wait too long, they will end up with lower yield bonds. Since investments are in such high demand from investors, lenders will offer lower shields which means that there will be lower rates for the consumer