Here's the deal:  if you have an owner occupied loan and 3 investor loans you cannot buy any more properties and get Fannie Mae / Freddie Mac financing, meaning you can't get a conventional 30-year fixed loan. Now, my hope is that someone reads this and tells me I'm wrong. That would be great!  But as far as I know that is the case.  

 Where does this leave you?  You can pursue loans that are warehoused by lenders, meaning they are not sold on the backend to Fannie or Freddie. You are probably looking at a minimum of 20% down but more importantly it will be almost impossible to get a 30-year loan.  But a 5/1 ARM is not out of the question. (Lenders, please start a dialogue here and let folks know who has what products available.)  There is also Hard Money available.  I met with a group of high-end Hard Money lenders today to discuss options and the consensus is that they are proceeding...but with extreme caution.