Do not hit your 401K to pay off a mortgage in the 3's. You're making more than 3.6% on your investments so you'd be crazy to pull it out of there just to go in hole/losing money, to save even less on interest. In addition, you'll have to pay an early withdrawl penalty of 10% plus the federal and state taxes on any money you pull out, just for the privilege of tapping the 401(K). You can take your cash and put it in a FDIC insured money market making more than the 3.6% you're paying in interest, and still have complete access to it, anytime you need it. Once you put that money in your house, it stays there until it is sold or you put a new mortgage on it, which will then be in the 6.5% range. 3% mortgages are long gone and are not coming back. If you're worried about the stock market, reposition into safety, within your 401(K).