Compare the amount you will lose by drawing that 401K to the amount you will save on interest by paying the mortgage off earlier and go from there. If you're losing more by drawing out the 401K, then you are better off making the payment.
Additionally,
if you decide that depleting the retirement savings to pay it off is beneficial, make sure you are reinvesting that "mortgage payment" every month back into a retirement plan, don't just consider it a $1500/month (or however much/month it is) "raise". Theoretically you shouldn't be banking any more money every month than you currently are - it should come out as a wash. The only difference is you're paying into a retirement account instead of paying the bank for a mortgage.
With that said, your interest rate is relatively low so I'm guessing most investment accounts are giving you a higher return than that...I think the average long-term 401k return is roughly 6-8% so in theory, your money makes more sitting there than it loses in mortgage interest
