Absolutely, it goes to the button. After all, the button is the last player holding live cards. That's the key. He's got live cards. The big blind's hand was mucked, so he doesn't have a live hand.
The button folding out of turn is a consideration, and in that event, it goes to the cut off.
For what it's worth, this happened to me, back in the day before I started to learn about the rules. About 10 years ago, I was at the Showboat, and after break, I was the button and both blinds were missing and everyone folded to me. As I did a quick look to see if anyone called, so I can jokingly announce all in, the dealer beat me to it, pushing the pot to me.