There's a really interesting podcast from Reply All about how Facebook "listens" to you without your consent. But don't get your pitchforks out yet. You actually have, in a weird way, given consent. But how Facebook "listens" to you isn't really through your cellphone's microphone (as it has been widely reported -- and debunked), but through your location history, your search preferences, and something called the Facebook Pixel.
Reply All's parent company Gimlet Media has a page set up with instructions on how not to be tracked by Facebook.