It's called security and it is a pain for a reason. yes, you have to give sites permission... without the extensions, sites do whatever whether you want them to or not. Many sites out there are rather malicious, either on purpose or through being hacked. Any protection you can get is that much more between you and it.