Because people are not very computer literate and get intimidated by any special actions they are required to do that they don't understand. They want the page to load and have everything right there for them with no extra effort or understanding required.
Out of experience; I do not accept or install plugins other than Flash, Java, Quicktime et cetera; i. e. well known safe technologies - as apposed to one-off unknown products.
Too often, "plugin" since Internet Explorer 6; means something that tried to add it's own searchbar, change the homepage, set itself to run on startup, and generally do all manner of things I'd rather that it didn't. "Unknown-third-party plugin X required" reads as "Quick! Go somewhere else or risk sitting here all night restoring sanity by tentativly ripping plugin X back out of my system". Be that lack of understanding if it be.
You can do all manner of 3D in Java, and even Flash if you have the right tools... Most people have those plugins because so many sites use them. Even then, I wouldn't let an unsigned Java applet/application have unlimited permissions (which seems be neccessary since Java 1.5 for 3D) unless I knew the developer(s) personally.
Generally, for me, it's more a case of effort and principles(!) I find the push to make websites more like multimedia experiences (or desktop applications for that matter) is at best counterproductive (loading/glitching/memory hogging/etc), and at worst downright annoying. (going to site, going to get plugin, maybe restarting browser, going back to site, waiting for site to load up the plugin, etc).
I think the key is moderation and independance. Don't make every aspect of your site dependant on ANY third party plugin, and make sure someone can use your site to a certain level without the plugin. You can't force people to make a configuration decision; so the best you can do is accomodate many potential configurations.
And not only that, any site trying to sell me something had better not require me to do anything special like installing plugins or open the block I placed on Flash or I'm gone before ever seeing what they have to sell.
You want to make using the site as easy as possible for users, and requiring them to install plugins and press buttons in dialogs telling them about potential security risks from the site is NOT going to do that.