You should be on Internet Explorer 10 by now. IE 8 and 9 have more than a few security issues, which might explain how your stealth popup got there.
Start - Control Panel - Windows Update
Click Change Settings
It should be set to "Install Updates Automatically (recommended)"
It's not a bad idea to check "Give me recommended updates the same way I receive important updates" either.
If you want, check "Give me updates for Microsoft products and check for new optional Microsoft software when I update Windows" as well. That will update things like Office, or Streets and trips, etc if you have them.
Click OK
Click "Check for Updates"
Let it search
Then click on "Install Updates"
Let it do its thing and it may ask for a reboot at some point.
Note: Truth or Myth: I don't have to update IE anymore because I have Firefox?
Myth. It's still installed, and you can't get rid of it. Additionally, some software will use it regardless of your preferred settings. IE still needs to be updated when there's an update to keep you safe and to let certain software continue to work.