I believe that Firefox doesn't show the Alt= attribute in a popup on purpose. You can get the same effect, though, by using the Title= attribute:
<img src="foo.jpg" Title="A Picture of Foo">
The reason why Firefox, Chrome, and other browsers than IE don't display a popup when the ALT tag is used is because they're not supposed to. This behavior is an old, old bug in IE. According to the rules of HTML, the alt tag is designed for when an image is not displayed to the user (either because the image is missing or because for whatever reason the user's browser doesn't display images (this is a setting in most browsers and is the only setting in text-based browsers, not to mention text-to-speech browsers for the blind.) You should use the Alt tag to specify placeholder text for the image (e.g. alt="Picture of my cat, Mr. Whiskers.") The Title tag is used to specify any additional information you want to tack onto the image, and almost all browsers will show it in a popup tooltip.
A popular example of using Title= for additional information is the XKCD webcomic