If I understand it all correctly, your router is/contains a NAT (Network Address Translation) firewall, which essentially accepts only packets in response to something you've requested. So, yes, it's a firewall. In addition, I believe your particular router has a built-in software firewall which can be configured by going to the modem's address (check your documentation - something like Linksys' 192.168.1.1, though yours will probably be different)
Some say that using the software firewall within the router will mess up the functionality - resetting the router or some such. It's not a bad idea to have a software firewall on your computer, such as Zone Alarm, which won't mess with your router, because the NAT firewall will protect from port searches but won't protect from junk in your computer trying to communicate with the net. The NAT firewall sees that as normal traffic. Zone Alarm protects from outside hits as well as inside hits.
Edited to add: sorry, while I was looking for info, I forgot you mentioning the Norton firewall. Just ignore the Zone Alarm part - others may benefit from it.
Edited by Herk, 11 March 2005 - 11:55 PM.