Yunalesca, it surprizes me that you can get online with either computer. Only one computer can access the modem's IP at a time, which usually denies both if they attempt access the internet together. The point of a router is it utilizes DHCP to share the connection with several computers. It will function as a gateway and control the internal (LAN) IP addresses. It can and does serve other purposes but for home/small office envioments this is it primary job.
The switch it like a smart hub, and is often used in conjunction with a router. On a closed LAN (no outside access) you can assign each computer an internal IP address and use a hub or switch to link them togther. You could also connect one computer to the internet directly thru the modem and "route" the other computers thru the first. Th first computer would be the "Gateway" computer. with a second NIC in the Gateway computer you would attach the switch or hub. That is what I would recommend if you cannot take back the switch and exchance it for a router.
I would recommend the Linksys Nwtwork Everywhere NR041-mw 4 port Router. I have use it for years and as a basic router it is bullet proof.http://www.walmart.com/catalog/product.do?product_id=3910287