use an ethernet cable (looks like a phone cord, but wider) and connect the new computer to an available port on the router. if there is nowhere to plug one of those in, purchase a PCI ehternet adapter and install. turn the computer on.
purchase and install an wireless adapter(external USB is easiest), and viola. no muss, no fuss with cables. many modern laptops have wireless capability built in.
it should be about that simple. all your computers need to do is be connected to the router(the wireless dingus) in some way, either with a wire or wirelessly, and that little box will do the rest. it's called peer-to-peer networking and was designed for just this purpose.
is there anything we should know about this new computer? is it 40 years old, or a leftover Sparc tower or an Ibook or something? running Windows 3.1 or BeOS or some craziness?