Well, it seems like this is something you need to take up with your ISP, perhaps.
It is out of my area of expertise. If anyone else has ideas, feel free to chime in.
It sounds like you're having to dial the broadband connection for each computer to use at a time. Broadband should not work like this, according to my knowledge. The router/modem should dial the connection by itself and keep it until you turn it off or unplug it. Then, any and all computers you want to connect to the network should be able to, whether by ethernet or Wi-Fi.
But it sounds like your ISP has screwed your service up somehow and the "dial PPPOE" is just a workaround. I would call to complain, if I were you.
Sorry I couldn't be of better help.