If you log into your router from your computer you will be able to see what devices are connected. You do this by typing the router's IP address into the address bar on your browser. Very common IP addresses are 192.168.0.1 and 192.168.1.1 , but there are others. You may well find the IP address on a label on the router.
If you are concerned about unauthorised users eating up your bandwidth allowance (1) Change the router password - you do this from the router's control panel which you access as above; (2) Apply security on the router, preferably WPA2-PSK. This requires you to generate a pass-code which you then enter onto each device authorised on your system. Once you have done this, only devices with this pass-code can access your internet connection.
Just as a general tip, once you have generated this code, make a copy and keep it in a safe place. It would be both embarassing and inconvenient if you were to say buy a new computer and discover you couldn't connect to your own network !