Most likely, yes. Your internet connection (be it DSL or otherwise) isn't like your phone line that allows you to split it with a doubler, tripler, etc.
The very purpose of a router is to perform that function, taking that single connection to the outside world and "juggling the plates" such that multiple devices can use it without problem.
A router is always hooked to the modem, when a single modem is the current setup, and the wired computers are connected to the ethernet port(s) on the router. Wireless is its own side issue that the router handles along with the wired ports.
Brian AKA Bri the Tech Guy (my website address is in my profile) Windows 10 Home, 64-bit, Version 1709, Build 16299
Here is a test to find out whether your mission in life is complete. If you’re alive, it isn’t.
~ Lauren Bacall