Most likely, yes. Your internet connection (be it DSL or otherwise) isn't like your phone line that allows you to split it with a doubler, tripler, etc.
The very purpose of a router is to perform that function, taking that single connection to the outside world and "juggling the plates" such that multiple devices can use it without problem.
A router is always hooked to the modem, when a single modem is the current setup, and the wired computers are connected to the ethernet port(s) on the router. Wireless is its own side issue that the router handles along with the wired ports.
Brian AKA Bri the Tech Guy (website in my user profile) - Windows 10 Home, 64-Bit, Version 1803, Build 17134
. . . the presumption of innocence, while essential in the legal realm, does not mean the elimination of common sense outside it. The willing suspension of disbelief has its limits, or should.
~ Ruth Marcus, November 10, 2017, in Washington Post article, Bannon is right: It’s no coincidence The Post broke the Moore story