MPC is embedded in your codec package, that is why you do not see it in Add\Remove. You can uninstall it by uninstalling the codec package. A better way to solve your issue is to just configure MPC to not be the default player for files you prefer to playback in something else.
Open MPC, click View>Options>Formats to do so.
Afterwards, double check the default playback options in plain old WMP as well.
You don't want both handling the same type of media files.