I am guessing here as to what your power thingy thing is.
Is it a power strip with multiple power sockets with one main switch that powers off all of the plugs in the strip at the same time?
Now from the sounds of things, it seems like what you are describing is a case of when you leave the switch turned on the power strip at all times and shut the computer down normally and start it normally, you have NO problems.
However, when you also power off the power strip through the power strip switch, your computer starts up, but acts funny and briefly shuts down, only to start up shortly later, normally.
Does this sound pretty close to what you are experiencing?
For the sake of my being correct with my general analysis above, I want to briefly explain to you that power supplies have what is called electrolytic capacitors built into the circuitry.
Those are like little batteries that can store energy, they discharge occasionally to inject this energy into the circuit to remove what is called an AC sine-wave ripple, changing the phase into pure DC current.
An AC sine-wave is similar to the letter S laying on its side, with a line running through it.
like this image below.
That image above is a PURE AC power sine wave, its down swing is negative phase and its up swing is positive phase.
DC power (such as batteries) does not have a sine wave, the wave is filtered out by the use of capacitors, which store either positive or negative pulses and injects those into the circuit when needed, this causes the flow above the line to be all positive and the flow below the line to be all negative in one straight line across, the S effect in the image above is removed.
In order to do this, capacitors are designed to store a current, even long after the computer is turned off for many days or sometimes years, just like how a rechargeable battery stores power for quite some time.
When power is cut off to the power supply via the power strip, those capacitors are drained down to zero by the continued demand of the computers circuitry, so they are no longer storing any current, such as a dead battery.
This is why you are having issues with the initial start up, however, once the power has been restored to the power supply via the power strip being turned on, those capacitors are once again recharged and the computer functions normally again.
What this tells me is that your power supply has a faulty circuit, which one exactly, I can not tell you, but there is something in there that fails to function correctly, if those capacitors are allowed to discharge fully.
Servicing a power supply is NOT user friendly and is very dangerous to say the least!!!
I recommend your doing one of two things:
The cheaper one is purchasing another power supply.
The more expensive one is having this one serviced at a repair shop, labor costs will bring up the cost of repair to the price of a brand new power supply, so in my opinion, it is better to just replace the power supply.
Replacing it will not subject you to any dangers like I listed above, because the power outside of the power supply is SAFE DC current, the power inside of a power supply is very DANGEROUS AC current that is anywhere above 120--1000 Volts! 1000 Volts can hurt you very badly and power supplies are known to have voltage that high even years after the thing has been unplugged.
So if you wish, it is safe if you purchase another power supply and install it yourself.
Hope this helps and sorry about the science lesson I published above, I just like stating facts and non-fiction science along with my explanations.