First, let's be clear about something. All currents partially take all paths relative to the inverse of the path impedances. It's not an all or nothing function. This "Takes the path of least resistance" is often heard, but really not the best mental picture.
For example, if a voltage is applied to a 100 Ω and 50 Ω resistor in parallel, ⅔ of the current will flow thru the 50 Ω resistor and ⅓ thru the 100 Ω resistor, even though the 50 Ω resistor is the "path of least resistance".
Second, there is nothing special about high-frequency currents behaving this way. All currents take paths inversely proportional to the path impedances.
Now, to your actual question. Your misconception comes from not seeing the chip being bypassed as the source of the high frequency current. The power supply fixes the voltage, then the loads connected to it draw whatever currents they want (within limits of the power supply current capability).
Let's say you have a digital chip that draws 100 mA at 3.3 V. But, that 100 mA is only an average. Let's say for sake of example that the current goes from 80 to 120 mA in a sine at 10 MHz. Mathematically, the current is
I = 100 + 20⋅sin(10M⋅2πt)
where I is the current in mA, and T the time in seconds.
Here is a schematic of the chip and the power supply:
Let's assume everything is ideal for now. SRC3 is putting out a constant 3.3 V. That's all it gets to control. The load, which is the chip in the box at left, is stuck with the voltage, but gets to chose the current. I have shown this current broken into its DC and AC components. SRC1 is the steady 100 mA. SRC2 is the high frequency current on top of that DC average.
The point to note is that while the power supply (SRC3 in this example) provides the voltage that causes the current, SRC1 and SRC2 are what actually decide the current.
So why does this splitting of hairs matter? Because the world isn't ideal like this picture. The leads to the power supply and the power supply itself will have some impedance. This can be modeled like so:
The longer the wires to the power supply, the larger both L1 and R1 get. Now consider that the current thru this L1+R1 impedance causes a voltage drop. That's just Ohm's law.
For example, ignoring L1 for now, let's say R1 is 1 Ω. The average voltage drop across R1 is (100 mA)(1 Ω) = 100 mV. So what started out as 3.3 V at the supply is only 3.2 V by the time the chip sees it. Considering the high frequency current, the drop across R1 actually goes up and down between 80 and 120 mV at 10 MHz. Most chips intended to run at 3.3 V would probably be OK with that.
The real problem comes from L1. It causes the impedance to the supply to go up with frequency. To get 1 Ω impedance magnitude at 10 MHz requires only 16 nH. At 100 MHz, that same 16 nH is now 10x the impedance of R1 and dominates the total impedance. That also means what was 40 mV peak to peak ripple becomes a 400 mVpp ripple. Now the chip sees its power supply voltage fluctuating from 2.8 to 3.6 V. That alone would be a problem for many chips, but the 100 MHz voltage signal capacitively coupling to various points inside the chip will likely make a mess.
The solution is to add a bypass capacitor:
One way to look at this is that C1 shunts the SRC2 current immediately around the chip. This current therefore never passed thru L1 and R1, and therefore doesn't cause an offset voltage across them.
If C1 is 1 µF, for example, then it has a impedance of 16 mΩ at 10 MHz. The 40 mApp flowing thru it cause less than 1 mVpp voltage ripple on the chip's power input.
So again, the point you are missing is the high frequency current is actually produced by the chip, not the power supply. By providing a low frequency shunt for that current physically close to the chip, that current never gets to the power supply.
This also explains why it's so important that bypass caps be close to their chips, using the minimum possible lead lengths. Any inductance in the loop between the chip and the cap adds impedance to the path, particularly at high frequencies. You need the total path impedance to be small, else the SRC2 current won't be harmlessly shunted by C1, and a voltage drop will appear someplace, causing voltage ripple on the supply as seen by the chip.