There actually seem to be two different questions here.
- Can a primary (non-rechargable) battery be damaged by forcing reverse current thru it?
Yes. Primary batteries are only intended to source power, not sink it. Forcing reverse current thru them, especially for significant lengths of time, can cause damage. That can vary from slowly reduced capacity, to leaking or even exploding.
- Can an oscillator cause reverse current thru its power supply?
Perhaps, for small portions of the oscillation cycle if you hold your head just right and squint. The oscillator will consume power on average. At most there may be a little power going back into the power supply for a short time each oscillator cycle, then more power taken from the supply at other times of the cycle. This might actually be tricky to arrange deliberately.
Just because an oscillator exhibits negative resistance on some internal node, does not mean the overall circuit exhibits negative resistance to the power supply. In fact, on average it can't since it consumes power. Any power dumped back onto the supply would have to be previously stored, then released. The total energy of each release is limited to the energy the circuit can store.
It would be possible, in theory, to create a "battery-buster" circuit. It would drain power from a battery normally to store a lot of energy locally, like charge up a large cap. That energy can then be released back onto the battery, which could damage primary batteries.
The chances of a normal oscillator doing this in a meaningful way without being designed for the purpose are quite slim.
In practice, if a circuit really varied its power supply current enough to occasionally go negative, you'd put a large capacitor on the supply. With a large enough capacitor across a battery, occasional small reverse currents of a load wouldn't cause reverse currents into the battery. The cap would keep the voltage more constant. Together with the impedance of the battery, that causes the battery current to be close to the average, which must always be positive for any real circuit that doesn't have a separate source of power.
The output of a oscillator is AC by definition so I dont think a capacitor will help
The output of an oscillator is AC, but I'm talking about a capacitor across its power supply.
So, as I understand now: Oscillator will try to force supply lines to a different value periodically (even if for the small period of time). If we put a capacitor between +and -, since it is a low impedance at the oscillator frequencies, than excess current will go through it, "bypassing" the battery.
Yes, that's the frequency-space view. The capacitor provides a shunt for the AC current created by the oscillator. By shunting the AC current thru the capacitor, it doesn't cause voltage variations against the power supplies non-zero impedance.
whether this capacitor will load/change frequency of the oscillator? And is it what is typically done in practice or not?
A capacitor across the power supply should not in theory change the oscillator operation. That's because in theory the power supply has 0 impedance, so adding a capacitor across it doesn't actually change anything.
In practice, power supplies don't have 0 impedance, and we add capacitance to lower that impedance, particularly at high frequencies. Whether that changes the oscillator operation depends on how tolerant the oscillator is of the power supply not being 0 impedance. Well designed circuits operate correctly with some finite power supply impedance.