You have a bug that is altering your destination height unexpectedly prior to drawing to the screen.
If you look at your own picture you will see that the destination Height is changing as the square changes.
It's changing from 118 to 122 then back to 118.
Every time the displayed destination rectangle Height value changes... the black square moves.
You can confirm that by just temporarily setting the destination height in the spriteBatch draw to some hard coded constant value.
var testDest = destination;
testDest.Height = 122;
spriteBatch.Draw(texture ,testDest, source, color);