One way to hold this word: something small is hiding inside, and it is making the whole thing behave badly. You cannot see it from outside. So you go in, look carefully, find it, and take it out. English calls that small hidden thing a bug, and de- means take away. Whether the hidden thing is a line of broken code, a microphone taped under a desk, or a real insect in the roses, the job is the same.