I think this is just the UI not being intuitive. From what I've played of Bebob, that number on the UI isn't the stacks, but the total bonus granted from the stacks. Since each stack grants a 3% bonus (as of this posting), 2 stacks lost would mean you lose 6% from your listed bonus, which is...