The consequences of sugar addiction don't manifest themselves in an acute form immediately. They usually happen later in life, in the form of diabetes or some other inflammatory disease. Being harmful doesn't necessarily have to be a property of addiction. It simply means when you have a piece of chocolate, you want another piece of chocolate, then another, etc.
>Addiction is the continued repetition of a behavior despite adverse consequences,[1] or a neurological impairment leading to such behaviors.[2] - Wikipedia
Adverse consequences are necessary for it to be called addiction. If the sugar isn't negatively affecting your life, it doesn't count as addiction.