Eliminating Certain Foods

I was watching a TV show on ion TV and the fitness expert said that you should allow yourself to eat whatever you want just limited the quantity or else you will feel deprived and are more likely to cheat.  This is not the first time that I have heard or read this statement.
Here are a few reasons why I totally disagree with that statement and why it doesn't work for me:
If you allow yourself to eat anything then you are definitely not eating healthy.
Certain foods have highly addictive qualities such as sugar, alcohol and caffeine.
Certain foods and edible food like substances are proven to cause cancer and other obesity related diseases.
Certain foods have way to many calories (even in small portions) and not enough nutrients.
Foods have the ability to act like a drug, negatively change your mood, make you tired and lethargic.

You seriously need to eliminate unhealthy and food like substances from your diet or you will always be "on a diet" and never be successful.