A bag charm should look like it arrived on the bag by accident, not like it was bought at the till as an afterthought, which is exactly what most of them were. The good ones read as something you've had for years and clipped on without thinking twice. The bad ones look like a sticker, something bolted on to make a plain bag seem more interesting than it is, and they usually succeed only in making the bag look cheaper than it was.
What it fights is clutter. One charm on a clean leather tote looks considered. Three charms, a tassel, a scarf tied to the handle and a logo keyring all going at once just looks like you emptied a drawer onto the strap, and it draws the eye away from the bag itself, which is usually the better looking object in that scenario. Pick one, maybe two if they're small and sit close together, and let the bag still be the thing people notice. The difference between a decent charm and a disappointing one is mostly in the hardware you can't see until it's failed you. A proper clasp, the kind with a spring mechanism you can actually feel click, will survive being opened and closed daily for a long time. The flimsy lobster clasps on the cheaper ones bend out of shape within weeks and then the charm either falls off on a train somewhere or you spend ten minutes trying to wrestle it back on with cold hands. Leather tassels are the same story twice over: a well finished one has edges that have been sealed or folded, not left raw to fray within a month, and the leather itself has a bit of weight to it rather than feeling like stiffened cardboard. Metal charms show their quality in whether they tarnish. A good plating holds its colour for years with only the gentlest polish now and then, while a poor one goes a dull brassy colour within a season, especially if it touches perfume or hand cream, which it inevitably will.
Where charms are the wrong choice is on anything with a lot of hardware of its own, chunky buckles, branded clasps, obvious stitching details. Adding a charm there just creates visual noise competing with noise. They're also not for bags you sling about carelessly, gym bags, school runs, anything that gets dropped on wet pavements, because charms catch on things and the clasps take the strain first. Treat a good one gently, keep it away from water and perfume, and it will age the way good costume jewellery does, with a soft, worn in look rather than a tarnished one, long after the cheap version has already gone in the bin.