You can spot a good neutral summer outfit from across a room by how it moves, not by its colour. A poor one just sits there flat, the stone or oatmeal looking a bit grey under shop lighting, the fabric gripping in the wrong places. A good one has a bit of life in the weave, catches the light when you walk, and looks more expensive than it was, simply because someone thought about drape before they thought about shade.
Neutral pieces only really work once you have a few to play off each other, so think about what else is already hanging up before buying more. A linen shirt wants a plain cotton vest underneath it, not another pattern, and a pair of wide leg trousers in oatmeal needs a top with some structure or the whole thing reads shapeless. Cream, sand, stone and soft white all sit together fine even though they're not identical, so don't chase an exact match, chase fabrics that feel similar in weight. In the shop, touch before you look. Run the fabric between your fingers and hold it up to the window if you can, because pale linen and viscose can go see through fast, and you want to know that before you've left the house in it, not after. Check how a skirt or dress falls when you move, not just when you're standing still, since clingy fabric in a pale colour shows every seam of what's underneath. Look at the stitching on hems and the way buttons sit, cheap neutral pieces often cut corners there because the fabric itself is already doing a lot of the work of looking nice. These outfits earn their keep from late spring through to early autumn, for anything outdoors in the heat, a long lunch, a wedding where you don't want to compete with the venue, or just a normal hot week where you want to feel cool without looking like you've given up. They photograph well in daylight, which matters more in summer when you're outside more often, but that's a side effect, not the point.
Fit is where neutrals either work or don't, because there's no pattern to distract from a bad line. Go up a size in linen if you're between sizes, since it shrinks slightly in the wash and softens with wear rather than stretching out. Watch the waistband on trousers and skirts, pale fabric pulls tight and creases exactly where it's too small, and that shows far more than it would in navy. Try sitting down in anything fitted before you commit, and check sleeves don't ride up when your arms are raised, since pale fabric that gapes at the underarm is obvious the moment you reach for something on a shelf.