Garments that leave space between the body and the fabric tend to be more comfortable than something tight and clingy for a number of reasons.
Freedom of movement is one from a waistband to a neck or arm or leg hole or overall contact. Head, feet, hands are also affected. The body responds to restriction and to lack of restriction, to the sense of 'breathing with the skin' or feeling 'suffocated'. We tend to breathe differently, move differently, think differently, behave differently in different levels of comfort. Clothing can be a stressor(s). More obvious examples are a too heavy coat that weighs on the shoulders and the neck; too tight shoes;
Our natural heating and cooling systems are compromised by clothing that does not allow the body to regulate its own temperature which it can and will do when given the chance.
Layering is one way of housing the body with clothing. A loose vest can be worn over indoor clothing or over a coat or instead of that coat. Hoods serve all the function of a hat without the grip. Waistbands that tie allow for changes both daily, monthly, yearly.
It's a mindset, really. Not changing an entire wardrobe but increasing awareness.