In public, the aristocratic women of medieval times were obliged to appear in elegant hair-dos that were usually pulled up. The only time they would let their hair down was when they came home and relaxed.
Hence this idiom means- “To relax or be at ease”
>I can never manage to let my hair down at office events.
>My roommate is such a gossip I find it hard to let my hair down in my own home.