"Hagrid led them through the bar and out into a small, walled courtyard, where there was nothing but a dustbin and a few weeds.... He tapped the wall three times with the point of his umbrella. The brick he had touched quivered - it wriggled - in the middle, a small hole appeared - it grew wider and wider - a second later they were facing an archway large enough even for Hagrid, an archway onto a cobbled street that twisted and turned out of sight. 'Welcome,' said Hagrid, 'to Diagon Alley.'" (PS5)
Though nondescript, this alley is one of the primary gateways in Britain from the Muggle world into the wizarding world. It has rarely been mentioned since Harry’s first time through, but presumably wizards need to pass through it any time they travel between the Leaky Cauldron and the rest of Diagon Alley. It’s interesting that magic is required to pass through it when it seems Muggles can’t see the Leaky Cauldron to begin with (PS5).