The arriving first-years travel across the Lake to the castle in a flotilla of small boats, propelled by magic and escorted by Hagrid (PS6).
The first-years find the boats on the shore by the lake. No more than four persons are allowed in each boat. Harry shares his boat with Ron, Hermione and Neville. Hagrid has a boat to himself. The boats sail across the lake towards the cliff on which Hogwarts stands onto an underground harbor.(PS6)
The first-years also take the boat back to the train at the end of the first year (PS17).
In Harry's fourth year, Dennis Creevey actually fell out of his boat. He was pushed back into it, probably by the giant squid according to Colin Creevey. (GF12)
The boats are mentioned by Harry when he says farewell to his son Albus Severus who is boarding the Hogwarts Express. (DH/e)
In her Pottercast interview in 2007, Joanne Rowling was asked what kind of traditions there were for leaving Hogwarts. She answered that leaving by boats would be "poetic and beautiful. It would have been a return to innocence, with an added symbolism of passage over water." (PC/JKR1)
The last part probably refers to the crossing of the river Styx, which -according to Wikipedia- forms the boundary between earth and the underworld. It is also supposed to have magical powers: Achilles was dipped into this water by his mother (who was holding him by his heel).
Crossing the Styx is also linked to death, because of the depiction of the ferryman Charon crossing the river Styx with recently departed souls. However, that was on the river Acheron.
In classical myths, heroes often crossed a body of water at the beginning of an adventure, according to Campbell's "Hero's Journey". They also went on a quest or journey on the water, such as Homer writes about in The Odyssey or the story of Jason and the Argonauts.