"... yer not ter use magic in the Muggle world except in very special circumstances..."
-- Harry and Hagrid in Diagon Alley (PS5)
“We are here to examine Harry Potter’s offenses under the Decree for the Reasonable Restriction of Underage Sorcery!” -- Cornelius Fudge (OP8)
The Decree for the Reasonable Restriction of Underage Sorcery was written in 1875 by the Ministry of Magic (CS2). This law forbids the use of underage magic outside of school, and is enforced by the Improper Use of Magic Office. Warnings are given for violation under paragraph C knowingly, deliberately, in full awareness of illegality, to do magic in a Muggle-inhabited area and in the presence of a Muggle. More than one offense may cause a student to be expelled from Hogwarts and their wand broken (OP8). Clause 7 magic may be used before Muggles in exceptional circumstances, which include situations that include the life of the wizard or witch himself, or witches, wizards, or Muggles present (OP8).
Hagrid told Harry the first time they went to Diagon Alley that he was not allowed to do magic outside of Hogwarts "except in very special circumstances." Therefore, cursing Dudley on a regular basis would not be a good idea (PS5).
Young wizards are also exempt from this rule because they have little control over what magic comes out of them, and they do not own a wand. Their magic is rarely dangerous to others, and usually accidental, such as when Harry flew onto the school roof while being chased by Dudley's gang, or made his hair grow after Petunia cut it. Even when Harry made the glass in the Reptile House disappear and released the boa constrictor, it was not counted against him (PS2).
Aunt Petunia said that Lily brought home frog spawn from Hogwarts and turned "tea cups into rats" (PS4), but the few warnings Lily had about underage magic were "nothing too serious" (JKR).
Harry was first accused of breaking the Restriction of Underage Magic just before his second year, when Dobby the House Elf made Petunia's violet pudding hover near the kitchen ceiling on Privet Drive before falling on top of Harry (CS2).
He received an almost instant Owl from the Ministry, signed by Mafalda Hopkirk, saying:
We have received intelligence that a Hover Charm was used at your place of residence this evening at twelve minutes past nine.
As you know, underage wizards are not permitted to perform spells outside school, and further spellwork on your part may lead to expulsion from said school (Decree for the Reasonable Restriction of Underage Sorcery, 1875, Paragraph C) (CS2).
Harry's second offense that same year was when he and Ron flew the blue Ford Anglia to Hogwarts, after Dobby made them miss the Hogwarts Express, and crashed it into the Whomping Willow. Snape and McGonagall gave them both a stern lecture about the seriousness of being "seen" by Muggles, and Molly Weasley sent Ron a Howler saying his father was being investigated at work because of the episode (CS5, CS6). Neither of them were expelled, however, which was fair since Fred and George had driven the car from the Burrow to Surrey and back without a warning.
The following year, Harry "blew up" his Aunt Marge and watched her float away, causing the Ministry to have to rescue her and modify her memory. That time, Harry knew he would be in trouble and ran away from home (PA2, PA3). However, Cornelius Fudge ignored the infraction because Harry was being protected from escaped convice (and Harry's godfather) Sirius Black (PA4).
Harry's worst problem came the summer before his fifth year at Hogwarts, when Dementors came to Privet Drive, attacking Harry and his cousin Dudley (OP1). Harry used his Patronus Charm to stop them, once again breaking the Decree for the Reasonable Restriction of Underage Sorcery. This time the warning was much more severe, with the Ministry threatening to expell him from Hogwarts and break his wand:
The severity of this breach of the Decree for the Reasonable Restriction of Underage Sorcery has resulted in your expulsion
from Hogwarts School of Witchcraft and Wizardry. Ministry representatives will be calling at your place of residence shortly to destroy your wand (OP2).
But immediately Dumbledore and other members of the Order of the Phoenix interceded on Harry's behalf (OP2). Arthur Weasley wrote an owl telling him not to surrender his wand. The Ministry sent another Owl saying he could keep his wand until his hearing, at which time they would also decide if he would be expelled (OP2). At the Hearing, Harry explained the danger he felt from the Dementors especially since Dudley was a Muggle, and Mrs. Figg, his Squib babysitter backed him up with testimony. Madam Bones and several others were impressed with Harry's ability to create a corporeal Patronus. In spite of an argument against Harry by Dolores Umbridge, the Ministry voted to clear Harry of the charges (OP8).
Apparition is another type of underage magic, however it seems to be less regulated under the magical radar of the Ministry. Dumbledore used legal side-along Apparition with Harry to go to the Sea Cave, but Harry had to do the spell on the way back because Dumbledore was weak from the Cave Potion (HBP26). According to Lupin, it is impossible to track anyone who Apparates, unless that person grabs hold as they disappear (DH11). This is exactly what happened to Hermione when Yaxley grabbed her and side-alonged to the entrace of Grimmauld Place, breaking the Fidelius Charm (DH14).
Harry was able to Apparate and Disapparate many times even though he never took the Apparition Test after he turned 17. Ron took it once and failed (HBP22). Another student who was able to Apparate without a license was Montague, who escaped from the Vanishing Cabinet where Fred and George had trapped him even though he never took the test. Unfortunately he Apparated into a toilet pipe (OP28, HBP27)
There are many small examples of wizards using magic underage. Hermione said she had tried some spells before coming to Hogwarts although she was a Muggleborn (PS6). Harry used the Disarming Charm in the Muggle graveyard against Voldemort when he was 14, causing the Reverse Spell Effect (Priori Incantatem) (GF34). During the Battle of the Seven Potters, Harry used magic to fight the Death Eaters and Lord Voldemort in the sky (DH4). Underage wizards are rarely caught in instances like this one because said wizards are surrounded by many overage wizards.
Harry also unintentionally called for the Knight Bus while still in Little Whinging (PA3). The Ministry was apparently aware of this because Fudge met him at the Leaky Cauldron, and perhaps there are exceptions for travel since the Knight Bus can't be seen by Muggles anyway.