Bring sugar and 1/2 cup water to a boil in a saucepan over medium heat, and cook, stirring constantly, 5 minutes or until syrup consistency. Let cool.
Stir together cooled syrup, half-and-half, brandy, and vanilla. Serve over ice. Sprinkle with nutmeg.