Whisk together milk and vanilla pudding mix in a bowl until pudding mix dissolves (about 2 minutes).
Cover and chill mixture 15 minutes or until thickened.
Add sugar, vanilla, and rum extract, stirring until sugar dissolves.
Cover and chill at least 1 hour. Pour mixture into cups, and top each serving with a scoop of ice cream and a dollop of whipped cream; sprinkle with freshly grated nutmeg.