Combine pudding mix and skim milk, stirring until smooth. Stir in sour cream; set aside. Combine marmalade and sherry, stirring with a wire whisk until blended.
Arrange half of cake cubes in a 1 1/2-quart trifle bowl or straight-sided glass bowl. Spoon half of pudding mixture over cake. Drizzle marmalade mixture over pudding mixture. Arrange half of oranges over marmalade mixture. Repeat layers with remaining cake, pudding mixture, and oranges. Cover and chill at least 3 hours.