To prepare the cranberry sauce, combine the first 5 ingredients in a medium saucepan, and bring to a boil. Reduce heat, and simmer, uncovered, 30 minutes, stirring occasionally. Press the cranberry mixture through a sieve over a bowl, reserving 1 2/3 cups cranberry purée, and discard solids. Cover sauce and chill.
To prepare the custard, combine 1/4 cup milk and egg in a small bowl, and stir well with a whisk. Place 1/3 cup sugar, cornstarch, and salt in a medium saucepan, and gradually add 1 1/2 cups milk, stirring with a whisk until well-blended. Bring to a boil over medium heat, and cook 1 minute, stirring mixture constantly. Gradually stir about one-fourth of hot milk mixture into egg mixture; add to remaining milk mixture, stirring constantly. Cook over medium heat 3 minutes or until thick, stirring constantly. Remove from heat, and stir in vanilla. Pour into a bowl, and cover surface of custard with plastic wrap. Chill.
Peel oranges, removing white pithy part. Cut each orange in half lengthwise. Cut each half crosswise into thin slices; set aside.
Split ladyfingers; tear each half into thirds. Line the bottom of a 3-quart straight-sided glass bowl or trifle bowl with half of ladyfingers. Spoon 1 cup custard over ladyfingers. Spoon half of the cranberry sauce over the custard. Arrange half of orange slices over the cranberry sauce and around the inside edge of bowl. Repeat the procedure, ending with orange slices. Cover and chill 8 hours. Top each serving with 1 tablespoon whipped topping and 1 teaspoon almonds, if desired.