Preheat oven to 350°F. Lightly grease a 13-by-9-inch cake pan; line with parchment paper. Whisk flour, ginger, baking soda and salt in a bowl; set aside.
Using an electric mixer, beat eggs and sugar at medium speed until pale and thick, about 5 minutes.
Heat milk in microwave until hot. Stir in butter. While mixing at low speed, drizzle hot milk into egg mixture; mix until smooth. Stir in vanilla. Sift a third of flour mixture into egg mixture; stir to blend. Repeat twice with remaining flour.
Scrape batter into pan; bake until a cake tester comes out clean, 25 to 30 minutes. Let cool in pan for 10 minutes, then turn out onto a rack; let cool completely. Remove parchment paper.
Whip heavy cream and confectioners' sugar. Cut cake into 12 portions. Spread jam and then whipped cream on each piece. Sprinkle with candied ginger, if desired