How to Make It
Preheat oven to 350°F. Line a 9-inch square baking pan with parchment paper, leaving a 2-inch overhang to use as “handles” later. Lightly coat parchment with baking spray.
Place heavy cream and duck fat in a medium saucepan over medium. Bring just to below a boil, stirring occasionally, about 6 minutes. Whisk together egg yolks in a medium bowl. Slowly pour 1 cup of the hot cream mixture into egg yolks, whisking constantly, until blended. Slowly whisk egg yolk mixture back into saucepan over medium. Cook, stirring often, until mixture is silky and coats the back of a spoon, about 4 minutes. Strain through a fine mesh strainer into a large glass measuring cup. Whisk in salt and pepper.
Spread 1/2 cup of the cream mixture in an even layer in prepared pan; top with 3 slices of the brioche. Slowly pour 1 cup of the cream mixture evenly over brioche. Repeat process with remaining 3 slices brioche and 1 cup cream mixture. Place any remaining bread on top to absorb excess moisture. Cover tightly with aluminum foil, and bake in preheated oven until just beginning to set, about 20 minutes. Uncover; bake until most of the liquid is absorbed, about 5 minutes.
Place an 8-inch square piece of parchment paper on top of bread pudding. Place a smaller baking pan or plate on top of parchment, and place a weight on top (such as a 28-oz. can). Refrigerate until completely chilled, about 2 hours.
Preheat oven to 350°F. Use parchment “handles” to remove bread pudding from pan. Cut into 4 squares. Melt 1 tablespoon of the butter in a large skillet over medium until sizzling. Add 2 slices of the bread pudding to skillet, and cook, until golden brown, about 3 minutes per side. Transfer to a baking sheet. Wipe skillet clean, and repeat process with remaining 1 tablespoon butter and 2 slices bread pudding. Bake in preheated oven until completely warmed through, about 5 minutes.
While bread pudding bakes, heat apple butter and caramel topping in a small saucepan over medium, stirring often, until warmed through, about 3 minutes. Top each serving of bread pudding with ice cream and apple butter mixture. Serve immediately.