Combine first 3 ingredients and 1/2 cup brandy in a medium glass bowl. Microwave at HIGH 2 minutes, stirring after 1 minute. Set aside.
Weigh or lightly spoon flours into dry measuring cups; level with a knife. Combine flours and next 6 ingredients (through allspice) in a bowl; stir well.
Place 4 tablespoons butter, oil, granulated sugar, and 1/2 cup brown sugar in a bowl; beat with a mixer at medium speed until blended. Add eggs and egg white, one at a time, beating well after each addition. Add ginger and orange rind; beat 30 seconds. Add flour mixture and buttermilk alternately to sugar mixture, beginning and ending with flour mixture. Fold in cherry mixture, apple, pear, and pecans.
Spoon batter into a 10-inch Bundt pan coated with baking spray; smooth top with a spatula. Bake at 350° for 45 minutes or until a wooden pick inserted in the center comes out clean. Cool in pan for 20 minutes on a wire rack. Remove from pan; cool completely.
Combine remaining 1/4 cup brandy, remaining 1/4 cup brown sugar, and remaining 1 tablespoon butter in a small saucepan over medium heat; bring to a boil. Cook 2 minutes or until reduced to 1/4 cup, stirring well. Drizzle over cake.