1. Preheat oven to 350°.
2. Combine first 4 ingredients in a large bowl; beat with a mixer at medium speed. Add granulated and brown sugars; beat until combined.
3. Weigh or lightly spoon flour into dry measuring cups; level with a knife. Combine flour and next 5 ingredients (through ground allspice). Add flour mixture to banana mixture; beat just until blended. Pour batter into a 9 x 5-inch loaf pan coated with cooking spray. Bake at 350° for 55 minutes or until a wooden pick inserted in center comes out clean. Remove from oven; cool 10 minutes in pan on a wire rack. Remove bread from pan; cool completely. Combine powdered sugar and milk, stirring until smooth; drizzle over bread.