1. Preheat oven to 350°.
2. To prepare bread, combine first 5 ingredients in a large bowl; beat with a mixer at medium speed. Add granulated and brown sugars; beat until blended.
3. Weigh or lightly spoon flour into dry measuring cups; level with a knife. Combine flour and next 5 ingredients (through allspice) in a small bowl. Add flour mixture to banana mixture; beat just until blended. Stir in nuts. Pour batter into a 9 x 5-inch loaf pan coated with cooking spray. Bake at 350° for 1 hour and 5 minutes or until a wooden pick inserted in center comes out clean. Remove from oven; cool 10 minutes in pan on a wire rack. Remove bread from pan; cool.
4. To prepare glaze, combine powdered sugar, milk, and 1 tablespoon peanut butter in a small bowl, stirring with a whisk. Drizzle glaze over bread.