Beat butter and sugar at medium speed with an electric mixer until light and fluffy. Gradually add flour, beating at low speed until blended. Divide dough into 6 equal portions; roll each portion into a 12- x 1-inch strip. Place strips on lightly greased baking sheets.
Make a 1/2-inch-wide by 1/4-inch-deep indentation down center of each strip using the handle of a wooden spoon. Spoon half of jam evenly into indentations.
Bake at 350° for 15 minutes. Remove from oven; spoon remaining jam into indentations. Bake 5 more minutes or until lightly browned.
Whisk together powdered sugar, water, and extract; drizzle over warm shortbread. Cut each strip diagonally into 1-inch slices. Cool in pans on wire racks.