Preheat oven to 400°.
Combine water, vinegar, 4 tablespoons sugar, and peppercorns, stirring with a whisk until sugar dissolves. Scrape seeds from vanilla bean; add seeds and bean to vinegar mixture. Place plums in a 13 x 9-inch baking dish. Pour vinegar mixture over plums. Nestle 2 rosemary sprigs around plums into vinegar mixture. Sprinkle evenly with 2 tablespoons sugar.
Bake at 400° for 20 minutes or until plums are tender (skin will begin to split on some plums).
Remove plums with a slotted spoon to a serving platter. Strain vinegar mixture into a small nonaluminum saucepan; discard solids. Bring vinegar mixture to a boil. Reduce heat to medium-high; cook until reduced to 3/4 cup (about 5 minutes). Pour syrup evenly over plums; garnish with rosemary sprigs.