In a large bowl, mix lamb, rosemary, roasted peppers, salt, and pepper to taste. Form into 6 burgers about 3/4 in. thick. Put on a plate, cover, and refrigerate 2 hours.
Cut 12 slices, each about 1/2 in. thick, from bread and save the rest for another use. Lightly brush slices all over with olive oil.
With a silicone brush or oiled paper towels, lightly oil a charcoal grill over a solid bed of hot coals or a gas grill on high heat (you can hold your hand 1 to 2 in. above cooking grate only 2 to 3 seconds). Lightly toast bread on grill, turning once, and transfer to a platter.
Lay burgers on grill; close lid on gas grill. Cook burgers, turning once, until they're done the way you like, about 6 minutes for medium-rare. Set each burger on a slice of grilled bread. Top with pea relish, then remaining bread.