Preheat oven to 375°.
Combine crushed dried rosemary, 1/4 teaspoon salt, and 1/8 teaspoon pepper.
Remove and discard giblets from hens. Rinse hens with cold water; pat dry. Remove skin; trim excess fat. Working with 1 hen at a time, place 1 lemon piece in the cavity of hen; tie ends of legs together with twine. Lift wing tips up and over back; tuck under hen. Repeat procedure with remaining hen and lemon piece. Rub hens with rosemary mixture. Place hens, breast sides up, on a broiler pan coated with cooking spray.
Toss potato with oil; sprinkle with 1/4 teaspoon salt and 1/8 teaspoon pepper. Arrange the potato around hens.
Insert a meat thermometer into meaty part of a thigh, making sure not to touch bone. Bake at 375° for 1 hour or until thermometer registers 180°. Remove twine.