Preheat oven to 400°.
Remove yellow part of rind from lemons, using a vegetable peeler and leaving inner white skin on fruit; finely chop rind, and set aside. Squeeze juice from lemons; set juice aside. Reserve juiced lemon halves.
Remove and discard giblets and neck from chicken. Rinse chicken with cold water; pat dry. Trim excess fat. Stuff body cavity with reserved juiced lemon halves.
Combine reserved lemon rind, rosemary, and next 3 ingredients in a small bowl; stir in 2 tablespoons reserved lemon juice. Discard remaining lemon juice.
Combine potatoes and olive oil in a large bowl; toss well. Add 2 tablespoons lemon rind mixture to potatoes, tossing to coat; set aside.
Starting at neck cavity, loosen skin from breast and drumsticks by inserting fingers, gently pushing between skin and meat. Sprinkle remaining lemon rind mixture under loosened skin and rub over breast and drumstick meat.
Place chicken, breast side up, in a broiler pan on a rack coated with cooking spray. Arrange potatoes in a single layer around chicken. Bake at 400° for 1 hour or until meat thermometer inserted in meaty part of thigh registers 180°. Cover chicken loosely with foil; let stand 10 minutes. Discard skin.