Carefully remove rind from oranges and lemon using a vegetable peeler, making sure to avoid white pithy part of the rind. Combine water and citrus strips in a saucepan; bring to a boil, reduce heat, and simmer, uncovered, 3 minutes. Remove from heat, and cool 15 minutes.
Remove citrus strips from water; discard strips. Combine juices in a bowl. Stir in citrus water and milk. Cover and chill 30 minutes.
Pour mixture into the freezer can of an ice-cream freezer; freeze according to manufacturer's instructions. Spoon sherbet into a freezer-safe container; cover and freeze 1 hour or until firm.