Combine 4 cups water and next 3 ingredients (through mint leaves) in a medium saucepan. Bring to a boil; reduce heat, and simmer, uncovered, 5 minutes. Cool 40 minutes. Remove and discard mint.
Place half of peach mixture in a blender; process until smooth. Pour pureed peach mixture into a 13 x 9-inch glass or ceramic baking dish. Combine remaining peach mixture and lime juice in blender; process until smooth. Add to baking dish.
Freeze until partially frozen (about 2 hours). Scrape with a fork, crushing any lumps. Freeze, scraping with a fork every 45 minutes, 1 1/2 hours, or until completely frozen. Scoop into dessert dishes, and garnish with peach slices and mint sprigs, if desired.