Combine water and sugar in a large saucepan; bring to a boil, stirring until sugar dissolves. Remove from heat, and add tea bags. Cover and steep 5 minutes; discard tea bags. Stir in juice; cool completely. Stir in milk.
Pour tea mixture into the freezer can of an ice-cream freezer; freeze according to the manufacturer's instructions.