1. Bring 4 cups water to a boil in a medium saucepan. Remove pan from heat, and add tea bags. Cover and steep 5 minutes. Remove and discard tea bags. Cool tea completely in pan.
2. Combine 3/4 cup water, sugar, chopped plums, and thyme in a medium saucepan. Bring to a boil, stirring until sugar melts. Reduce heat, and simmer, uncovered, 15 minutes or until plums are very tender and have lost their shape. Strain plum mixture through a sieve into tea, pressing with the back of a spoon to extract juice. Discard solids. Stir tea mixture; cover and refrigerate until thoroughly chilled.
3. Serve tea over ice. Garnish with plum and lemon slices, if desired.