Place apples in a large Dutch oven. Peel and juice lemons; add lemon peel and juice to apples. Stir in water and sugar; bring to a boil. Reduce heat; cover and simmer 20 minutes.
Add tea bags; cover and steep 5 minutes. Remove tea bags, apples, and lemon peel; discard. Stir in rum.
Place fresh apple and lemon slices in mugs; pour tea mixture over fruit. Sprinkle with nutmeg, if desired. Serve hot.