Combine first 11 ingredients in a medium saucepan. Stir well; bring to a boil over medium heat. Reduce heat; simmer 30 minutes. Strain brandy mixture through a sieve into a bowl, and discard solids. Return brandy mixture to pan. Add figs; bring to a boil. Reduce heat; simmer, uncovered, 1 hour or until figs are soft. Remove figs from pan, reserving brandy mixture. Let figs cool slightly; cut each fig in half.
Arrange 5 fig halves on each of 6 dessert dishes; drizzle 1 1/2 tablespoons brandy syrup over figs. Top each serving with 2 tablespoons ice cream. Garnish with lemon zest, if desired.
Note: This can be made a day ahead; cover and chill, and bring back to room temperature and serve.