Cut a lengthwise slit in each banana, being careful not to cut all the way through; spread open slightly. Gently pack 1 tablespoon brown sugar inside each. Drizzle liqueur over brown sugar.
Grill, without lid, over medium heat (300° to 350°) 4 minutes on each side. Top each grilled banana with a small scoop of ice cream or a dollop of whipped cream. Serve immediately.