Combine milk and cinnamon stick in a medium saucepan over medium heat; bring to a simmer (do not boil). Remove from heat, and cool to room temperature. Discard cinnamon stick.
Combine cocoa, sugar, and coffee granules in a small bowl; stir well. Add cocoa mixture to milk mixture in pan, stirring with a whisk until smooth. Stir in marshmallow creme. Cook mixture over medium heat 3 minutes or until thoroughly heated, stirring constantly with a whisk. Reduce heat to low, and simmer 1 minute, stirring vigorously with a whisk to froth mixture. Remove from heat; stir in vanilla. Pour into mugs.