Beat butter, milk, cinnamon, and nutmeg at medium speed with an electric mixer until creamy. Gradually add powdered sugar and vanilla, beating until light and fluffy.
To make ahead: Prepare as directed. Chill in an airtight container up to 1 week. Let stand 20 minutes before using.