Whisk together buttermilk, sugar, cornstarch, and egg yolks in a heavy 3-qt. saucepan. Bring to a boil over medium heat, whisking constantly, and boil 1 minute. Remove from heat, and stir in vanilla. Serve warm or cold. Store leftovers in an airtight container in refrigerator up to 1 week.