In a medium bowl, whisk together butter-milk, vanilla, almond extract and 3 Tbsp. honey until smooth. Pour 2 Tbsp. buttermilk mixture into each of 6 1/2-cup ice pop molds and freeze until slightly firm, about 1 hour.
If using fresh peaches, bring a pot of water to a boil over high heat. Have a large bowl of ice water ready. Add peaches to boiling water and cook just until skins loosen, about 30 seconds. Using a slotted spoon, transfer immediately to ice water. Let stand until cool enough to handle, about 30 seconds.
Peel peaches and cut into wedges; discard peels and pits. Place peach wedges (or frozen peaches) in a food processor with remaining 2 Tbsp. honey and puree until smooth. Transfer 3/4 cup peach puree to a small bowl. Stir remaining 3/4 cup puree into remaining buttermilk mixture. Reserve any extra puree for a separate use.
Remove popsicle molds from freezer and top each with 1/4 cup peach-buttermilk mixture. Freeze until slightly firm, about 1 hour. Remove from freezer and top each mold with 2 Tbsp. peach puree. Insert sticks according to manufacturer's instructions. Freeze until firm, at least 3 hours and up to 1 week. Unmold and serve.