Combine potatoes, butter, sugar, baking powder, and salt in a medium mixing bowl; mix well. Gradually add whipping cream and milk, stirring until well blended. Stir in lemon rind, juice, vanilla, and nutmeg. Add eggs; mix well.
Pour mixture into pastry shell; bake at 350° for 55 minutes or until a knife inserted in center comes out clean. Cool.