Heat 4 teaspoons oil in a medium skillet over low heat. Add shallots to pan, and cook for 5 minutes, stirring occasionally. Sprinkle with sugar; cook 20 minutes or until shallots are golden, stirring occasionally.
Place potatoes in a medium saucepan; cover with water. Bring to a boil; cook 8 minutes or until tender. Drain. Place potatoes in a large bowl; beat with a mixer at medium speed until smooth. Add rosemary, salt, and pepper; beat until blended. Spoon into a bowl; top with shallots, and drizzle with remaining 1 1/2 teaspoons oil.