Place potatoes in a large saucepan, and cover with water; bring to a boil. Reduce heat, and simmer 15 minutes or until tender; drain and cool slightly. Cut potatoes into 1/4-inch-thick slices. Combine potatoes, 2 tablespoons vinegar, salt, and pepper; toss gently. Let stand 5 minutes.
Combine 1 tablespoon vinegar, bell peppers, and remaining ingredients except parsley sprigs and whole anchovy fillets in a bowl; stir with a whisk. Pour over potatoes; toss gently to coat. Garnish with parsley sprigs and whole anchovy fillets, if desired. Serve at room temperature.