Place potatoes in a large pot and cover with cold salted water. Bring to a boil, reduce heat to medium and cook until potatoes are tender, about 15 minutes. Drain and return potatoes to pot; cover. Let stand, off heat, until softened, about 10 minutes more.
In a small bowl, whisk together yogurt, mustard and vinegar.
When potatoes are cool enough to handle, cut into quarters. Place in a large bowl and add celery, shallot and chives. Add yogurt mixture, season with salt and pepper and gently stir.