Combine vinegar, sugar, salt, and olive oil in a small saucepan, stirring well; cook over low heat until slightly heated. Place horseradish in a small mixing bowl; pour warm mixture over horseradish, stirring well.
Ladle horseradish into hot sterilized jars, leaving 1/4-inch head space. Cover with metal lids, and screw bands tight. Process in boiling-water bath 20 minutes. Use as an accompaniment to cold meats. Store in refrigerator until ready to serve.