In a large bowl, combine butter, orange zest, lime zest and lemon juice. Beat with a handheld mixer or a wooden spoon until blended. (Butter can be made up to 2 days in advance. Cover and refrigerate; remove from fridge 15 minutes before serving to soften.)
Prepare a charcoal grill and let burn until coals are glowing, or set a gas grill to medium-high.
Shuck corn, discarding husks and silk. Brush with a small amount of olive oil and place on hot grill. Cook, turning often, until slightly charred in spots and soft, 5 to 10 minutes. Sprinkle with salt and pepper. Place corn on a large platter and top with a small scoop of butter. Serve remaining butter on the side.