Preheat broiler or grill.
Remove pulp from lemons and discard, rinse rinds, and cut each rind into eighths.
In a large bowl, toss together chicken, lemon rinds, and pepper. Cover and refrigerate 20 to 30 minutes.
Thread chicken, alternating with lemon rinds and olives, onto 4 to 6 skewers. Grill or broil until meat is cooked through, about 5 minutes per side.