1. Combine the first 7 ingredients in a large bowl.
2. Trim tough outer leaves from fennel; mince feathery fronds to measure 1 cup. Remove and discard stalks. Cut fennel bulb in half lengthwise; discard core. Thinly slice bulbs. Add fronds, fennel slices, and orange sections to bowl; toss gently to combine. Sprinkle with olives.