Soak rosemary spears in water several hours or overnight.
Combine olive oil, lemon juice, salt, and pepper in a large heavy-duty zip-top plastic bag; add swordfish, bell pepper, and onion. Seal bag, and chill 30 minutes, turning occasionally. Thread fish and vegetables onto rosemary. Grill kebabs, covered with grill lid, over high heat (400° to 450°) 5 minutes on each side.