On a cutting board, finely chop the drained capers with the oregano leaves and garlic. Transfer the mixture to a bowl and stir in 1/2 cup plus 2 tablespoons of the olive oil, along with the lemon zest and lemon juice. Season the sauce with pepper.
Light a grill. In a large bowl, toss the shrimp with the remaining 2 tablespoons of olive oil and season lightly with salt and pepper. Thread the shrimp onto metal skewers and grill over high heat, turning once, until the shrimp are lightly charred and cooked through, about 3 minutes per side. Remove the shrimp from the skewers and transfer them to a platter. Spoon the sauce on top and serve.