Preheat oven to 450°. Soak fillets in buttermilk. Coat bottom of a 13- x 9-inch metal baking pan with 1 tablespoon of oil; set aside. Put panko on a plate; add salt and pepper, and mix well to combine.
When oven is hot, pull fish from buttermilk; let it drain. Dredge still-wet fish in panko, patting to make sure panko adheres, then lay fillets in the prepared pan; drizzle with remaining 2 tablespoons oil.
Bake fish in top third of oven for 8-15 minutes, depending on thickness of the fish; it will be crisp and golden on the outside and tender and opaque inside when done.
Serve 1 fillet on each of 4 plates with lime wedges and 1/4 of new potatoes and ketchup.