Combine first 5 ingredients in a shallow dish. Combine buttermilk and hot sauce in a medium bowl; place flour in a shallow dish. Dredge 1 fillet in flour. Dip in buttermilk mixture; dredge in breadcrumb mixture. Repeat procedure with remaining fillets, flour, buttermilk mixture, and breadcrumb mixture.

Heat 1 1/2 teaspoons oil in a large nonstick skillet over medium-high heat. Add 2 fillets; cook 3 minutes on each side or until fish flakes easily when tested with a fork. Repeat procedure with remaining oil and fillets. Serve with lemon wedges.