Heat toaster oven to 450° F.
Place the potatoes, 2 tablespoons of the oil, 1 teaspoon of the salt, 1/4 teaspoon of the pepper, and 4 sprigs of the thyme in the toaster-oven baking tray and toss. Spread in a single layer and bake, stirring occasionally, until golden, 25 to 30 minutes. Transfer to a large piece of aluminum foil and wrap to keep warm. Wipe out tray. Place the cod in tray.
Thinly slice half the lemon. Top the cod with the lemon slices, garlic, capers, and the remaining salt, pepper, and thyme sprigs. Drizzle with the remaining oil. Bake until cooked through, 8 to 12 minutes, depending on size.
Transfer the cod and potatoes to individual plates. Squeeze the remaining lemon half over the tray and return to oven until the juices are warmed through, about 5 minutes. Pour the sauce over the fish and potatoes.