Heat 2 teaspoons peanut oil in a large nonstick skillet over medium-high heat. Add the pork, and stir-fry for 3 minutes. Remove pork from pan, and keep warm. Add 2 teaspoons peanut oil, ginger, and garlic, and stir-fry for 1 minute. Add the onion, asparagus, bell pepper, and peas, and stir-fry for 5 minutes. Remove from pan, and keep warm.
Add curry paste, sugar, and coconut milk to pan, and stir well. Bring the curry paste mixture to a boil; reduce heat, and simmer for 10 minutes. Add the pork, asparagus mixture, and rice noodles to pan, and cook for 1 minute or until mixture is thoroughly heated. Sprinkle with basil; garnish with lime wedges.