Coat a large nonstick skillet with cooking spray; add margarine. Place over medium-high heat until margarine melts. Add celery, onion, and mushrooms; saute 3 to 4 minutes or until tender.
Combine flour and 2 tablespoons milk, stirring until smooth; add to vegetable mixture. Add remaining milk, sherry, pepper, and salt; cook, stirring constantly, until mixture is thickened. Stir in rice; reduce heat to low, and simmer 10 minutes.