Melt butter in a large saucepan over medium-low heat. Add leeks and sauté until leeks are soft, but not browned, about 8 minutes. If leeks start to stick to saucepan, stir in a few teaspoons of water.
Add diced potatoes, 3 cups chicken broth and bay leaf. Bring just to a boil, lower heat and simmer until potatoes are tender, about 10 minutes. Add broccoli and remaining 1 cup of chicken broth. Raise heat to medium-high, bring mixture back to a boil, lower heat and simmer until broccoli is bright green and just tender, about 5 minutes. Take saucepan off heat and remove bay leaf.
In a blender, puree soup in batches until fairly smooth (soup will be very thick). Return puree to saucepan and slowly stir in cream. Warm soup over medium-low heat until just warmed through; do not let soup boil. Season with salt, pepper and nutmeg, if using. Serve soup hot, with homemade croutons or crackers, if desired.