If you can believe the EPA's fuel mileage testing, the x-drive accounts for 1-2mpg.
All-wheel drive means more parts, more weight, and potentially greater expense. It also means that depending on the wear, if you need to replace one wheel, you may need to replace all 4. There is a maximum difference in rolling diameter that is fairly small when using an all-wheel drive vehicle or you induce excess wear in the differentials.
There are some really enlightening discussions of the differences in all-wheel verses rear-wheel drive in bad conditions on
www.tirerack.com website. Even with snow tires, they had a huge difference in accelleration and cornering capability on similar cars between the two drive systems. The all-wheel drive accellerated significantly quicker, and went around corners at higher speeds under control compared to rear wheel drive in bad conditions.
That extra help in adverse conditions seems worth it to me. The system BMW uses keeps many of the characteristics of a rear-wheel drive until there is some slip, so the impact on fuel economy is more from the weight rather than say on an Audi with the torsen differential which has all wheels driven all the time (not all Audis use the same system).