The skylight is your problem. Whenever it comes to skylights that emit a uniform amount of light, it completely washes out the effect of normal maps. Some times skylights are nasty things and to get correct lighting out of them requires either modeled geometry or displacement maps. You need actual surfaces to occlude and shadow that light for you, normal maps just don't do it because they require some kinda directionality of the light to work.