The explosion results from a fire that spreads into the ammonium nitrate itself (Texas City, TX; Brest; West, TX; Tianjin; Beirut), or from a mixture of ammonium nitrate with a combustible material during the fire (Repauno, Cherokee, Nadadores). The fire must be confined at least to a degree for successful transition from a fire to an explosion (a phenomenon known as "deflagration-to-detonation transition"). Pure, compact AN is stable and very difficult to ignite, and numerous cases exist when even impure AN did not explode in a fire.
Been a lot of AN disasters over the years. Makes you wonder why they stored it in such a built up area?