A high temperature (up to 950 °C) sensor was proposed and demonstrated based on a micro taper in-line fiber Mach-Zehnder interferometer (MZI) structure. The fiber MZI structure comprises a single mode fiber (SMF) with two micro tapers along its longitudinal direction. Annealing at 1000 °C was applied to the fiber sensor to stabilize the temperature measurement. The experimental results showed that the sensitivity was 0.114 nm/°C and 0.116 nm/°C for heating and cooling cycle, respectively, and, after two days the sensor still has sensitivity of 0.11 nm/°C, showing good stability of the sensor. An probe-type fiber MZI is designed by cutting the sandwiched SMF, which has good linear temperature responses of 0.113 nm/°C over large temperature range from 89 °C to 950 °C. The probe-type fiber MZI temperature sensor is independent to the surrounding RI and immunity to strain. The developed sensor has a wide application prospect in the fields of high temperature hot gas flow and oil and gas field development.