Habit formation apps are intended to help instigate and maintain new behaviors. Prior research has established that these apps mostly do not support the theoretical 'habit' construct defined in psychology, yet are generally popular and well reviewed in app stores. This apparent mismatch between theory and 'in-the-wild' usage has not been investigated to date. Through an in-depth qualitative study of a popular application Lift, this research establishes that common techniques such as reminders and streaks are effective at supporting repetition of new behaviors, but at the same time create a dependency: on-going app use is often required to achieve lasting change. This dependency introduces fragility in users' attempts to change their behavior, as they often abandon the app and subsequently disengage with their new behaviors.