One Step Inside Doesn't Mean You Understand