Vapid Shallow Models Must Die