Expanding the context window is a double-edged sword. The potential for enhanced learning and interaction is apparent, but so is the risk of reinforcing inherent biases at a larger scale. We must tread carefully in the field of artificial texts.