It seems like a good idea, or at least a plausible idea. They follow the light for a while, but it's not getting any closer. Joly's about to ask Combeferre if he doesn't think it might be some sort of optical illusion when he notices that they are not exactly walking through the usual Outdoors of Milliways. It's stone under their feet.

...and stone around them, too. Lot of twisty little passageways sprawling out on every side, all of them looking the same. He reaches out to catch at Combeferre's arm. "Ah-- look. When did this happen?"

Because it seems like something one of them should have noticed, surely.