Users banned on Xbox One will not lose access to the games they've purchased, Director of Programming Larry Hyrb (Major Nelson, colloquially) told Reddit Games during an E3 video interview.
This became a popular question following an Xbox Support Twitter answer from June 13 that read, "If your account is banned, you also forfeit the licenses to any games that have licenses tied to it as listed in the ToU." The question specified Xbox One, but later, Xbox Support said this answer was in reference to Xbox 360.
Major Nelson had a clear answer for future banned users on Xbox One: "Absolutely not, you will always have access to the games you purchased."
As for what will happen if, down the road, Xbox One's authentication servers are shut down, Major Nelson couldn't say. "I'll get the real answer, I just don't know it yet," he said.