Ianto's expression tightens slightly, because there he goes again, even after Ianto's pointed out he's on to him at this point too. On to the fact that he's dodging his questions, and avoiding Ianto's interest in him as well. Ignoring his interest when Ianto knows for a fact he hadn't pushed him away before.

"Is that all you're interested in then?" he pushes. "A little verbal back and forth to keep you amused? Is that why you're keeping me held out at arm's reach?"