I differ from Chalmers in that I think it is possible they actually see different colors, because of the fact that as a change occurs in the color areas of Sheila's brain, an analogous change will occur in the color areas of Sam's brain. Consider that I have the switch flipped to "Sheila's color lobe", meaning that all the color processing that Sam would normally do with his own brain areas is now being done by Sheila's. So I show Sam a red fire truck, the information travels into his eye, along his optic nerve, is then sent by radio signal to Sheila's brain, where it is processed, and sent back to some deeper area in Sam's brain. I ask him what color it is, and he says "red", when he actually experiences green, because he is using Sheila's brain areas to process the color, and Sheila experiences green where most people experience red. While he is experiencing a green fire truck, changes in Sheila's brain areas are occurring, so something is detecting them and analogous changes are being made to occur in the currently idle analogous areas of Sam's brain. Leaving the fire truck in front of Sam, I go flip the switch to "Sam's color lobe", and ask him "is the fire truck still the same color that it was a second ago?". Upon hearing this information, Sam's brain begins processing to try and answer the question. As some part of the process, he searches through the area of his brain that has just been switched on, and finds the memory of the fire truck's color a few seconds ago. Of course, this memory is no reflection of the color sensation that he "really experienced" because changes were being made in this area when it was not even being used. The changes that Sheila's color areas underwent which were recorded in Sam's color areas corresponded to memories of what color he had just seen. Sam will recall these memories and report on them, and because the changes were completely analogous, the report will be that he saw the color red a second ago and still sees it. It might seem that this is a result of the "analogous changes recorded in one person's brain area will be made to occur in the other's brain area" requirement, but this requirement is necessary for functional equivalency. Otherwise, if changes in Sheila's brain areas while they are active are not made in Sam's brain areas while they are inactive, then when we switch to Sam's brain areas and ask him about the fire truck, his brain will search around and discover that he doesn't remember what color the fire truck was a few seconds ago, because no memory was recorded. 
Chalmers' is a very creative attempt at a solution to the problem, the most thoughtful one I have seen, but it too has flaws, so I must conclude that it is completely possible for two people to have different experiences of color, but that even with the most advanced neuroscience it would be impossible for anyone to ever tell. I've read of a study where people with brain lesions in their visual cortex can be put in a situation where a half green and half red bar is displayed on a computer screen, and that an eye tracking device makes sure that no matter where they move their eyes, the border between the red and green always stays in the area of their visual field which they can not see because of their lesion (people with lesions in their visual cortex "fill in" the missing information). When asked where the border stops, the subjects report that instead of a discrete border, they see a steady gradient between red and green, and that the color that makes up the center of the gradient is a completely new color which is somehow both green and red. Assuming that they aren't lying, the fact that I cannot possibly imagine such an experience starkly illustrates the inaccessibility we have to other people's color experiences. 
