The shortening of mortal lives is found in The Silmarillion. That's why the Numenoreans were forbidden from seeking out Valinor, not because Eru wanted to be mean and keep immortality from them, but because it would actually shorten their lives significantly to go there. To attempt to use the power of Valinor to try to expand their lives would have been a torment to them, similar to the effects of mortals who bear the One Ring. Tol Eressea is somewhat removed from Valinor proper, so the effects wouldn't have been as intense if Frodo and Bilbo had gone to Valinor but they would have still been present. As for time moving differently there, there's nothing in cannon to support that, at least not that I've been able to find.