At a resort, the water should be safe but still it is not a good idea to drink it. It is ok to use for brushing teeth, and the ice cubes are made from treated water, so are also ok.

Many resorts provide free bottled water to their guests or fill up your bottle for free; if buying bottled water is not that expensive (but usually costs more in the resort store than in town).