Foul tastes in water tanks on the fridge have been an interest of mine for several years. It took lots of trial and error to finally grasp what was contributing to the problem. The first problem came to arise from the latest trend of filtering the incoming water. Although it is a good idea to remove certain contaminants, such as cryptosporidium cysts, etc., these filters also tend to remove the chlorine elements from the treated water, thus rendering the output water 'without preservatives", so to speak. Since the refrigerator conveniently holds 2 quarts of chilled water in a plastic reservoir, the water has time to sit and develop microscopic byproducts, many of which add taste and even white floaties to the water from the reservoir. In these cases, a new reservoir is recommended to rid the unit of the contamination. Not a difficult nor very expensive venture.
The other thing I found extremely interesting was the choice of water line used from the wall to the refrigerator. Many installers and hardware stores promote white or blue plasic vinyl water lines due to their flxibilty. These are a soft plastic, often seen with interwoven string in the opaque skin for reinforcement. The only problem with these is that they invariably add a nasty taste to the water which filters do not react to. It is a chemical taste in my opinion, but customers have described it in other, less tactful terms. My recommendation is to replace such an installation with the old standard 8-foot of 1/4" copper tubing coiled up for easy movement of the refrigerator. Copper will not addd one bit of taste to the water and will last a long time. Do the water line first, then replace the reservoir if the taste does not completely go away. You may not be able to flush the taste from the reservoir, as it is porous and retains precipitates.