The taste may mellow with time, as many flavors do. I would give it a chance.
Have you considered using a bottle rinser like the one in the link below and using an iodophor solution, 12.5 ppm, which is no-rinse at that concentration?
Taste tests have shown this level to not affect taste and you don't have to drain the bottles. The rinser in the link sprays the solution well around inside the bottle. I use it in beermaking. The solution comes out pretty quickly and is used again for the next bottle.
If you wish, you can also use a bottle draining tree like the one below but I don't know that it will handle regular wine bottles. It is designed to hold regular 12 or 16 ounce beer bottles. The rinser above can clip onto the bottle tree, by the way, or be used separately.
Don