If swimming was a part of your training and you swim hard, then a smaller taper would be called for. If you are just doing laps, though, then I wouldn't worry about it. Getting the exercise in w/o the stress on your legs will probably do well for you. You might consider taking a few days off from the pool leading up to the race, but I wouldn't worry about swimming through Thursday or even Friday.

Agree with Blaine. The main goal of the taper is to allow your legs to recover from all the wear and tear and reacharge. Swimming would not be a problem at all, and as long as you don't really do very hard swimming workouts, your swims should be fine.