Coastguard Rescue Helicopter 175 is sweeping the area between Cowes and Yarmouth, and Coastguard Rescue Teams from Bembridge, Ventnor, and The Needles are working along the coast.
The Cowes RNLI lifeboat is also involved in the incident, which started around 3.30pm
Search teams are looking for a male who is now officially missing.
The search is being conducted by a number of agencies.
HM Coastguard is now seeking information.
At 3pm this afternoon, a male swimmer was seen potentially in difficulty 400m off the Royal Yacht Squadron, heading towards Gurnard Bay.
If you have any information, please contact the Solent Coastguard at 02392552100. In an emergency, dial 999.
The ongoing search has been joined by police and WightSAR.