Firstly, one cannot just go to the Chagos Archipelago. There are very strict rules pertaining to any yachts wishing to visit. The BIOT (British Indian Ocean Territory) is administered by Great Britain's Foreign and Commonwealth Office, to whom application must be made. Suggest that you read up on our Cruising Wiki :- HERE for INFO
As far taking a boat from Gan, that would be difficult as most charter operations are based in Male.
Again because special conditions are applied to visiting yacht in terms who is the skipper, who are the crew and who are the passengers. This would require everything to have been approved in advance of arrival in Male.
How long would it take to get from Male to Chagos ? Well it is some 600nm there and 600nm back, and because the passage is across the equatorial belt - there is often very little wind. Another problem is a very strong current on the journey down. In all probability this adds up to at least 6 days down and 6 days return. Another factor is that in Chagos there no facilities of ANY
kind whatsoever. Any visiting yacht is required to be totally self sufficient.