I'll take you there and watch you climb the cliff.
If you get out of the wrecked boat and sit on the beach, you are looking at the whole Atlantic running straight in at you, as you sit there, in November, with no shelter, for three days..
It's this sort of terrain.
They were only likely to be spotted from the sea, particularly at this time of the year and recent weather would keep people a long way off the coast there, for their own safety.
There is certainly an element of suspicion about what the purpose of their trip really was, but getting up out of there is not for the fainthearted. I suspect that it was only attempted when, after being there for three days, and realising that nobody had spotted them, it became a 'now or never' moment.