A man who grew up as part of the only family on a remote island has recalled his experiences in a book