Oliver comes through Old French Olivier. One theory connects it to the Latin word for olive, while other explanations trace it to a Germanic name.
The olive association has become part of how people read the name. The historical derivation remains open, so the two explanations should be kept in view.