Yes it was the case, broadly pomme meant fruit.

Originally tomatoes were called pommes d’amour. There is pommes de terre (potatoes) pommes de pin (pinecones). If we would translate literally pomme de pin it would give us pineapple.

There were also other fruits beginning with pomme originally but I can’t seem to recall them.