In My Childhood In A Different Country (R) the rule for picking fruit off trees seemed to be that if it was reachable from the sidewalk one could have some, and if it was on public land one could have a reasonable amount. However since fruiting trees were expected to provide a fair amount of the diet of the people who owned them, it wasn't done to strip someone else's tree.

Comparing fallen fruit to toys on a lawn seems odd, though. We didn't generally let the fruit get so overripe it fell -- we harvested it off the trees.