I think Pirsig went over that in ZATAOMCM, but it's a fair point. I vaguely recall an African tribe who don't have words for some of the colours we take for granted and it appeared to change their perception of colour. In the end what the we see is interpreted by the brain, so if you spend your formative years being told "orange is red" you might well see (what we call) orange as red and an orange on a red tablecloth might be hard for you to resolve.
Neither did English till the late 1500's (at the earliest); anything that today we'd consider "orange" was lumped on the red spectrum, as we did have words for "red". This explains why we call foxes red; why the chest plumage of a robin is "red"; why people with hair which is too light to be brown and too dark to be blonde are called "red haired", when in all three cases, the colour ranges involved would not be termed "red" in the strict modern association of the word "red". More a russety browny-orange hue with red overtones.
the word "orange" arrived with the fruit - of unquestionably orange colour. As English didn't have a word for them, their not being native to northern Europe, the Spanish word
naranja was adopted. English mouths had problems with the Spanish pronunciation. And English grammar gave a single
naranja the indefinite article "a".
A naranja mutated into
an aranja. And that pesky
"j" softened into something like
an aranga. And then "an orange" - for both the fruit and by extension anything of that bright distinctive colour that was nether red nor yellow-gold but something in between.
An interesting related mutation involved the colours of the Dutch flag which was - still is - red-white-blue. At around this time the Dutch discovered oranges as a new fruit and had no native word for it either. And they noted that the national colours, especially after some months at sea or continual exposure to the elements on a flag, saw the dye of the red band fade to.... guess what colour. Hence their pesky ofshoot in the South African colony dispensed with the red and
their national flag became orange, white and blue from the start. And at home, the ruling house adopted the new colour and became - The House of Orange. (exporting this, indirectly, to Ireland where it also caused trouble, but different story).
It's an interesting linguistic and psychological point. Does adding a new colour term, opening up a new range of hues which people had hitherto lumped in with a different colour, open up new possibilities, expands the frontier of art and design? If the fruit had not come to Britain, would Sir Isaac Newton only have seen
six colours in the prismatic spectrum and gone straight from red to yellow? Richard York Gave Battle In Vain? Would the more militant Northern Irish groups march in "Red" sashes and be called the "Red"men?