The phrase ‘by the way’ appeared in Old English with a literal meaning as follows:
(1) 3if ic þe læd be þam weзe. (if I lead you along the way) (c888 K. Ǽlfred Boeth.XL.ξ5) [OED]
In Middle English, ‘by the way’ still had a literal meaning as in
(2) Þah he seo .. wundres bi þe weie, he ne edstont nawt as foles doð. (Though he saw wonders in the road-way, he doesn’t understand as a fool does ) (c1230 (?a1200) Ancr. 178/10) [MED]
It is around the sixteenth century that this phrase began to possess its idiomatic meaning (= used when saying something that is not related to the main subject you were talking about before) as in present-day English:
(3) By the way, thys place teacheth vs, that, etc. (1574 tr. Marlorat’s Apocalyps) [OED]
A further development of the phrase into parenthetical use has been seen in the following example:
(4) Though, by the way, this may not pass so generally, but that it must admit the exception, which the Rule of Law upon which it is grounded, carries with it. (a1614 DONNE Biathanatos) [OED]
A process whereby the phrase 'by the way' has changed from literal to figurative results from the flexibility of 'by the way' in terms of co-occurrence. In its literal meaning, 'by the way' often co-occurred with verbs of motion and rest, such as come, pass, travel; lodge, loiter, rest, stay and stop, including contexts where the meaning of 'on the road-side' is indicated. The following are examples:
(5) For feare (quoth he) that I be forlaid by the way, and rifled by him. (1603 Holland Plutarch's Mor. 667) [OED]
(6) If you fall by the way, don't stay to get up again. (1738 Swift Pol. Conversant. 98) [OED]
In due course, these co-occurrences began to be loosened, and 'by the way' was freed from this restriction. Consequently, this phrase has come to be placed in various positions in a sentence.
Traugott (1995: 31) suggests a unidirectional increase in subjectification, which she defines as "a pragmatic-semantic process whereby 'meanings become increasingly based in the speaker's subjective belief/attitude toward the proposition'" The sequential change of 'by the way' seems to reflect the tendency towards the expression of an emotive/subjective state. During this change, 'by the way' has gone through the stage of developing subjective and textual functions.
'Incidentally' comes from 'incident (adj.) + ly', meaning 'in an accidental manner.' The OED gives the following example:
(7) A Theme, that fell but incidentally under my consideration. (1665 BOYLE Occas. Refl. (1848) 56)
In present-day English, 'by the way' and 'incidentally' compete with each other. I will investigate when these two expressions began to be in rival relationship.
I shall use The Oxford English Dictionary CD-ROM, the Helsinki Corpus, the Archer Corpus, and Collins Wordbanks.
Traugott, Elizabeth C. 1995. Subjectification in grammaticalization. Dieter Stein and Susan Wright (eds.) Subjectivity and Subjectivisation, 31-54. Cambridge: Cambridge University Press.