Alright, here's my first crack at something like a Xen Wiki post. Of course some bits are lifted directly from your writings on this topic:
N2D3P9 is a function which, given a pitch ratio, estimates its rank in popularity among all pitch ratios in musical practice. A low N2D3P9 indicates that the ratio is used often, while a high N2D3P9 indicates that the ratio is used rarely.
N2D3P9 was developed to help decide which ratios are the most important for a JI microtonal notation system to notate in the simplest way. Because factors of 2 and 3 in pitch ratios can already be notated by changes in the nominal (C, D, E, F, G, A, B) or the count of sharps and flats, N2D3P9 ignores factors of 2 or 3 in the ratio.
As an example, N2D3P9(5/4) ≈ 1, representing the fact that 5/4 is the simplest possible ratio with a factor other than 2 or 3, and therefore the most popularly used in musical practice. N2D3P9(8/5) shares the same value as N2D3P9(5/4), because 5/4 and 8/5 are essentially equivalent. As another example, N2D3P9(80/77) ≈ 39, suggesting that approximately 38 other undirected 5-rough ratios exist which are more popularly used than 77/5.
Formula
The formula for N2D3P9(`r`) is:
$$ \frac{nd⋅\text{gpf}(nd)}{2^{Ω(n)}3^{Ω(d) + 2}} $$
where
Note that `nd` is established in music theory as a ratio's
Benedetti height, or product complexity.
N2D3P9 can be rewritten as
$$\frac{n}{2^{Ω(n)}}×\frac{d}{3^{Ω(d)}}×\frac{\text{gpf}(nd)}{9} $$
Then note that where
`n = 5^{n_5}×7^{n_7}×...`
we have
`2^{\Ω(n)}=2^{n_5}×2^{n_7}×...`.
and so
`\frac{n}{2^{Ω(n)}}`
can be written as
`(\frac{5}{2})^{n_5}×(\frac{7}{2})^{n_7}×...`
and
`\frac{d}{3^{Ω(d)}}`
can be written as
`(\frac{5}{3})^{d_5}×(\frac{7}{3})^{d_7}×...`
These can be described as "product of half prime factors of the numerator (with repeats)" and "product of one-third prime factors of the denominator (with repeats)". It may be simpler to describe it verbally as:
Take the prime factorization of the numerator and divide all the primes by 2, then multiply it out again. Do the same with the denominator but divide by 3 instead of 2. Multiply these two results together then multiply by the prime limit of the ratio and divide by 9.
Development
N2D3P9 was developed from May to August 2020 as a collaborative effort among members of the
Sagittal forum, led by Sagittal co-creator
Dave Keenan and
Douglas Blumeyer. Many functions besides N2D3P9 were considered before selecting it as the best function for its purpose.
Estimation of pitch ratio popularity is possible because it correlates with numeric simplicity. N2D3P9 is most useful when comparing ranks of more complex ratios, because usage data about such ratios is sparse. By fitting a function to the statistical usage data which is available for simpler ratios, N2D3P9 enables the extension of the patterns found in these simpler ratios.
Actual pitch ratio usage statistics were taken from
the Huygens-Fokker Foundation's scale archives for their popular microtonal software Scala. Each occurrence of a pitch ratio in a scale was counted as one vote for that ratio. Data on the relative popularities of the scales in the archive was not available.
Rather than attempt to fit functions to the exact counts of votes for each ratio, the functions were fit to the rank indices of each ratio; in other words, a function only needed to sort ratios the same as the actual data, and within each rank position it was unimportant how close its estimate of votes was. In technical parlance, the goal was to minimize the
Spearman’s rank coefficient between the estimated ranks and the actual ranks. For purposes of comparing competing functions, minimizing Spearman’s rank coefficient could be simplified to minimizing the sum of squared differences between the ranks. And because fitting to the simpler ratios which had more votes is more important, the squared differences were weighted; the vote counts were observed to conform to the
Zipf’s law, so this was selected as the weighting mechanism. A
fractional ranking strategy was used to ensure that stretches of the data with tied vote counts did not distort the measurement.
The overall strategy, then, was to minimize this weighted rank correlation, while also minimizing the complexity of the function, to avoid
overfitting. The 5-rough-ratio notational popularity ranking function that had been used by the creators of Sagittal was `\text{sopfr}` (
sum of prime factors with repetition), and as simple as this function is, it does a remarkably good job of estimating the rank of pitch ratios. For comparison, the weighted sum of squares `\text{sopfr}` gives for the Scala stats is about 0.026, while the weighted sum of squares N2D3P9 gives is about 0.010. Functions giving sums of squares as low as 0.008 were found, however, these functions were so complex that they probably were fitting to noise in the Scala stats instead of to the true nature of musical pitch. An informal “chunk” metric was devised to compare function complexity in terms of fit to the data, with considered functions ranging from one chunk (`\text{sopfr}`) to eight chunks; the winning function N2D3P9 has five chunks.
Several techniques were used to find and decide on N2D3P9 as the best 5-rough ratio notational popularity rank estimation function. Initial observations about shortcomings of `\text{sopfr}`, such as its failure to differentiate balanced ratios from their imbalanced equivalents — such as 11/5 versus 55/1 — or those with different prime limits such as 13/5 and 11/7, despite those pairs of ratios exhibiting remarkably different actual ranks in the Scala stats, formed the basis of the investigation. Psychoacoustic plausibility of functions was used as a top-down guide for experimentation.
Optimization tools such as
Excel's Evolutionary Solver were used to navigate toward ideal values for each parameter. A brute-force technique was also utilized whereby nearly 2 billion functions combined out of constituent "submetrics" were checked automatically. In the end, one of the functions generated from the brute-force checker was recognized as being re-writable in a much simpler form with parameter values rounded to whole numbers without doing much damage to its sum-of-squares, and thus N2D3P9 was born.