developing a notational comma popularity metric

User avatar
cmloegcmluin
Site Admin
Posts: 783
Joined: Tue Feb 11, 2020 3:10 pm
Location: San Francisco, California, USA
Real Name: Douglas Blumeyer
Contact:

Re: developing a notational comma popularity metric

Post by cmloegcmluin »

I've finished refactoring my code. It now:
  • supports testing different powers on the monzo term between the diminuator and the numinator for the same submetric (which the way my code looks at things meant allowing multiple submetrics with the same type, and a weight on the numinator so it can be zeroed out).
  • uses fractional ranks for the real Scala votes (as well as in the estimated antivotes).
  • supports your mcopfr, where the 5-term gets half a count.
  • supports weighting submetrics by powers or bases instead of coefficients.
  • supports weighting numinator or diminuator by powers or bases instead of coefficients.
I've gone back and gathered out every metric specification we've dropped on the forum (since page 8) and represented them in my code. I confirm most of your findings. Not perfectly, though, due to the newfound fractional ranking system. One of them I even got slightly better results for, even without optimizing, e.g. your ones from way back on page 8 which still included prime limit in them I get 0.00612 for.

The ones I can't get exactly are your mcopfr ones. Your "2 parameter" one you claimed 0.00651 but I find 0.00721. Your "3 parameter" one you claimed 0.00614 but I find 0.00740. Your "4-parameter" one you claimed 0.00660 but I find 0.00838 (and yes, I did add a little hack in to handle your "h").

I have not sought optimizations for any of these yet (some you explicitly told me not to do so for).

I'm sorry that I didn't notice your middle-of-my-night post with the sanity check; yes, I got 0.00832555386875032 as my SoS for that one which checks out exactly with yours.

My next steps:
  • experiment with these new weighting styles
  • experiment with whatever you get back to me with re: the 23-param per-term weight vals subproject of yours
  • finally get around to implementing an automated minima finder

User avatar
Dave Keenan
Site Admin
Posts: 1084
Joined: Tue Sep 01, 2015 2:59 pm
Location: Brisbane, Queensland, Australia
Contact:

Re: developing a notational comma popularity metric

Post by Dave Keenan »

cmloegcmluin wrote:
Sun Jul 05, 2020 5:52 am
Exactly correct. I promise I'm not intentionally speaking in riddles! I really am striving for unambiguity here. It's just friggin' hard. (I think your AYBABTU reference was particularly astute, since the fidelity of translation between my language and your language about this material is still improving!)
I never imagined otherwise. Yes. The remarkable thing is that we chimpanzees can communicate at all, and thereby cooperate, on something that has almost nothing to do with what helped our ancestors survive for the last 200,000 years. Never mind a tiny hiccup like this.
The powers I was referring to were "These fit as y = 0.86871884, v = 0.712508978." Those are powers right? Maybe my mistake had been assuming that those were the grand conclusion of that subproject. Was the grand conclusion actually the coefficients per term of the monzo?
Technically they are exponents, but they are often loosely called powers, yes. Yes the grand conclusion is the general shapes of the coefficients per term of the monzos (the terms of the vals) as a function of the primes.
If I'm interpreting the charts correctly, we could say that there's something a bit "off" about 17's in the denominator; for some reason we need them to contribute fewer antivotes than they otherwise would in order for our metric to correlate best with the real data. And for 41's in the numerator, on the other hand, we need them to contribute more antivotes than they otherwise would.

I was assuming these per-term coefficients would not exist in the final metric, and that you were using them as a stepping-stone to find a best-fit line which would then give us our y (and v) powers for the final metric.
Quite right. I have no desire for our metric to give special treatment to favour 17 or disfavour 41. I believe 17 is favoured only because of the fact that it appears in many rational approximations of 12edo. And 41 is disfavoured because 2,3-reduced ratios of 41 are barely distinct from ratios of 5, due to the comma 729/656 = 81/80 × 81/82.

Hike day.

BTW, You should know that I often read your posts and emails on my hikes, and think about my future response, while slogging up a slope, when my sister has left me far behind. Of course the slope must be not too steep or too rocky, so I can get away with not looking so closely at the ground. :)

User avatar
cmloegcmluin
Site Admin
Posts: 783
Joined: Tue Feb 11, 2020 3:10 pm
Location: San Francisco, California, USA
Real Name: Douglas Blumeyer
Contact:

Re: developing a notational comma popularity metric

Post by cmloegcmluin »

Image
Dave Keenan wrote:
Sun Jul 05, 2020 10:47 am
Technically they are exponents, but they are often loosely called powers, yes. Yes the grand conclusion is the general shapes of the coefficients per term of the monzos (the terms of the vals) as a function of the primes.
Oh no way! I was familiar with the difference between a power function and an exponential function — f(x) = xn and f(x) = nx, respectively — but I had always assumed that these names were chosen somewhat arbitrarily, since the operation was the same, and the only difference was the location of the variable, so they just had to pick one synonym for one and the other synonym for the other. A brief web search turns up multiple informal sources suggesting that an exponent is the bit typically written in superscript while a power is the base along with the exponent. Is that what you mean?

I guess I had always thought when I said something like "two to the fifth power" I would diagram the phrase where "fifth power" was a thing to which "two" was raised, but it would seem the truth is closer to "power" being a thing and the answer to "what power" would be "two to the fifth"!

I continued my search to see if perhaps an analogy could be formed with base and logarithm, where the base is the bit in subscript and the logarithm is the whole thing, i.e.:

exponent : power :: base : logarithm

I couldn't find this thought asserted explicitly anywhere, but general practice seems to fit with this conception.

I'll have some tweaks to make to my nominally-typed arithmetic system I built for my musical patterns project. Thanks for bringing this to my attention.
Of course the slope must be not too steep or too rocky, so I can get away with not looking so closely at the ground. :)
Careful out there!

If I was you, I'd be more worried for my phone. The only times I've ever lost or damaged my phone were dropping it on rocks or gravel. But that I've done not once, not twice, but thrice.

User avatar
Dave Keenan
Site Admin
Posts: 1084
Joined: Tue Sep 01, 2015 2:59 pm
Location: Brisbane, Queensland, Australia
Contact:

Re: developing a notational comma popularity metric

Post by Dave Keenan »

Good work on your software. I loved the 2001 image, thanks. A timely warning about dropping my phone. I was extra careful.
cmloegcmluin wrote:
Sun Jul 05, 2020 12:32 pm
A brief web search turns up multiple informal sources suggesting that an exponent is the bit typically written in superscript while a power is the base along with the exponent. Is that what you mean?
Yes.
I guess I had always thought when I said something like "two to the fifth power" I would diagram the phrase where "fifth power" was a thing to which "two" was raised, but it would seem the truth is closer to "power" being a thing and the answer to "what power" would be "two to the fifth"!
Yes. But it's not that clearcut. It's messy. Consider this:
https://mathworld.wolfram.com/Power.html
They define power as synonymous with exponent, but then they proceed to use power to also mean the combination of base and exponent, and completely fail to notice or mention this!
I continued my search to see if perhaps an analogy could be formed with base and logarithm, where the base is the bit in subscript and the logarithm is the whole thing, i.e.:

exponent : power :: base : logarithm

I couldn't find this thought asserted explicitly anywhere, but general practice seems to fit with this conception.
Agreed.

User avatar
cmloegcmluin
Site Admin
Posts: 783
Joined: Tue Feb 11, 2020 3:10 pm
Location: San Francisco, California, USA
Real Name: Douglas Blumeyer
Contact:

Re: developing a notational comma popularity metric

Post by cmloegcmluin »

Dave Keenan wrote:
Sun Jul 05, 2020 7:40 pm
Good work on your software. I loved the 2001 image, thanks. A timely warning about dropping my phone. I was extra careful.
For a moment I thought you'd failed to recognize my implication that the 2001 image was about your chimp survival → comma metric formulation, instead misconstruing it as about you dropping your phone! :lol:

I did right away figure out that you had almost certainly construed things correctly, though:

The first sentence in this paragraph – a reaction to a completely different part of my previous reply – suggests that the sentences of this paragraph are not meant to be read as flowing from one into the next, building upon each other. They are instead united by a strategy: responding to each main topic of my previous reply, one sentence at a time. In other words, this paragraph is a montage sequence, not a narrative one using continuity editing. This is an effective and expeditious strategy and one I'm sure I've used myself several times, on this forum and elsewhere.

(So, you used montage in your response to my use of arguably the most famous instance of montage in cinema history itself as a montage response to your comment about our communication about this subject. Meta much???)

My main critique of the paragraph, then, would be the fourth sentence, which does flow out of the third sentence, and that's what triggered my concern that the third sentence in turn flowed out of the second. It also dents my description of the paragraph as being "one sentence per topic". But my objections could be completely addressed with a change as simple as altering the period between the third and fourth sentences to a semicolon.

:mrgreen:
It's messy. Consider this:
https://mathworld.wolfram.com/Power.html
They define power as synonymous with exponent, but then they proceed to use power to also mean the combination of base and exponent, and completely fail to notice or mention this!
Well, at least I can get it right in what I write now.

User avatar
Dave Keenan
Site Admin
Posts: 1084
Joined: Tue Sep 01, 2015 2:59 pm
Location: Brisbane, Queensland, Australia
Contact:

Re: developing a notational comma popularity metric

Post by Dave Keenan »

Your accurate understanding of the first paragraph of my hastily knocked-out post, shows the importance of "theory-of-mind" in enabling human communication. But this depended entirely on your ability to accurately model my state of mind when writing it.

However you are mistaken when imagining that my (inept) use of textual montage was in response to the sequence you referenced from 2001, which, by the way, is not really an example of montage, but arguably the most famous example of a match cut in cinema history — 2 million years of technological advancement summed up in an instant — although not nearly as well matched (in angle and screen location) as it might have been.

I agree that a semicolon would have improved my paragraph immensely, although since WinCompose, I've taken to using thin-space em-dash thin-space instead. This can be obtained by typing ⎄␣- (compose space hyphen).

While walking, I mentally tried various ways to consistentise the whole power/base/exponent/log terminology debacle. The two major categories of such attempts involve power = constant exponent (similar to coefficient = constant multiplier) versus power = baseexponent.

My best attempt to retain "power = constant exponent" goes like this:

"Power function" could be considered a shortening of "poweral function", as opposed to "exponential function". A "poweral" function must have a constant exponent, so the argument must become the base. An "exponential" function can have a variable exponent, and if it is to be different from a poweral function, then it must have a variable exponent, so the argument must become the exponent and the base must be a constant.

The operations could be called "poweration" versus exponentiation, although by analogy with "exponentiation" it would make more sense if causing the argument to become the base was called "basiation" and a power function was instead called a "basal" function (as contrasted with an exponential function).

But then what do we call the binary function xy when both are variable?

And why isn't there a word for a constant base if there is one for a constant exponent. Ah, but a "base" is a constant. Consider the case of a logarithm base or a number-system base. So we actually need a word for a variable base. It turns out we already have one. It's called a "root". The 4th root of 34 is 3. So why don't we say that the base-3 exponent of 34 is 4. Instead we say the base-3 logarithm of 34 is 4. But the logarithmic is considered the inverse of the exponential, not a synonym for it! And my head explodes. :exploding_head: :boom:

Going the other way works much better, i.e. power = baseexponent or even power = rootlog. One only has to invoke a historically recent slip of the tongue in going from 34 being "the 4th power of 3" to the still-acceptable "3 to the 4th power" to the mistaken "3 to the power 4". The mistake is in dropping the "th".

Actually, no, the mistake is in keeping the word "power" while dropping the ordinal "th". Calling it simply "3 to the 4" is fine.

User avatar
cmloegcmluin
Site Admin
Posts: 783
Joined: Tue Feb 11, 2020 3:10 pm
Location: San Francisco, California, USA
Real Name: Douglas Blumeyer
Contact:

Re: developing a notational comma popularity metric

Post by cmloegcmluin »

Dave Keenan wrote:
Mon Jul 06, 2020 12:10 pm
However you are mistaken when imagining that my (inept) use of textual montage was in response to the sequence you referenced from 2001, which, by the way, is not really an example of montage, but arguably the most famous example of a match cut in cinema history — 2 million years of technological advancement summed up in an instant — although not nearly as well matched (in angle and screen location) as it might have been.
True. For the record, film & media studies major here. I debated throwing in more specific jargon. I won't gloss over such things again.
So why don't we say that the base-3 exponent of 34 is 4. Instead we say the base-3 logarithm of 34 is 4. But the logarithmic is considered the inverse of the exponential, not a synonym for it! And my head explodes. :exploding_head: :boom:
I think "of" is functioning differently in these two cases. In the first case it dereferences an element of the thing as it is. In the second case it to transforms the thing into something else. Maybe?

This is interesting stuff, but we've been doing such a good job so far staying on topic ever since we dedicated this thread to the task at hand. Perhaps we should relegate the film theory and math lingo talk elsewhere.

User avatar
cmloegcmluin
Site Admin
Posts: 783
Joined: Tue Feb 11, 2020 3:10 pm
Location: San Francisco, California, USA
Real Name: Douglas Blumeyer
Contact:

Re: developing a notational comma popularity metric

Post by cmloegcmluin »

New all-time low SoS. This is a direct result of implementing an automatic recursive unpopularity metric finder. Well, you have to give it a starting configuration, still — my next goal is to implement a layer that will start with 1 parameter, and find the lowest SoS possible with a single parameter, then look at every combination of 2 parameters, and find the lowest SoS possible with any of those, etc.

I asked it to find something with 0<k<1, 1.5<a<2.5, 0<y<2, -3<w<-1, 0<c<1. A pretty wide range, and it pretty quickly turned up this:

SoS: 0.004059522
k = 0.1796875
a = 2.0234375 (base)
y = 0.4921875
w = -1.986328125
c = 0.5615234375

Pretty darn close to the one I found "manually" a week ago.

I had to kill that run in order to make an improvement before I let it run overnight. It's designed to find the best answers as soon as possible, so it's not highly likely that it will find anything better, but it might. And if it does, then it will have accomplished exactly what I built it for, because I was even less likely to find such results doing it manually!

User avatar
Dave Keenan
Site Admin
Posts: 1084
Joined: Tue Sep 01, 2015 2:59 pm
Location: Brisbane, Queensland, Australia
Contact:

Re: developing a notational comma popularity metric

Post by Dave Keenan »

Good work. I confirm that SoS. And I remind you that, with this scenario, the log base is not constrained by the data. You can lock it to whatever you want and you will find the same minimum. You only have to scale w and c by lognew_base(old_base). e.g.

SoS: 0.004059522
k = 0.1796875
a = 2 (base)
y = 0.4921875
w = -2.0197149
c = 0.570961685

or

SoS: 0.004059522
k = 0.1796875
a = e (base)
y = 0.4921875
w = -1.399959688
c = 0.395760482

So this is a 4-parameter metric.

User avatar
Dave Keenan
Site Admin
Posts: 1084
Joined: Tue Sep 01, 2015 2:59 pm
Location: Brisbane, Queensland, Australia
Contact:

Re: developing a notational comma popularity metric

Post by Dave Keenan »

When you look at the curves I got for the ideal weights for each prime when y = v = 1, you can see why SoPF>3 (now called sopfr) worked so well to approximate the Scala popularity rank.



I remind you that SoPF>3 = sopfr is equivalent to a soapfar with ap = p → p (i.e. linear, not logarithmic) and ar = r → r1 (i.e. y = 1). And it makes no difference to the ranking if you multiply or divide a metric by a constant, e.g. if you changed it to ap = p → p/6.

On the above graph, p →p/6 would be a straight line that starts at the origin (0, 0) and passes through the point (15, 2.5). You can see that this gives a good approximation of the ideal weights for primes up to 17 in the numerator and up to 13 in the denominator. However, we should expect it to perform very poorly for higher primes. And I remind you that this is only for y = 1. The ideal weight curve for the denominator looks very different when it is allowed to choose its own compression exponent.

That suggests another question. What scale factor do we have to apply to your winning metric above (in its log2 version), to make it give almost the same values as sopfr, for the first few ratios? (Looks like we have to multiply it by about 5.7) And can we then describe it as a correction-function applied to sopfr?

Post Reply