I'm experimenting with Azure's Content Moderator, especially with the Classify parameter. There are 3 categories.
The categories are pertaining to the text content that may be undesirable.
Category 1 - content could be sexually explicit or adult related
Category 2 - language may be considered sexually suggestive or mature
in certain situations
Category 3 - potentially offensive language
The score is between 0 and 1. The higher the score, the higher the model is predicting that the category may be applicable.
So, being apolitical and simply curious, I used the Facebook posts from Donald J. Trump and the current Head of State Joe Biden, regarding the recent events in the Capitol.
However, I find the results very dubious, and can't seem to understand why one post with no negative words gets marked as highly (potentially) offensive, another - with more words with negative meaning - is not marked as such.
Donald J. Trump's:
Post-Trump-Content-Moderator
And Joe Biden's:
Post-Biden-Content-Moderator
I'd super appreciate it if someone could help me understand where this comes from. Maybe it's something very simple and obvious and I just don't get it.
question from:
https://stackoverflow.com/questions/65939147/getting-dubious-results-in-azure-content-moderator-explainable-ai 与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
