Can you explain some more. If it is used to justify the offer that would suggest someone else won't get the offer because their test score was lower but they could have performed better in the rest of the interview? You said it's only a small part of the interview, but forms the butt of the justification for offer which is a contradiction in my eyes.
Does it have to be language agnostic? If the role isn't language agnostic, surely the test should not be either? Otherwise you're criteria for selection are wrong.
What are empirical measures for team fit? You could actually ask HR that (I assume this is driven by company HR policy). The goal isn't to show how wrong their approach is, they may actually have an answer, there is a lot of research on this stuff.
If it's only going to be a small part of the interview, that means it will have to be short, which would suggest not many questions. There is no language agnostic empirical measure that I could even begin to describe that would be useful with a small question set.
There are a number of third party companies who provide technical tests online, tailored to specific roles. I do them occasionally, companies use them to filter down the candidates. If you adopted one of those, you have introduced a layer of impartiality, you have set a technical bar to be reached already and the test providers do all the hard work. There are two companies I've sat tests through; IKM and BrainBench. You do them at home, it takes an hour or so and gives a pretty fair indication of ability.