Not Less Than 23% Wrong in This AI Evaluation

[marcus]

From https://finance.yahoo.com/technology/ai/articles/test-ai-mortgage-assistants-got-222127345.html
Title : In a New Test, AI Mortgage Assistants Got Nearly 1 in 4 Answers Wrong
By Allaire Conte , August 2026

... Matthew Toles, a Columbia University doctoral student and co-author of the new study, "MortarBench: Evaluating Mortgage Loan Origination Agents,"...

On the benchmark's strictest measure—whether the complete answer matched the known correct one—Gemini 3.1 Pro was correct 77.1% of the time, GPT-5.5 76.8%, and Claude Sonnet 4.6 51.4% ...

When researchers asked 3 of the leading general-use AI models which deposits in a bank statement "could be of foreign origin," transactions connected to English names were flagged 13.3% of the time. But when the deposit came from a non-English name, that number jumped to 77% ...

Even as concerns persist about AI's ability to provide accurate and unbiased information, mortgage lenders are moving to adopt it. More than 80% were evaluating the technology as of June, according to a survey by The Mortgage Collaborative, while 17% had deployed it in live production workflows ...

 

[snpark]

2+2(2+2)

 

[pantaiema]

From https://finance.yahoo.com/technology/ai/articles/test-ai-mortgage-assistants-got-222127345.html
Title : In a New Test, AI Mortgage Assistants Got Nearly 1 in 4 Answers Wrong
By Allaire Conte , August 2026

... Matthew Toles, a Columbia University doctoral student and co-author of the new study, "MortarBench: Evaluating Mortgage Loan Origination Agents,"...

On the benchmark's strictest measure—whether the complete answer matched the known correct one—Gemini 3.1 Pro was correct 77.1% of the time, GPT-5.5 76.8%, and Claude Sonnet 4.6 51.4% ...

When researchers asked 3 of the leading general-use AI models which deposits in a bank statement "could be of foreign origin," transactions connected to English names were flagged 13.3% of the time. But when the deposit came from a non-English name, that number jumped to 77% ...

Even as concerns persist about AI's ability to provide accurate and unbiased information, mortgage lenders are moving to adopt it. More than 80% were evaluating the technology as of June, according to a survey by The Mortgage Collaborative, while 17% had deployed it in live production workflows ...

Anyone can see real world proof that sensible people don't rely on AI to make big calls:

Bank Support: Try fixing a mistaken transaction or looking into your account with an AI bot; what normally happen is that it usually hits a wall, forcing you to wait for a human agent to fix the mistake.

AI trading Bot. If there were an AI trader with a reliable 60% success rate, every tech-savvy trader would be an overnight millionaire. The reality on Wall Street and in crypto says otherwise. A minimum of 60% definitive win rate (e.g confidence level of 100%) is all you need to become an overnight millionaire from trading. Keep in mind in trading people could use a very high leverage.

Most bankers, Hedge funds (if not all) use AI to help them. But noone will solely rely on the decision made by the AI. Also they certainly have fundamental knowledge on the subject as well as how and when to use the tools.

 

[jstar]

…sensible people… AI

Remember the recent scandals with the trading robots (automated forex and crypto sw) that operated in Indonesia? Even a lot of ‘sensible’ people in my family in law that -unlike back then- don’t want to talk about it anymore. It’s amazing what they lost to these fake tech and Ponzi schemes.

 

[pantaiema]

Remember the recent scandals with the trading robots (automated forex and crypto sw) that operated in Indonesia? Even a lot of ‘sensible’ people in my family in law that -unlike back then- don’t want to talk about it anymore. It’s amazing what they lost to these fake tech and Ponzi schemes.

Suggest to your family in law members on your spouse’s side that they look into chair Tai-Chi or take yoga class as they might need that to improve their health, fitness and appearance.

Once they've achieved better health and peace of mind, have them take a look at another Gen-Z influencers, popular among Indonesian youth who sells unrealistic dreams and misleads young people.

https://www.youtube.com/@TimothyRonald/videos

A lot of his content is essentially plagiarized from Andrew Tate

You can see in another thread about his victim in here :
https://www.expatindo.org/community/threads/how-do-you-guys-invest.4963/page-40#post-117439

Previously, other Gen-Z figures like Indra Kenz, Doni Salmanan, and Wahyu Kenzo pulled similar stunts, and all of them ended up in prison for fraud running a ponzie scheme. However, the Timothy Ronald situation appears slightly different: instead of directly pooling or managing client funds, he generates revenue by charging a paid Discord membership fee (reportedly around 500k members) to provide buy/hold/sell trading signals. So his wealth and lifestyle he is flaunting stems from subscription fees rather than genuine financial or trading skills.

Anyone with a basic grasp of trading, finance, and economics can see that most of his material is nonsense, no value at all. The few parts that hold any value are extremely basic and readily available online for free.

 

[pantaiema]

This witch/sorceress lady with her crystal ball might produce a better result than some people who are using AI without a good grasp of fundamental knowledge on the subject as well as sense to make a good judgement.