Methodology: how we score stock calls
How we score stock calls
Last updated: 10 November 2026
Every number on this site comes from the rules on this page. They are the same for everyone, and we apply them automatically.
The short version
- We find calls in public posts on X: a commentator's own, forward-looking position on a specific stock, ETF or index.
- Each call starts at the next market open after the post.
- We check it after 1 week, 1 month and 3 months, and compare it with the market. The site shows 1 month unless you choose another window.
- Each call gets a call score from −100 to +100. A gain counts for more on a calm stock than on a wild one.
- Each commentator gets an accuracy score from 0 to 100. 50 is a coin flip, and commentators with few calls are pulled towards 50.
- To appear on a leaderboard, a commentator needs at least 10 settled calls in that window and a call in the last 90 days.
Who we track
- Public X accounts with at least 1,000 followers. We never track protected accounts.
- We find them by searching X for stock symbols and stock talk in many languages, and through the accounts that tracked commentators follow.
- Before we track an account, we read 40 of its recent posts. We track it if at least 20% of them are about stocks.
- We collect posts from 1 January 2026 onwards: the commentator's own posts and quote posts. We don't score replies or reposts.
- We only score stocks, ETFs and stock indexes. Crypto, forex and futures aren't scored.
What counts as a call
A post contains a call only if all four of these are true:
- It is the commentator's own view.
- It looks forward, not back.
- It is about a specific stock, ETF or index that we can identify.
- It takes a direction, either plainly or by strong implication: "I'm buying", "all in", "going to 300", "stay away", "I sold everything".
These are not calls:
- news and headlines ("BREAKING: Nvidia under investigation");
- reports of someone else's trades or options activity;
- questions and polls;
- jokes and sarcasm;
- market commentary without a position;
- boasting about past gains ("NVDA up 5% today");
- ads and promotions;
- a chart with no opinion.
A post that names several stocks gives one call for each. "Adding to $AAPL and $MSFT" is two bullish calls.
An AI model reads each post and records, for every call: the stock, the type of call, any price levels and time frame, how strongly it is worded, a quote of at most 200 characters, how confident it is, and a one-line reason. If its confidence is below 0.6, the call isn't scored and doesn't appear on the site. If it can't tell which stock is meant ("that small AI stock"), there is no call.
Types of call
- Bullish: buying, holding, expecting a rise. Right if the stock beats the market.
- Bearish: expecting a fall, or shorting. Right if the stock trails the market.
- Don't buy: "stay away", "overvalued". Scored like a bearish call: if the stock then beats the market, the call was wrong.
- Sell: "I sold", "took profits". Also scored like a bearish call: a good sale is followed by the stock trailing the market.
- Buy at a price: "buying if it drops to 150". See "Buy-at-a-price calls" below.
- Price target: "going to 250", sometimes with a stop-loss. See "Price targets" below.
- Options: a call option ("$NVDA 200c 12/19") counts as bullish, with the strike as the target and the expiry as the time frame. A put option is the mirror image.
How strongly it is worded
- Strong ("must buy", "all in", "load up"): weight 1
- Normal ("I like it", "buying"): weight 1
- Hedged ("could go up", "maybe a small position", "waiting for a dip"): weight 0.5
Hedged calls count half as much in a commentator's statistics. We also show a separate hit rate for strong calls only, to answer the question "are the loudest calls the most accurate?"
Prices
The entry price is the next open
A call's entry price is the first opening price in that market after the post, whether it was posted during trading, after hours or on a weekend.
This is the realistic price for someone who reads the post and then acts on it, and it means a commentator gets no credit for moves that happened before the post. For example, a post on a Friday at 8 pm New York time enters at the next trading day's open.
Windows
- 1 week = 5 trading days
- 1 month = 21 trading days
- 3 months = 63 trading days
Each window starts on the entry day and ends at the closing price on its last day. Each window is scored separately, so a call can be right after 1 week and wrong after 3 months. The site shows 1 month unless you choose another window. We also show the result after 1 day, for reference only. It never counts towards any score.
Benchmarks
We compare each call with the market it trades in:
- United States: SPY
- Japan: 1306.T (a TOPIX ETF)
- Hong Kong: 2800.HK
- Mainland China: 510300.SS
- South Korea: 069500.KS
- Taiwan: 0050.TW
- Other markets: the main local index
The benchmark is measured over exactly the same dates as the call.
Adjusted prices
All prices are adjusted for stock splits and dividends, and daily highs and lows are adjusted the same way. A 1-for-10 split doesn't show up as a 90% fall.
Scoring a call
Return and excess return
direction = +1 for bullish calls, −1 for bearish, don't-buy and sell calls
return = direction × (exit price ÷ entry price − 1)
excess return = direction × (stock's return − benchmark's return)
Excess return is what decides a call. A bullish call on a stock that rose 2% while the market rose 5% has an excess return of −3%.
Volatility
σ = (standard deviation of the stock's daily returns over the 60 trading days before entry)
× √(number of trading days in the window)
σ measures how much the stock usually moves over a window of that length. If a stock has fewer than 20 days of price history, we use the typical (median) value for its market.
Hit, miss or flat
threshold θ = the larger of 0.5% and 0.25 × σ
hit: excess return above +θ
miss: excess return below −θ
flat: anything in between
Volatile stocks get a wider flat zone, so ordinary day-to-day noise doesn't count as being right or wrong.
Examples (1-month window):
- Bullish call; the stock rose 6% and SPY rose 2%; θ is 1%. The excess return is +4%: hit.
- Bullish call; the stock rose 2% and SPY rose 5%. The excess return is −3%: miss. The page also shows the absolute return of +2%.
- Don't-buy call; the stock rose 8% and SPY rose 1%. The excess return is −7%: miss.
We always show the absolute return as well, and we point out cases like "rose, but trailed the market", where the call made money in absolute terms but missed against the benchmark.
Call score
z = excess return ÷ σ
call score = 100 × tanh(z) (always between −100 and +100)
Dividing by σ makes calls on calm and volatile stocks comparable. Suppose two bullish calls both beat the market by 5% in a month:
- on a utility stock with σ of 3%, z is about 1.7 and the call score is about +93 (a hit);
- on a volatile stock with σ of 25%, z is 0.2 and the call score is about +20, and the call counts as flat, because 5% is within that stock's normal noise.
The tanh function bends large values back towards ±100, so one call that doubled can't outweigh everything else a commentator has said.
Buy-at-a-price calls
"Buying if it drops to 150" only counts if the price actually gets there.
- Trigger period: the time frame in the post, or 30 trading days if it doesn't give one. It starts on the first trading day after the post.
- Triggered if the day's low reaches the price during the trigger period. For "shorting if it gets to X", the day's high has to reach X.
- Not triggered: the call is marked "not triggered". It doesn't count towards the hit rate, but it counts towards the commentator's trigger rate. We also show how much the price moved over the trigger period anyway (the move the commentator missed).
- If triggered, the entry price is the stated price, or the opening price if the stock opens beyond it. The entry day is the day it triggered. From there, the call is scored like any other.
Examples: if the lowest price in the trigger period is 151, the call is not triggered. If the stock one day opens at 148, the call enters at 148 on that day.
Price targets
progress = direction × (price − entry price) ÷ (target − entry price)
For bullish targets we use each day's high, and for bearish targets each day's low.
- Time frame: the one in the post, or 252 trading days (about 12 months) if it doesn't give one.
- Target reached (progress hits 100%) within the time frame: hit, with a call score of +100. We show how many trading days it took.
- Time frame ends without reaching the target: the call score is 100 × the progress at the final close, limited to between −100 and +99. Halfway there scores +50; moving the wrong way scores below zero.
- Still running: marked "in progress", with the progress so far. It isn't scored yet.
- With a stop-loss: if the stop is hit first, it's a miss with a call score of −100. If the target and the stop are both touched on the same day, we count it as a miss, to be conservative.
- Target on the wrong side of the entry price (for example, a bullish target below the entry): the target isn't scored.
Examples, with an entry at 200 and a target of 250: if the day's high reaches 251 on trading day 11, the target is hit (+100, day 11). If the time frame ends with the price at 225, the call score is +50.
A call with a target is still scored over the 1-week, 1-month and 3-month windows like any other call. The target result is shown as an extra line, and summed up in the commentator's target hit rate.
Repeats and reversals
- Repeats: if a commentator makes the same call (same stock, same direction) again within 30 days of the first one, only the first counts. We show "repeated N times". Saying something many times doesn't change a score.
- Reversals: if a commentator takes the opposite direction on the same stock within 30 days, the earlier call closes at the closing price on the day of the reversal. Windows that hadn't finished are scored over the days that had passed. The reversal then starts a new call. Profiles show how often a commentator reversed.
Scoring a commentator
Accuracy score
accuracy score = 50 + 0.5 × Σ(w × s) ÷ (Σw + 8)
s = each call's call score
w = each call's weight (1, or 0.5 for hedged calls)
- It uses only settled calls in that window: hits, misses and flats. Calls still in progress, calls that weren't triggered, and calls we couldn't settle don't count.
- It runs from 0 to 100. 50 means no better than a coin flip.
- The 8 in the formula pulls small samples towards 50. Three calls that each scored +70 give 59.5; thirty such calls give 77.6. A lucky streak of a few calls can't put someone at the top.
- Each window has its own accuracy score.
Grades
- 65 and above: Very accurate
- 55 to 65: Leaning accurate
- 45 to 55: Coin flip
- 35 to 45: Leaning inverse
- Below 35: Inverse
Other statistics
- Hit rate: hits ÷ (hits + misses), with hedged calls counting half. Flat calls are left out. We show it with a 95% range (a Wilson interval), which is wide when there are few calls.
- Average excess return: the weighted average of each call's excess return. Roughly, how much more (or less) than the market each call made.
- Follow and inverse curves: what would have happened on paper if someone had taken every call in date order, in equal amounts, holding each for one window. The inverse curve does the opposite of every call. These are hypothetical and ignore costs and taxes.
- Trigger rate: the share of buy-at-a-price calls that triggered.
- Target hit rate: the share of price targets whose time frame has ended that were reached, with the average number of days it took.
- Strong-call hit rate: the hit rate of strongly worded calls only.
- Counts: calls, reversals and repeats.
Leaderboards
- Who qualifies: at least 10 settled calls in that window, and at least one new call in the last 90 days. Commentators who don't qualify still have a profile, marked "not enough data".
- Overall leaderboards rank qualifying commentators by accuracy score: the most accurate from the top, the inverse list from the bottom.
- Weekly leaderboard: calls whose 1-week window ended this week, at least 3 per commentator.
- Monthly leaderboard: calls whose 1-month window ended this month, at least 5 per commentator.
A call belongs to the period in which its window ends. A call whose 1-month window ends on 14 November counts towards November.
Week by week and month by month
- Cohort table (on each profile): one row for each week or month in which calls were made, from January 2026, with the hits, the number of calls and the average call score for each window. It shows whether someone has been consistent or only right for a while.
- Accuracy trend: every Sunday, we calculate the 1-month accuracy score from calls made in the previous 90 days that have settled, and plot it as a line.
Updates
Every day after the markets close, we recalculate everything: new prices come in, windows that have ended are settled, new calls are added and the leaderboards refresh. Once a post has been judged, the AI doesn't judge it again. Only an appeal changes a judgement, and then all the scores are recalculated.
Special cases
- Trading halted, delisted or symbol changed: the call is marked "unable to settle", with the reason, and doesn't count towards anything.
- Splits: we use adjusted prices, and the call's page notes the adjustment.
- Posts on weekends, holidays or during trading: the entry is always the next open, and the call's page shows the entry date.
- Time zones: calls are settled on the trading dates of the stock's own exchange. Times on the site are shown in your local time zone.
- Renamed accounts: we follow accounts by their X user ID, so the record carries over to the new handle. Links that use the old handle may stop working.
- Deleted posts: the call's page says "original post deleted" and no longer shows the quote. The call is still settled and still counts.
- Deleted or protected accounts: the profile stays, marked "account no longer visible", and stops updating.
How we check the AI
- Anyone can appeal a call. A person reviews every appeal, and corrections are recorded on the call's page.
- Every week we check a random sample of 50 calls by hand, to measure how often the AI gets it right.
Known limitations
- The AI can misread posts. Sarcasm, jokes, context from earlier posts and unusual stock names are the hardest cases.
- Some calls are missed. We only give the AI posts that look like they are about stocks (by stock symbols and common trading words). Calls made only in an image, a video or a linked article, or without any of those words, may be missed.
- Deleted posts. If a post is deleted before we collect it, we never see it. A commentator who quickly deletes bad calls can look better than they are.
- Edited posts. We judge the version of a post we collected.
- We only see what people post, not what they actually trade.
- Fixed windows. A call meant for the next five years is still measured over 1 week, 1 month and 3 months. Price targets get their own time frame.
- Daily prices only. We use each day's open, high, low and close, not every trade during the day. That's why a target and a stop touched on the same day count as a miss.
- Broad benchmarks. We compare with the whole market, not with the stock's industry.
- No costs. Results ignore trading costs, taxes and slippage.
- Options are simplified. We treat an options trade as a call on the stock with the strike as the target; we don't price the option itself.
- Coverage. We don't track every commentator, every market or replies. Leaderboards only compare the commentators we track.
Changes to these rules
When we change a rule, we update this page, note the date and what changed, and recalculate every record under the new rules, so everyone is always measured the same way.
Questions and corrections
If you think a call was recorded wrongly, appeal it. For questions about these rules, email support@callcheck.akifukaku.com. Nothing on this site is investment advice. See the Disclaimer.