Mark Clowes (38M 🇬🇧)
Index - YouTube Super-Thanks Totalising
2023-12-31I watched this YouTube video recently. It is "Travel Vlogging cost me an Organ…☠️" by Timmy Karter. While abroad Timmy had to have emergency surgery to remove his gallbladder. This was not covered by his travel insurance (apparently; there's a comment that suggests his insurer perhaps thought this was an elective rather than life-saving operation). This left unfortunate Timmy with a bill of 17,000 somethings (USD? unclear).
People were very kindly sending Timmy "Super Thanks" comments to contribute towards this cost. I was curious; how much had Timmy received thus far? Obviously, Timmy will have this info on his YouTube dashboard. And YouTube knows. The public could "know" but the information is not presented accessibily, but it is accessible.
So I set out to find out. The yt-dlp project (a fork of youtube-dl) has some interesting features, including dumping all comments on a video to JSON. So that's where I started.
This command will dump all of the comments from the video to a JSON file:
yt-dlp --no-download --write-comments --dump-single-json "https://www.youtube.com/watch?v=bfckS7F_E1Y" > bfckS7F_E1Y.json
Unfortunately the JSON dump does not contain any info about if a comment is a "super-thanks". So let's patch it in! Luckily someone had already requested this feature on Github: https://github.com/yt-dlp/yt-dlp/issues/5411. The request was essentially rejected as out of scope but their was enough info there to point me in the correct direction for my own modification.
I git cloned the yt-dlp repo and altered yt-dlp/yt_dlp/extractor/youtube.py. We just need to add a couple of lines to _extract_comment:
def _extract_comment(self, comment_renderer, parent=None):
<... rest of function ...>
paid_comment = traverse_obj(comment_renderer, ('paidCommentChipRenderer', 'pdgCommentChipRenderer', 'chipText', 'simpleText'))
if paid_comment:
info['paid_comment'] = paid_comment
return info
Side note: invocation is slightly different for the non-zipapp'd version I modified:
python3 -m yt_dlp --no-download --write-comments --dump-single-json "https://www.youtube.com/watch?v=bfckS7F_E1Y" > bfckS7F_E1Y.json
Our JSON dump now contains the needed info.
{
"id": "UgzsKIyvQDbQnZGNP_N4AaABAg",
"text": "Thanks!",
"like_count": 1,
"author_id": "UCRwPc0dPQysGcMKjar5-glA",
"author": "@MyrtleBeachInAction",
"parent": "root",
"_time_text": "18 hours ago",
"timestamp": 1703977200,
"author_url": "https://www.youtube.com/channel/UCRwPc0dPQysGcMKjar5-glA",
"author_is_uploader": false,
"is_favorited": false,
"paid_comment": "$19.99"
}
We can use jq to extract a list of payments made in the video:
jq '.comments | .[] | .paid_comment // empty' ~/bfckS7F_E1Y.json > ~/paid-comments.txt
wc -l shows 403 donations as of today.
We have the data but not really any insight. The "super-thanks" shows the original type of currency. Some currencies are shown with the ISO 4217 currency code, and some are shown with a symbol (e.g. € instead of EUR). We need to clean the data to use all currency codes, and we need to total all of the donations in each currency, and lastly find the exchange rates for each so we can have a grand single total value (I chose USD).
I found a free (rate limited) API from exchangerate-api.com at https://open.er-api.com/v6/latest/USD and I wrote a python script:
import requests
import re
import sys
# Dictionary to store total amounts for each currency
totals = {}
USD_grand_total = 0
# Currency mapping dictionary
currency_mapping = {
'$': 'USD',
'CA$': 'CAD',
'£': 'GBP',
'€': 'EUR',
'₱': 'PHP',
'A$': 'AUD',
'₹': 'INR',
'₪': 'ILS',
'R$': 'BRL',
'NZ$': 'NZD',
'MX$': 'MXN',
}
# Function to extract currency and amount
def extract_currency_and_amount(value):
match = re.search(r'"(.+?)\s*([\d,\.]+)"', value)
if match:
currency = match.group(1)
amount = float(match.group(2).replace(',', ''))
return currency, amount
return None, None
# Fetch exchange rates from the API
def fetch_exchange_rates():
url = "https://open.er-api.com/v6/latest/USD"
response = requests.get(url)
if response.status_code == 200:
return response.json().get('rates', {})
return {}
# Calculate total USD value for all currencies
def calculate_total_usd():
global USD_grand_total
exchange_rates = fetch_exchange_rates()
if not exchange_rates:
print("Unable to fetch exchange rates.")
return
for currency, amount in totals.items():
mapped_currency = currency_mapping.get(currency, currency)
rate = exchange_rates.get(mapped_currency)
if rate is not None:
usd_value = amount / rate
print(f"Original Currency: {currency} {amount:.2f}, Exchange Rate: {rate}, USD Value: {usd_value:.2f}")
USD_grand_total += usd_value
else:
print(f"Unable to convert unknown currency {currency} with value {amount}")
# Check if the file path is provided as a command-line argument
if len(sys.argv) < 2:
print("Please provide the path to the file as a command-line argument.")
sys.exit(1)
file_path = sys.argv[1] # Get file path from command line argument
try:
# Read data from file
with open(file_path, 'r') as file:
data = file.readlines()
# Replace currency identifiers with mapped values and calculate totals
for item in data:
currency, amount = extract_currency_and_amount(item)
if currency and amount:
mapped_currency = currency_mapping.get(currency, currency)
if mapped_currency in totals:
totals[mapped_currency] += amount
else:
totals[mapped_currency] = amount
calculate_total_usd()
# Print total USD value for all currencies
print(f"\nGrand Total USD value for all currencies: {USD_grand_total:.2f}")
except FileNotFoundError:
print(f"File '{file_path}' not found.")
Let's go!
% python3 calculate_totals.py ~/paid-comments.txt Original Currency: USD 2353.39, Exchange Rate: 1, USD Value: 2353.39 Original Currency: GBP 482.81, Exchange Rate: 0.785482, USD Value: 614.67 Original Currency: CAD 514.76, Exchange Rate: 1.324315, USD Value: 388.70 Original Currency: SEK 2328.00, Exchange Rate: 10.068328, USD Value: 231.22 Original Currency: EUR 709.82, Exchange Rate: 0.905356, USD Value: 784.02 Original Currency: CHF 160.00, Exchange Rate: 0.840935, USD Value: 190.26 Original Currency: PHP 50.00, Exchange Rate: 55.400068, USD Value: 0.90 Original Currency: MKD 1250.00, Exchange Rate: 55.666284, USD Value: 22.46 Original Currency: AUD 189.97, Exchange Rate: 1.468171, USD Value: 129.39 Original Currency: INR 2898.00, Exchange Rate: 83.216825, USD Value: 34.82 Original Currency: CZK 1519.00, Exchange Rate: 22.363047, USD Value: 67.92 Original Currency: MYR 134.80, Exchange Rate: 4.594136, USD Value: 29.34 Original Currency: RON 144.99, Exchange Rate: 4.497283, USD Value: 32.24 Original Currency: ILS 6.00, Exchange Rate: 3.620425, USD Value: 1.66 Original Currency: NOK 1047.00, Exchange Rate: 10.169728, USD Value: 102.95 Original Currency: SGD 29.98, Exchange Rate: 1.31974, USD Value: 22.72 Original Currency: BRL 95.00, Exchange Rate: 4.853094, USD Value: 19.58 Original Currency: BGN 96.49, Exchange Rate: 1.770694, USD Value: 54.49 Original Currency: CRC 10000.00, Exchange Rate: 517.246406, USD Value: 19.33 Original Currency: DKK 165.00, Exchange Rate: 6.7521, USD Value: 24.44 Original Currency: TRY 64.99, Exchange Rate: 29.533355, USD Value: 2.20 Original Currency: RSD 250.00, Exchange Rate: 105.904948, USD Value: 2.36 Original Currency: HUF 15979.00, Exchange Rate: 346.75748, USD Value: 46.08 Original Currency: THB 500.00, Exchange Rate: 34.353528, USD Value: 14.55 Original Currency: IDR 20000.00, Exchange Rate: 15408.602137, USD Value: 1.30 Original Currency: PLN 129.99, Exchange Rate: 3.93099, USD Value: 33.07 Original Currency: NZD 49.99, Exchange Rate: 1.581952, USD Value: 31.60 Original Currency: QAR 100.00, Exchange Rate: 3.64, USD Value: 27.47 Original Currency: COP 90000.00, Exchange Rate: 3870.96011, USD Value: 23.25 Original Currency: MXN 250.00, Exchange Rate: 16.959626, USD Value: 14.74 Original Currency: CLP 5000.00, Exchange Rate: 884.730748, USD Value: 5.65 Original Currency: MAD 20.00, Exchange Rate: 9.852723, USD Value: 2.03 Grand Total USD value for all currencies: 5328.82
Not bad! Unfortunately Google takes a 30% cut out of "Super-Thanks". And if the payment is made via an Apple device, Apple take a 30% cut before Google! So in the situation of a $100 donation, Apple would take $30 leaving $70, then Google would take $21 leaving $49. The recipient would get less than half what was spent!
So better to use some other payment method if available instead of a SuperThanks.