Ask the data in plain English

Headline numbers quoted in the team report, each with its source section.

• key text · identifier, e.g. tweets_processed, toots_harvested, vic_sa2_income_kept
• value real · the number
• unit text · unit of value, e.g. tweets, GB, SA2s
• source text · where the number comes from

Editions of the boundaries and how the database was built.

• key text · e.g. twitter_period, sa2_edition
• value text · value

The paragraphs the 2023 dashboard showed beside each chart (written by the team).

• source text · 'twitter', 'sudo' or 'mastodon'
• dataset text · dataset the paragraph describes
• paragraph integer · paragraph number
• text text · paragraph text

Counts of 1-9 sentiment scores for Twitter and the three Mastodon servers, as plotted in 2023. Twitter covers geotagged tweets Australia-wide, Feb-Jul 2022.

• source text · 'twitter', 'mastodon.social', 'mastodon.au' or 'tictoc.social'
• topic text · 'all', 'income' or 'crime' (crime only for twitter)
• score integer · sentiment score 1 (extremely negative) to 9 (extremely positive); 5 is neutral
• count integer · number of posts with this score

CouchDB MapReduce _stats per suburb and topic (all tweets, income keywords, crime keywords), Feb-Jul 2022, every Australian state. Rows are keyed by SAL code: join regions_sal for names.

• sal_code text · ABS 2021 suburb/locality code (join regions_sal.sal_code)
• topic text · 'all', 'income' or 'crime'
• state text · state name, e.g. 'Victoria', 'New South Wales'
• tweet_count integer · tweets in this suburb and topic
• score_sum integer · sum of the tweets' 1-9 scores
• score_min integer · lowest score
• score_max integer · highest score
• score_sumsqr integer · sum of squared scores
• avg_score real · score_sum / tweet_count rounded to 2 decimals (the 2023 dashboard's value)

The three servers the harvesters followed.

• server text · 'mastodon.social', 'mastodon.au' or 'tictoc.social'
• label text · display name
• url text · server URL
• description text · short description

The surviving week of mastodon.social toots (1-9 May 2023) re-scored with the original pipeline (aggregates only).

• server text · always 'mastodon.social'
• topic text · 'all', 'income' or 'crime'
• score integer · sentiment score 1-9
• count integer · toots with this score

Re-scored mastodon.social toots per UTC hour, 1-9 May 2023, with score sums and 1-9 bucket counts.

• server text · always 'mastodon.social'
• hour_utc text · hour start in UTC, ISO format e.g. '2023-05-01T13:00:00Z'
• toots integer · toots in the hour
• score_sum integer · sum of 1-9 scores (average = score_sum / toots)
• income_toots integer · toots matching the income keywords
• income_score_sum integer · sum of scores of income toots
• crime_toots integer · toots matching the crime keywords
• b1 integer · toots scored 1 on the 1-9 scale
• b2 integer · toots scored 2 on the 1-9 scale
• b3 integer · toots scored 3 on the 1-9 scale
• b4 integer · toots scored 4 on the 1-9 scale
• b5 integer · toots scored 5 on the 1-9 scale
• b6 integer · toots scored 6 on the 1-9 scale
• b7 integer · toots scored 7 on the 1-9 scale
• b8 integer · toots scored 8 on the 1-9 scale
• b9 integer · toots scored 9 on the 1-9 scale

Re-scored mastodon.social toots (1-9 May 2023) per declared language, with score sums and bucket counts.

• server text · always 'mastodon.social'
• lang text · ISO 639-1 language code declared by the toot, e.g. 'en', 'de', 'ja'; 'und' = undetermined
• toots integer · toots in this language
• score_sum integer · sum of 1-9 scores
• income_toots integer · toots matching the income keywords
• b1 integer · toots scored 1 on the 1-9 scale
• b2 integer · toots scored 2 on the 1-9 scale
• b3 integer · toots scored 3 on the 1-9 scale
• b4 integer · toots scored 4 on the 1-9 scale
• b5 integer · toots scored 5 on the 1-9 scale
• b6 integer · toots scored 6 on the 1-9 scale
• b7 integer · toots scored 7 on the 1-9 scale
• b8 integer · toots scored 8 on the 1-9 scale
• b9 integer · toots scored 9 on the 1-9 scale

ABS suburbs and localities that had tweets, every state, with a representative point and (Victoria only) the SA2 and LGA they fall in.

• sal_code text · ABS 2021 SAL code
• name text · suburb name, e.g. 'Melbourne', 'Geelong', 'Ballarat Central'
• state text · state name, e.g. 'Victoria'
• area_km2 real · area in square kilometres
• lat real · latitude of a representative point
• lon real · longitude of a representative point
• sa2_code16 text · 2016 SA2 containing the point (Victoria only; join regions_sa2.sa2_code)
• lga_code19 text · 2019 LGA containing the point (Victoria only; join regions_lga.lga_code)
• on_original_map integer · 1 if drawn on the 2023 Victorian Twitter map

ABS Statistical Area Level 2 regions in Victoria.

• sa2_code text · 2016 SA2 code
• name text · SA2 name
• sa3_name text · parent SA3
• sa4_name text · parent SA4
• gcc_code text · Greater Capital City code: '2GMEL' Greater Melbourne, '2RVIC' rest of Victoria
• area_km2 real · area in square kilometres
• lat real · latitude of a representative point
• lon real · longitude of a representative point

ABS Local Government Areas in Victoria.

• lga_code text · 2019 LGA code
• name text · LGA name with type suffix, e.g. 'Ballarat (C)', 'Alpine (S)'
• area_km2 real · area in square kilometres
• lat real · latitude of a representative point
• lon real · longitude of a representative point

SUDO / ABS personal income 2015-16 (mean, median, sum, median age of earners) for every Australian SA2.

• sa2_code text · 2016 SA2 code
• sa2_name text · SA2 name
• sa3_name text · parent SA3
• sa4_name text · parent SA4
• gcc_code text · Greater Capital City code
• gcc_name text · Greater Capital City name, e.g. 'Greater Melbourne'
• state text · state name
• mean_aud real · mean personal income, AUD
• median_aud real · median personal income, AUD
• sum_aud real · total personal income, AUD
• median_age real · median age of earners
• on_national_map integer · 1 if on the 2023 national income map
• vic_iqr_kept integer · Victoria only: 1 if kept by the team's IQR outlier rule (420 of 457), else 0; NULL outside Victoria

The SA2 rows grouped by Greater Capital City Statistical Area, as in the original summary.

• gcc_code text · e.g. '1GSYD', '2GMEL', '8ACTE', '2RVIC'
• gcc_name text · e.g. 'Greater Sydney', 'Rest of Vic.'
• mean_aud real · mean of the SA2 mean incomes, AUD
• median_aud real · median of the SA2 median incomes, AUD
• sum_aud real · total income, AUD
• median_age real · mean of the SA2 median ages
• sa2_count integer · number of SA2s

ABS Jobs in Australia 2018-19 indicators (mean, sd and median across SA2s) from the original bar chart.

• indicator text · machine name of the indicator
• label text · readable label, e.g. 'Health care social assistance - median income per job'
• kind text · 'income_aud' (median income per job, AUD) or 'jobs_000' (thousands of jobs)
• mean real · mean across SA2s
• std real · standard deviation across SA2s
• median real · median across SA2s

Victorian Crime Statistics Agency offence divisions per LGA (reference year 2019).

• lga_code text · LGA code (join regions_lga.lga_code)
• lga_name text · LGA name with suffix, e.g. 'Melbourne (C)', 'Greater Geelong (C)'
• against_person integer · offences against the person
• property_deception integer · property and deception offences
• drug integer · drug offences
• public_order integer · public order and security offences
• justice integer · justice procedures offences
• other integer · other offences
• reference_period integer · year, always 2019
• iqr_kept integer · 1 if kept by the team's IQR outlier rule (72 of 79)
• total integer · all recorded offences

Median income joined with suburb tweet sentiment pooled to each Victorian SA2 (revival analysis). avg_* are NULL where an SA2 had no tweets.

• sa2_code text · 2016 SA2 code
• sa2_name text · SA2 name
• median_aud real · median personal income, AUD
• mean_aud real · mean personal income, AUD
• vic_iqr_kept integer · 1 if kept by the team's IQR outlier rule
• tweets_all integer · geotagged tweets pooled from the SA2's suburbs
• avg_all real · average 1-9 score of those tweets
• sal_count integer · suburbs with tweets in the SA2
• tweets_income integer · income-related tweets
• avg_income real · average score of income-related tweets

Offence totals joined with suburb tweet sentiment pooled to each Victorian LGA (revival analysis).

• lga_code text · LGA code
• lga_name text · LGA name with suffix
• total integer · recorded offences, 2019
• iqr_kept integer · 1 if kept by the team's IQR outlier rule
• tweets_all integer · geotagged tweets pooled from the LGA's suburbs
• avg_all real · average 1-9 score of those tweets
• sal_count integer · suburbs with tweets in the LGA
• tweets_crime integer · crime-related tweets
• avg_crime real · average score of crime-related tweets

Pearson, Spearman and least-squares fits at several minimum-tweet thresholds (computed with scipy).

• scenario text · 'income' or 'crime'
• unit text · 'sa2', 'lga' or 'sal'
• x_metric text · 'median_aud', 'total' or 'log10_tweets'
• y_metric text · 'avg_income', 'avg_all', 'avg_crime' or 'avg_raw'
• weight_metric text · column used for the minimum-tweet threshold
• min_tweets integer · minimum tweets per region: 1, 5, 10 or 30
• n integer · regions in the fit
• pearson_r real · Pearson correlation
• pearson_p real · two-sided p-value of pearson_r
• spearman_rho real · Spearman rank correlation
• spearman_p real · two-sided p-value of spearman_rho
• slope real · least-squares slope
• intercept real · least-squares intercept
• r2 real · R squared