https://clickhouse.com/docs/zh/getting-started/example-datasets
1.英国房地产支付价格
CREATE TABLE uk_price_paid
(
price UInt32,
date Date,
postcode1 LowCardinality(String),
postcode2 LowCardinality(String),
type Enum8('terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4, 'other' = 0),
is_new UInt8,
duration Enum8('freehold' = 1, 'leasehold' = 2, 'unknown' = 0),
addr1 String,
addr2 String,
street LowCardinality(String),
locality LowCardinality(String),
town LowCardinality(String),
district LowCardinality(String),
county LowCardinality(String)
)
ENGINE = MergeTree
ORDER BY (postcode1, postcode2, addr1, addr2);
---预处理和插入数据
我们将使用 url 函数将数据流式传输到 ClickHouse。我们需要首先预处理一些传入的数据,其中包括:
将postcode 拆分为两个不同的列 - postcode1 和 postcode2,因为这更适合存储和查询
将time 字段转换为日期因为它只包含 00:00 时间
忽略 UUid 字段,因为我们不需要它进行分析
使用 transform 函数将 Enum 字段 type 和 duration 转换为更易读的 Enum 字段
将 is_new 字段从单字符串( Y/N) 到 [UInt8](/docs/zh/sql-reference/data-types/int-uint#uint8-uint16-uint32-uint64-uint256-int8-int16-int32-int64 -int128-int256) 字段为 0 或 1
删除最后两列,因为它们都具有相同的值(即 0)
url 函数将来自网络服务器的数据流式传输到 ClickHouse 表中。以下命令将 500 万行插入到 uk_price_paid 表中
---
INSERT INTO uk_price_paid
WITH
splitByChar(' ', postcode) AS p
SELECT
toUInt32(price_string) AS price,
parseDateTimeBestEffortUS(time) AS date,
p[1] AS postcode1,
p[2] AS postcode2,
transform(a, ['T', 'S', 'D', 'F', 'O'], ['terraced', 'semi-detached', 'detached', 'flat', 'other']) AS type,
b = 'Y' AS is_new,
transform(c, ['F', 'L', 'U'], ['freehold', 'leasehold', 'unknown']) AS duration,
addr1,
addr2,
street,
locality,
town,
district,
county
FROM url(
'http://prod.publicdata.landregistry.gov.uk.s3-website-eu-west-1.amazonaws.com/pp-complete.csv',
'CSV',
'uuid_string String,
price_string String,
time String,
postcode String,
a String,
b String,
c String,
addr1 String,
addr2 String,
street String,
locality String,
town String,
district String,
county String,
d String,
e String'
) SETTINGS max_http_get_redirects=10;
---验证数据
-让我们通过查看插入了多少行来验证它是否有效:
SELECT count()
FROM uk_price_paid
-在执行此查询时,数据集有 27,450,499 行。让我们看看 ClickHouse 中表的大小是多少:
SELECT formatReadableSize(total_bytes)
FROM system.tables
WHERE name = 'uk_price_paid'
---运行一些查询来分析数据:
-1.每年平均价格
SELECT
toYear(date) AS year,
round(avg(price)) AS price,
bar(price, 0, 1000000, 80
)
FROM uk_price_paid
GROUP BY year
ORDER BY year
-2.伦敦每年的平均价格
SELECT
toYear(date) AS year,
round(avg(price)) AS price,
bar(price, 0, 2000000, 100
)
FROM uk_price_paid
WHERE town = 'LONDON'
GROUP BY year
ORDER BY year
-3.最昂贵的社区
SELECT
town,
district,
count() AS c,
round(avg(price)) AS price,
bar(price, 0, 5000000, 100)
FROM uk_price_paid
WHERE date >= '2020-01-01'
GROUP BY
town,
district
HAVING c >= 100
ORDER BY price DESC
LIMIT 100
---使用 Projection 加速查询
Projections 允许我们通过存储任意格式的预先聚合的数据来提高查询速度。
在此示例中,我们创建了一个按年份、地区和城镇分组的房产的平均价格、总价格和数量的 Projection。
在执行时,如果 ClickHouse 认为 Projection 可以提高查询的性能,它将使用 Projection(何时使用由 ClickHouse 决定)
---构建投影
让我们通过维度 toYear(date)、district 和 town 创建一个聚合 Projectio
ALTER TABLE uk_price_paid
ADD PROJECTION projection_by_year_district_town
(
SELECT
toYear(date),
district,
town,
avg(price),
sum(price),
count()
GROUP BY
toYear(date),
district,
town
)
---填充现有数据的 Projection。(如果不进行 materialize 操作,则 ClickHouse 只会为新插入的数据创建 Projection):
ALTER TABLE uk_price_paid
MATERIALIZE PROJECTION projection_by_year_district_town
SETTINGS mutations_sync = 1
---让我们再次运行相同的 3 个查询, 同样,结果是相同的,但请注意查询性能的改进2.蜂窝信号塔
CREATE TABLE cell_towers
(
radio Enum8('' = 0, 'CDMA' = 1, 'GSM' = 2, 'LTE' = 3, 'NR' = 4, 'UMTS' = 5),
mcc UInt16,
net UInt16,
area UInt16,
cell UInt64,
unit Int16,
lon Float64,
lat Float64,
range UInt32,
samples UInt32,
changeable UInt8,
created DateTime,
updated DateTime,
averageSignal UInt8
)
ENGINE = MergeTree ORDER BY (radio, mcc, net, created);
---插入数据集:
clickhouse-client --query "INSERT INTO cell_towers FORMAT CSVWithNames" < cell_towers.csv
-按类型划分的基站数量:
SELECT radio, count() AS c FROM cell_towers GROUP BY radio ORDER BY c DESC
-各个移动国家代码(MCC)对应的蜂窝信号塔数量:
SELECT mcc, count() FROM cell_towers GROUP BY mcc ORDER BY count() DESC LIMIT 10
-创建一个 External Dictionary 来解码这些值
# 合并地理数据
CREATE TEMPORARY TABLE
moscow (polygon Array(Tuple(Float64, Float64)));
-以下点大致上构造了莫斯科的地理围栏(除“新莫斯科”外):
INSERT INTO moscow VALUES ([(37.84172564285271, 55.78000432402266),
(37.8381207618713, 55.775874525970494), (37.83979446823122, 55.775626746008065),...
-检查莫斯科有多少个蜂窝信号塔:
SELECT count() FROM cell_towers
WHERE pointInPolygon((lon, lat), (SELECT * FROM moscow))3.GitHub 事件数据集
https://abc.htmltoo.com/thread-46645.htm
4.Star Schema Benchmark
5.布朗大学基准
6.纽约公共图书馆“菜单上有什么?”
7.众包空中交通数据
8.食谱数据集
9.WikiStat
https://clickhouse.com/docs/en/getting-started/example-datasets/wikistat
---Creating a table:
CREATE TABLE wikistat
(
time DateTime CODEC(Delta, ZSTD(3)),
project LowCardinality(String),
subproject LowCardinality(String),
path String CODEC(ZSTD(3)),
hits UInt64 CODEC(ZSTD(3))
)
ENGINE = MergeTree
ORDER BY (path, time);
---Loading the data:
clickhouse-local --query "
WITH replaceRegexpOne(_path, '^.+pageviews-(\\d{4})(\\d{2})(\\d{2})-(\\d{2})(\\d{2})(\\d{2}).gz$', '\1-\2-\3 \4-\5-\6')::DateTime AS time,
extractGroups(line, '^([^ \\.]+)(\\.[^ ]+)? +([^ ]+) +(\\d+) +(\\d+)$') AS values
SELECT
time,
values[1] AS project,
values[2] AS subproject,
values[3] AS path,
(values[4])::UInt64 AS hits,
(values[5])::UInt64 AS size
FROM file('pageviews*.gz', LineAsString)
WHERE length(values) = 5 FORMAT Native
" | clickhouse-client --query "INSERT INTO wikistat FORMAT Native"10.Terabyte of Click Logs from Criteo
11.Anonymized Yandex.Metrica Data