ã2026幎çãdbtã®å§ãæ¹ â ããŒã¿å€æã¯ãŒã¯ãããŒã®å ¥é
- ELTãã€ãã©ã€ã³ã«ãããdbtã®TïŒå€æïŒã®åœ¹å²ã説æã§ãã
- dbt Coreãã€ã³ã¹ããŒã«ããŠã¢ãã«ïŒSQLãã¡ã€ã«ïŒãäœæã»å®è¡ã§ãã
- sourcesã»ãã¹ãã»ããã¥ã¡ã³ãã®åºæ¬çãªèšå®ãã§ãã
- dbt CloudãšdbtCoreã®äœ¿ãåãã倿ã§ãã
ã¯ã€ãã¯ãªãã¡ã¬ã³ã¹
dbt Core ã®ã€ã³ã¹ããŒã«ãšåæå
# pip ã§ã€ã³ã¹ããŒã«ïŒBigQueryã¢ããã¿ãŒäŸïŒ
pip install dbt-bigquery
# ãã®ä»ã®ã¢ããã¿ãŒ
pip install dbt-snowflake
pip install dbt-redshift
pip install dbt-postgres
# ãããžã§ã¯ãåæå
dbt init my_project
cd my_project
# æ¥ç¶ãã¹ã
dbt debug
ãããžã§ã¯ãæ§é
my_project/
âââ dbt_project.yml # ãããžã§ã¯ãèšå®
âââ profiles.yml # DBæ¥ç¶èšå®ïŒ~/.dbt/profiles.yml ãæšæºïŒ
âââ models/
â âââ staging/ # çããŒã¿ã®æŽåœ¢å±€
â â âââ stg_orders.sql
â âââ marts/ # åæçšã®éèšå±€
â âââ fct_orders.sql
âââ tests/ # ã«ã¹ã¿ã ãã¹ã
âââ macros/ # åå©çšå¯èœãªJinja2颿°
ã¢ãã«ã®æžãæ¹
-- models/staging/stg_orders.sql
SELECT
order_id,
customer_id,
CAST(order_date AS DATE) AS order_date,
amount
FROM {{ source('raw', 'orders') }} -- ãœãŒã¹å®çŸ©ãåç
§
WHERE amount > 0
-- models/marts/fct_monthly_sales.sql
SELECT
DATE_TRUNC(order_date, MONTH) AS month,
COUNT(*) AS order_count,
SUM(amount) AS total_amount
FROM {{ ref('stg_orders') }} -- ä»ã¢ãã«ãåç
§
GROUP BY 1
schema.ymlïŒãã¹ãã»ããã¥ã¡ã³ãå®çŸ©ïŒ
version: 2
models:
- name: stg_orders
description: "åæ³šããŒãã«ã®æŽåœ¢æžã¿Stagingå±€"
columns:
- name: order_id
description: "åæ³šID"
tests:
- unique
- not_null
- name: amount
tests:
- not_null
äž»èŠã³ãã³ã
dbt run # ã¢ãã«ãå®è¡ïŒããŒãã«/ãã¥ãŒäœæïŒ
dbt test # ããŒã¿å質ãã¹ãå®è¡
dbt build # run + test ãäžæ¬å®è¡ïŒCIåãïŒ
dbt docs generate # ããã¥ã¡ã³ããµã€ãçæ
dbt docs serve # ããã¥ã¡ã³ããããŒã«ã«ã§ç¢ºèª
dbt run --select stg_orders # ç¹å®ã¢ãã«ã®ã¿å®è¡
ããããè©°ãŸããã€ã³ã
Q. dbt run ã§ã¢ãã«ãå®è¡ãããã空ã®ããŒãã«ãã§ãã
A. SQLã®WHEREå¥ã®æ¡ä»¶ã source() åç
§ã®ã¹ããŒãåã確èªããŠãã ãããdbt debug ã§æ¥ç¶èšå®ãæ£ããã確èªããdbt compile ã§å®éã«å®è¡ãããSQLã target/compiled/ ãã©ã«ãã§ç¢ºèªãããšãããã°ããããã§ãã
Q. {{ ref('model_name') }} ã§åç
§ããŠããã®ã«ãšã©ãŒã«ãªã
A. åç
§å
ã®ã¢ãã«ãã¡ã€ã«åïŒ.sql ã®æ¡åŒµåãé€ããéšåïŒãš ref() å
ã®æååãå®å
šäžèŽããŠããã確èªããŠãã ããããã©ã«ãéå±€ã¯é¢ä¿ãªãããããžã§ã¯ãå
ã§ãŠããŒã¯ãªã¢ãã«åã䜿ããŸããååã¢ãã«ãè€æ°ååšãããšãšã©ãŒã«ãªããŸãã
Q. dbt test ã§uniqueness testã倱æãã
A. å®éã«ããŒã¿ã«éè€ãããã±ãŒã¹ãå€ãã§ããSELECT ã«ã©ã , COUNT(*) FROM ããŒãã« GROUP BY 1 HAVING COUNT(*) > 1 ã§éè€ããŒã¿ã確èªããŠãã ãããæ³å®å€ã®éè€ãããã°ãœãŒã¹ããŒã¿ã®å質åé¡ãªã®ã§ãStaging局㧠QUALIFY ROW_NUMBER() OVER (PARTITION BY id ORDER BY updated_at DESC) = 1 ã®ããã«éè€é€å»ããåŠçã远å ããŸãããã
dbtã£ãŠäœã®ç¥ãªã®ïŒäœãããããŒã«ãªã®ïŒ
dbtã¯ãdata build toolãã®ç¥ã§ãããŒã¿ãŠã§ã¢ããŠã¹äžã®ããŒã¿ãSQLã§å€æããããã®ããŒã«ã ããELTïŒExtract â Load â TransformïŒãã€ãã©ã€ã³ã®ãTãã®éšåãæ åœãããã ãBigQueryãSnowflakeã«ããŒããããçããŒã¿ããåæããããããããªåœ¢ã«æŽããäœæ¥ãSQLãã¡ã€ã«ãšããŠç®¡çã§ãããã
ETLã¯å€æããŠããããŒãïŒExtract â Transform â LoadïŒãELTã¯ããŒãããŠãã倿ïŒExtract â Load â TransformïŒã ããã¯ã©ãŠãDWHãå®äŸ¡ã§åŒ·åã«ãªã£ãä»ã¯ããŸãçããŒã¿ããã®ãŸãŸDWHã«å ¥ããŠãããŠãå¿ èŠãªãšãã«DWHå ã§SQLã§å€æããELTãäž»æµã«ãªã£ãŠããŠãããã ãdbtã¯ãã®ãDWHå ã§ã®SQL倿ãã管çããããŒã«ã ãã
dbtã®ã¢ãã«ã¯SELECTæã ãæžãã°ããSQLãã¡ã€ã«ã ãã`models/` ãã©ã«ãã« `.sql` ãã¡ã€ã«ã眮ããšãdbtãèªåã§CREATE TABLE/VIEWã«å€æããŠå®è¡ããŠããããã ã`ref('ä»ã®ã¢ãã«å')` ãšããæ§æã§å¥ã¢ãã«ãåç §ãããšãäŸåé¢ä¿ãèªåã§è§£æ±ºããŠæ£ããé åºã§å®è¡ããŠãããã®ã䟿å©ãªãšããã ãã
ãã¹ãæ©èœã£ãŠäœããã¹ãããã®ïŒ
ããŒã¿ã®å質ãã§ãã¯ã ãã`schema.yml` ã«ãã¹ãå®çŸ©ãæžããšã`dbt test` ã§èªåæ€èšŒããŠããããã ãäž»ãªãã¹ãçš®é¡ã¯ `unique`ïŒéè€ãªãïŒã`not_null`ïŒNULLãªãïŒã`accepted_values`ïŒèš±å¯ãããå€ã®ã¿ïŒã`relationships`ïŒåç §æŽåæ§ïŒã®4ã€ã§ãããŒã¿ãã€ãã©ã€ã³ãå£ãããšãã«æ©æçºèŠã§ãããã
ããã¥ã¡ã³ãæ©èœã£ãŠäœã䟿å©ãªã®ïŒ
dbt docs generateãå®è¡ãããšãã¢ãã«éã®äŸåé¢ä¿ã°ã©ããšåã«ã©ã ã®èª¬æãå«ãWebãµã€ããèªåçæãããããschema.ymlã«descriptionãæžããŠããã°ãããåæ ããããã ãããã®ããŒãã«ã®ã«ã©ã äœã ã£ãïŒãããã®ã¢ãã«ã¯ã©ã®ããŒãã«ã«äŸåããŠãïŒããããŒã å šå¡ã確èªã§ãããããŒã¿ã«ã¿ãã°ãšããŠæ©èœãããã
dbt CoreãšdbtCloudã£ãŠäœãéãã®ïŒ
dbt Coreã¯OSSã®CLIããŒã«ã§ç¡æãdbt Cloudã¯ManagedãµãŒãã¹ã§ææïŒéçºè 1人ãŸã§ç¡ææ ããïŒã ããCoreã¯ããŒã«ã«ãCI/CDãã€ãã©ã€ã³ã§äœ¿ã圢ã§ãCloudã¯ãã©ãŠã¶äžã®IDEã»ã¹ã±ãžã¥ãŒã«å®è¡ã»éç¥ãæã£ã䟿å©ãªç®¡çç»é¢ã ããå人åŠç¿ã¯CoreãããŒã éçºãæ¬çªéçšã¯Cloudãéžã°ããããšãå€ããã
å®åã§ã¯ã©ããªé¢šã«dbtã䜿ãã®ïŒ
äžè¬çãªæ§æã¯ãFivetran/Airbyteã§ããŒã¿ãDWHã«ããŒã â dbtã®Stagingå±€ïŒçããŒã¿ãæŽåœ¢ïŒâ Martå±€ïŒBIããŒã«åãã«éèšïŒâ TableauãLookerã§å¯èŠåããšããæµãã ããã¢ãã«ãGitã§ç®¡çããŠPRæã«CIïŒdbt buildïŒãåãéçšãæ®åããŠãŠãããŒã¿ãšã³ãžãã¢ãªã³ã°ã«ãœãããŠã§ã¢éçºã®å質管çãæã¡èŸŒãã ã®ãdbtã®é©æ°çãªãšãããªãã ã