Data Engineer
Построение и поддержка data pipelines, хранилищ данных, обеспечение доступности и качества данных
Уровень:
Data Engineering
14 навыков · 70 требований| Навык | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Batch-обработка данных | |||||
| Apache Spark | Awareness | Working | Advanced | Expert | Expert |
| dbt | Awareness | Working | Advanced | Expert | Expert |
| Pandas / Polars | Awareness | Working | Advanced | Expert | Expert |
| SQL-based ETL | Awareness | Working | Advanced | Expert | Expert |
| Stream-обработка | |||||
| Stream Processing | Awareness | Working | Advanced | Expert | Expert |
| Data Lakehouse | |||||
| Delta Lake / Apache Iceberg | Awareness | Working | Advanced | Expert | Expert |
| Архитектура Data Lake | Awareness | Working | Advanced | Expert | Expert |
| Data Warehousing | |||||
| Data Warehouse Design | Awareness | Working | Advanced | Expert | Expert |
| Управление данными | |||||
| Data Catalog | Awareness | Working | Advanced | Expert | Expert |
| Data Lineage | Awareness | Working | Advanced | Expert | Expert |
| Data Contracts | Awareness | Working | Advanced | Expert | Expert |
| Качество данных | |||||
| Data Quality | Awareness | Working | Advanced | Expert | Expert |
| Оркестрация данных | |||||
| Apache Airflow | Awareness | Working | Advanced | Expert | Expert |
| Dagster / Prefect | Awareness | Working | Advanced | Expert | Expert |