proofstone / become / data science & big data / data engineering

Data engineering

The pipes data flows through: collection, storage, pipelines — and the databases themselves. Pick a profession: each one is a map you walk node by node, and every card says, up front, what you need before you start.

4 professions · 4 planned

Data engineer

planned

The pipes data flows through: collection, cleaning, storage, orchestration.

stack: SQL · Python · Airflow · dbt · Spark

before you start: you already write code

Big data engineer

planned

The same pipes at cluster scale: distributed compute and storage.

stack: Spark · Kafka · Hadoop · Flink

before you start: you already build data pipelines

Database engineer

planned

The databases themselves: schemas, performance, backups, replication.

stack: PostgreSQL · MySQL · indexes · replication

before you start: you already write SQL

Data architect

planned

The whole data platform on one drawing: models, flows, storage, who may read what.

stack: warehouses · data modelling · governance

before you start: you already run data pipelines in production