PROJECT 01
Retail medallion lakehouse
Raw sales files on ADLS Gen2 become clean silver tables and business ready gold tables, loaded incrementally.
Auto LoaderDelta LakeMERGE
01 / OVERVIEW
What this course is about
Databricks is where large scale data processing happens on Azure. This course takes you from Spark basics to production lakehouse pipelines: PySpark transformations, Delta Lake, bronze, silver and gold layers, Auto Loader, Unity Catalog, Workflows and performance tuning. Everything is built on Azure with ADLS Gen2, the same setup I use at work.
You learn by doing: every week has hands-on exercises on Google Classroom, and every module ends with something you built yourself.
01
Think in Spark
Drivers, executors, partitions, lazy evaluation and how jobs really run.
02
Transform data with PySpark
Joins, aggregations, window functions and clean reusable code.
03
Use Delta Lake properly
MERGE, time travel, OPTIMIZE, VACUUM and schema evolution.
04
Build a medallion lakehouse
Bronze, silver and gold layers with incremental loads.
05
Govern with Unity Catalog
Catalogs, external locations, permissions and lineage.
06
Tune and deploy
Spark UI, shuffles, broadcast joins, Workflows and CI/CD.
02 / CURRICULUM
Week by week
A clear plan from the first session to the last. The pace adapts to you: if you already know a topic we move faster, and if something needs more time we take it.
WEEK 1Spark and Databricks basics5 topics
- The lakehouse idea
- Workspace, compute and notebooks
- Spark architecture
- Lazy evaluation and actions
- Volumes and file access
WEEKS 2 TO 3PySpark DataFrames5 topics
- Reading CSV, JSON and Parquet
- select, filter and withColumn
- Joins and aggregations
- Window functions
- Built-in functions vs UDFs
WEEK 4Spark SQL5 topics
- Temporary views
- SQL cells in notebooks
- CTEs and complex queries
- Mixing SQL and Python
- Parameters and widgets
WEEK 5Delta Lake5 topics
- ACID transactions
- MERGE for upserts
- Time travel
- OPTIMIZE, liquid clustering and VACUUM
- Schema enforcement and evolution
WEEK 6Medallion architecture5 topics
- Bronze, silver and gold design
- Auto Loader
- Incremental processing
- Structured Streaming basics
- Late and bad data
WEEK 7Unity Catalog5 topics
- Metastore, catalogs and schemas
- Access connector and managed identity
- External locations on ADLS Gen2
- Grants and permissions
- Lineage
WEEK 8Orchestration5 topics
- Databricks Workflows and jobs
- Task dependencies and parameters
- Calling Databricks from Azure Data Factory
- Retries and alerts
- Job clusters vs all-purpose compute
WEEK 9Declarative pipelines and data quality5 topics
- Lakeflow Declarative Pipelines (DLT)
- Expectations for data quality
- Streaming tables and materialized views
- Quarantine patterns
- Monitoring pipeline runs
WEEK 10Performance and cost5 topics
- Reading the Spark UI
- Partitions and shuffles
- Broadcast joins and caching
- Adaptive query execution
- Cluster sizing and cost
WEEK 11CI/CD5 topics
- Git folders
- Databricks Asset Bundles
- Dev, test and prod environments
- Unit testing PySpark
- Code review habits
WEEK 12Capstone and certification prep4 topics
- End to end lakehouse project
- Databricks Data Engineer Associate study plan
- Spark interview questions
- Mock interview with feedback
03 / PROJECTS
What you will build
Real projects you can put on GitHub and explain with confidence in interviews.
PROJECT 02
Streaming IoT ingestion
Ingest device events continuously, handle late data and publish near real-time aggregates.
Structured StreamingWatermarksDelta
PROJECT 03
Pipeline tuning challenge
Take a slow, expensive job and make it faster and cheaper, with before and after numbers from the Spark UI.
Spark UIPartitioningBroadcast joins
04 / TOOLS
Tools and technologies
The same tools used by data teams in real companies.
Azure DatabricksPySparkSpark SQLDelta LakeUnity CatalogAuto LoaderDatabricks WorkflowsADLS Gen2Azure Data FactoryGit
05 / IS IT FOR YOU
Who this course is for
Who it is for
- Data engineers who want strong Spark and lakehouse skills
- SQL and Python developers moving into big data
- ETL developers modernising to Azure
- Anyone preparing for the Databricks Data Engineer Associate
What you need
- SQL basics
- Python basics (variables, functions, lists)
- An Azure free account or company sandbox, I guide the setup
Roles it leads to
- Databricks Engineer
- Data Engineer
- Big Data Engineer
- Lakehouse Engineer
06 / HOW YOU LEARN
What you get
Live and one-to-one
Every session is just you and me, never a batch. No recordings, so we can talk openly about your code.
3 sessions a week
Each session is 1 hour 15 minutes, on weekdays, weekends or both. We agree the schedule before starting.
Google Classroom
Assignments, projects, quizzes, interview questions and homework, all organised in one place.
Real projects
You build projects like the ones companies run, and get feedback on your code and design.
Interview preparation
Common questions, live coding practice and mock interviews with honest feedback.
Job placement support
CV, LinkedIn, applications and interviews. I stay with you until you are hired, as long as you do the work.
07 / PRICING
Simple pricing
You pay for your time with me, not per course. All prices are in US dollars.
Estimated total for Azure Databricks Engineer at the typical pace (3 months)
About 600 USD
Full refund if it is not right for you after the first session. Go faster and it costs less.About 600 USD
08 / YOUR MENTOR
Hikmat Ullah
Senior Data Engineer at Algo · Lahore, Pakistan
I build data platforms with Snowflake, Azure Databricks, Azure Data Factory and Microsoft Fabric every day, and I have been mentoring since 2021. I teach what I use at work, and I stay with you until you land the job.
7+years in data
100+students trained
80+placed in jobs
1:1always, never a batch
09 / FAQ
Questions about Azure Databricks Engineer
Will I need an Azure subscription?+
Yes, a free or pay as you go Azure account is enough. I show you how to keep costs very low with small clusters, auto termination and job compute.
Is PySpark hard if I only know basic Python?+
No. We build PySpark skills step by step, and most of it reads like SQL. Basic Python is enough to start.
Is Databricks still used if a company has Fabric or Snowflake?+
Yes. Many teams run Databricks next to Snowflake or Fabric. I work with Databricks and Snowflake together every day.
How much does it cost?+
200 USD per month whichever course you choose. You can also pay 100 USD every 15 days or 50 USD per week. All prices are in US dollars wherever you live.
What is the refund policy?+
If the training is not right for you after the first session, you get a full refund. After that, payments already used for completed sessions are not refunded, but unused prepaid time can be refunded.
Is there a job guarantee?+
Yes. As long as you attend your sessions and complete the assignments and projects, I keep helping you with CV, LinkedIn, mock interviews and job applications until you are hired.
Can I join from another country or time zone?+
Yes. Students join from many countries. We pick session times that work for your time zone before we start.
10 / KEEP GOING
Related courses
3 months
Python Developer
Python for data work: pandas, APIs, automation, testing and real ETL.
View course → 6 monthsAzure Data Engineer
The complete path: SQL, Python, ADF, ADLS Gen2, Databricks, Synapse and Event Hubs.
View course → 3 monthsSnowflake Developer
Warehousing, Snowpipe, streams and tasks, Snowpark and real cost control.
View course →START WHEN YOU ARE READY
Start the Azure Databricks Engineer course
Tell me your goals and your time zone. We plan your schedule together, and your first session can start this week.