The Sr Data Engineer track is appropriate for data team leads and those who are responsible for being an architect over multiple Keboola projects, or over a complex data pipeline.

This track is good for those who find themselves in the current situation, or have ambitions to be:

  • Leading a data team
  • Lead a data team for a medium - large size organization
  • Lead a data team with ambitious goals, including data science and modelling, beyond BI and analytics. 


The certificate requires completion of Introduction, Best Practices, Debug Techniques, Common Components and Processors, Generic Extractor, KBC APIs, Business Data Model, and Multi Project Architecture

If you have already completed any of the assignments, you will not be required to resubmit in order to qualify for completion of this course.

Some knowledge or experience with calling APIs is recommended in order to complete the Generic Extractor. Refer to Generic Extractor course description for references to introductory materials. 

Knowledge and experience with integrating with APIs is recommended for completing KBC API

Course curriculum

  • 1

    Certificate Course Introduction

    • Certificate Course Introduction

  • 2

    Introduction

    • Architecture Overview

    • Extractors

    • Storage

    • Transformations and Workspaces

    • Old Transformations

    • Jobs

    • Writers

    • Orchestrations

    • Additional Transformations

    • Scaffolds

    • Support

    • Trash

    • Resources

  • 3

    Best Practices

    • Core Architecture Explanation

    • Storage Part 1

    • Storage Part 2

    • Snowflake Transformation Tips

    • ** Note

    • Dependencies and Phases

    • Data Science Transformations

    • Variables

    • Shared Code

    • Workspaces

    • Input Mapping & Output Mapping - Incrementals

    • Orchestrations

    • Assignment Overview

    • Assignment

    • Resources

  • 4

    Debug Techniques

    • Agenda and Recap of Important Concepts

    • Common Error Root Causes

    • Storage and Job Event Details

    • Practical Example - Invalid SQL Queries

    • Practical Example - Python and Snowflake Output Errors

    • Practical Example - Missing Rows in Output

    • **Note

    • Practical Example - Merged Load Types

    • Practical Example - Unable to Merge Primary Key

    • Practical Example - Invalid Cypher

    • Dropped Table

    • Resources

    • Assignment

  • 5

    Business Data Model

    • Methodology Overview

    • Keboola BDM Guide

    • Exercise Intro

    • Live BDM Session Exercise

    • Assignment Overview

    • Assignment

    • Submission Requirements

  • 6

    Common Components and Processors

    • Intro and Components Overview

    • Public vs Private Components

    • Component Developers

    • Component Configurations

    • Processors

    • Sample Solution with Processors

    • S3 Processors Deep Dive

    • Common Components 1 - FTP, Email, HTTP

    • Common Components 2 - KBC, Geocode, Apify, Selenium

    • Common Components 3 - Generic Extractor

    • Common Components - Textual Data Applications

    • Common Components 5 - Mailgun

    • Common Processors

    • Assignment Overview

    • Assignment

    • S3 Intro

    • How to Perform a Debug Job via API

    • Resources

  • 7

    KBC APIs

    • Introduction

    • KBC APIs Overview

    • Tokens and Security

    • Note on KBC Regions

    • Example 1

    • Example 2

    • Example 3

    • Other Tips and Tricks

    • Assignment

    • Resources

  • 8

    Generic Extractor

    • Session Overview

    • Documentation and Getting Started

    • Model Example and Configuration Tutorial

    • API Section

    • Config Section Part 1

    • Config Section Part 2

    • Config Section Part 3 - Child Jobs

    • Config Section Part 4 - Functions

    • Authentication

    • Pagination

    • Mappings

    • User Data

    • Publishing a Generic Extractor

    • Assignment Overview

    • Assignment

    • Resources

  • 9

    Multi Project Architecture

    • Introduction

    • From Data Tool to Data Platform

    • Architecture Design

    • Model Example - Bank Part 1

    • Model Example - Bank Part 2

    • From Single Project to Multi Project

    • Architecture Blueprint

    • Examples

    • Sharing Data

    • Orchestrations

    • Quiz

  • 10

    Certificate Submission

    • Certificate Submission