This course will walk you through how to set up your own Generic Extractor.  That's an extractor which is not predefined between Keboola components. Prior experience with REST APIs will be required to make the most use of this course.

We will walk through step by step, how to set up a connection to a REST API with Generic Extractor, and cover topics such as authentication, pagination, jobs and endpoints, recursive child endpoints, and more.

After completing this course, you will be able to create your own Generic Extractor.

Required Knowledge:

Knowledge of what an API is and how to make API calls is required.

  • If you have little to no knowledge of APIs, Zapier has a great guide to get yourself familiar. 

Knowledge in reading and writing JSONs will be necessary

  • JSON is a type of data structure (ie. a table in a database is acolumnar data structure).  If you're not familiar, there are guides around the internet, including here and here
  • JSONs are how REST APIs generally will return data.
  • The Generic Extractor is configured by writing a JSON configuration (don't worry, we provide templates in many places so you typically won't write them from scratch).


The course is a part of these certificates: Sr Data Engineer, Developer

Total Video Lessons Length: 57 Minutes

Time for your notes, coffee breaks, video replay: N/A (individual)

Assignment Length: 90 - 150 Minutes

Course curriculum

  • 1

    Introduction

    • Session Overview and Introduction

    • Documentation and Getting Started

  • 2

    Configuration

    • Configuration Tutorial and Session Example

    • API Section

    • Config Section Part 1

    • Config Section Part 2

    • Config Section Part 3 - Child Jobs

    • Config Section Part 4 - Functions

    • Authentication

    • Pagination

    • Table and Column Mapping

    • User Data

    • Publishing a Generic Extractor Component

  • 3

    Assignment

    • Assignment Overview

    • Assignment

  • 4

    Resources

    • Resources

  • 5

    Before you go ...

    • Course Feedback