Skip to content

Commit 167009d

Browse files
Merge pull request #888 from keboola/devin/1773929518-custom-python-docs
docs: add Custom Python component documentation
2 parents ee0dc37 + 6a22530 commit 167009d

7 files changed

Lines changed: 141 additions & 0 deletions

File tree

_data/navigation.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -573,6 +573,8 @@ items:
573573
title: dbt Cloud Job Trigger
574574
- url: /components/applications/triggers/deepnote-notebook-execution-trigger/
575575
title: Deepnote Notebook Execution Trigger
576+
- url: /components/applications/custom-python/
577+
title: Custom Python
576578
- url: /components/applications/data-gateway/
577579
title: Data Gateway
578580
- url: /components/applications/ai/
240 KB
Loading
27.4 KB
Loading
208 KB
Loading
86.4 KB
Loading
59.2 KB
Loading
Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
---
2+
title: Custom Python
3+
permalink: /components/applications/custom-python/
4+
---
5+
6+
* TOC
7+
{:toc}
8+
9+
This component lets you run custom Python code directly in Keboola. Use it to call external APIs,
10+
process files, push data to third-party systems, or build any integration that isn't covered by an existing connector —
11+
all without leaving Keboola.
12+
13+
{: .image-popup}
14+
![Screenshot - Custom Python Component Overview](/components/applications/custom-python/custom-python-0.png)
15+
16+
## When to Use Custom Python
17+
18+
| Scenario | Use |
19+
|---|---|
20+
| Transform or aggregate data already in Keboola Storage | [Python Transformation](/transformations/python-plain/) |
21+
| Connect to an external API or service, download or push data | **Custom Python** |
22+
| Store and use sensitive credentials (API keys, passwords, tokens) | **Custom Python** |
23+
| Share logic across multiple Keboola projects | **Custom Python** |
24+
25+
## Key Features
26+
27+
- **Secure parameters** — Encrypt API keys, tokens, and passwords directly in the Keboola UI using the `#` prefix.
28+
- **Flexible code source** — Write code in the Keboola UI, or load it from a Git repository.
29+
- **Clean environment** — Each run uses an isolated container. Install only the packages you need.
30+
- **Multiple Python versions** — Choose Python 3.12, 3.13, or 3.14.
31+
32+
## Configuration
33+
34+
[Create a new configuration](/components/#creating-component-configuration) of the **Custom Python** application.
35+
36+
### Python Version
37+
38+
{: .image-popup}
39+
![Screenshot - Python Version Selection](/components/applications/custom-python/custom-python-1.png)
40+
41+
Select the Python version for your code:
42+
43+
- **3.14** *(recommended)* — Isolated environment with only the packages you install.
44+
- **3.13** — Isolated environment.
45+
- **3.12** — Isolated environment.
46+
- **base** — Shared environment (Python 3.10) with many pre-installed packages in legacy versions. Not recommended; will be deprecated in the future.
47+
48+
{% include tip.html content="Use an isolated environment (Python 3.12 or newer) to avoid package version conflicts and keep your code maintainable." %}
49+
50+
### Source Code & Dependencies
51+
52+
Choose where your Python code comes from:
53+
54+
**Enter manually into text areas below** *(default)*
55+
56+
{: .image-popup}
57+
![Screenshot - Inline Code Source](/components/applications/custom-python/custom-python-3-code.png)
58+
59+
Write or paste your code directly in the Keboola UI. Use the **Python Packages** field to list any extra packages to install.
60+
61+
**Get from Git repository**
62+
63+
Load code from a public or private Git repository. See [Git Configuration](#git-configuration) below.
64+
65+
### User Parameters
66+
67+
{: .image-popup}
68+
![Screenshot - User Parameters](/components/applications/custom-python/custom-python-2.png)
69+
70+
Use **User Parameters** to pass configuration values to your code — for example an API base URL,
71+
a target table name, or debug flags. Enter them as a JSON object.
72+
73+
Any key starting with `#` (e.g., `#api_key`) will be **encrypted** when you save the configuration.
74+
The decrypted value is available to your code at runtime via `CommonInterface`.
75+
76+
## Git Configuration
77+
78+
When using **Get from Git repository** as your code source, fill in the repository settings:
79+
80+
{: .image-popup}
81+
![Screenshot - Git Repository Configuration](/components/applications/custom-python/custom-python-3-git.png)
82+
83+
- **Repository URL** — Supports both HTTPS and SSH formats.
84+
- **Repository Visibility & Authentication** — Choose one of:
85+
- **Public – None** — Public repository, no credentials needed.
86+
- **Private – Personal Access Token** — The token is encrypted in Keboola Storage.
87+
- **Private – SSH Key** — The private key is encrypted in Keboola Storage.
88+
- **Branch Name** — The branch to check out. Defaults to `main`. Use **List Branches** to pick from the repository.
89+
- **Script Filename** — The Python file to run. Defaults to `main.py`. Use **List Files** to pick from the repository.
90+
91+
### Dependencies
92+
93+
Place your dependency file in the repository root:
94+
95+
- `pyproject.toml` + `uv.lock` *(recommended)* — Modern approach using [uv](https://docs.astral.sh/uv/).
96+
- `requirements.txt` — Legacy approach.
97+
98+
If both are present, `pyproject.toml` + `uv.lock` takes precedence.
99+
100+
## Writing Your Code
101+
102+
Your code communicates with Keboola Storage and configuration via the
103+
[`keboola-component`](https://pypi.org/project/keboola-component/) Python library and `CommonInterface`.
104+
105+
The following snippet is pre-filled in every new configuration and shows the basic pattern:
106+
107+
```python
108+
from keboola.component import CommonInterface
109+
110+
ci = CommonInterface()
111+
# Access user parameters defined in the configuration
112+
print(ci.configuration.parameters)
113+
```
114+
115+
For the full `CommonInterface` reference — including reading input tables, writing output tables,
116+
working with files, state files, error handling, and logging — see the
117+
[Developer Documentation](https://developers.keboola.com/extend/component/tutorial/).
118+
119+
## Using Kai to Configure Custom Python
120+
121+
[Kai](/kai/), Keboola's AI assistant, can help you set up and work with the Custom Python component directly
122+
from the Keboola UI. Open Kai in your project and describe what you need — for example:
123+
124+
- *"Create a Custom Python configuration that fetches data from the Stripe API and saves it to Storage."*
125+
- *"Help me set up a Custom Python component to push data from my orders table to a REST API."*
126+
- *"Debug my Custom Python job — it's failing with a connection timeout."*
127+
128+
Kai understands the Custom Python component's configuration, the `CommonInterface` library,
129+
and Keboola best practices. It can generate code snippets, configure user parameters with
130+
encrypted credentials, set up Git repository integration, and troubleshoot failed jobs —
131+
all within the chat.
132+
133+
{% include tip.html content="When asking Kai to build a Custom Python configuration, be specific about the external service,
134+
authentication method, and which tables or files you want to read or write. The more context you provide, the better the result." %}
135+
136+
## Example Repository
137+
138+
We have prepared a [simple example project](https://github.com/keboola/component-custom-python-example-repo-1)
139+
to help you get started with running code from a Git repository. It can also serve as a template for your own projects.

0 commit comments

Comments
 (0)