Commit Graph

21 Commits (dev)

Author SHA1 Message Date
陈家名 8a8b63cd96
[Improve][python] Support create table syntax and custom sql type param (#9673) 2022-04-25 10:17:20 +08:00
Jiajie Zhong 1f48601c75
[python] Add task decorator for python function (#9496)
* [python] Add task decorator for python function

* Add decorator `@task`
* Add a tutorial about it
* Change tutorial doc and combine into traditional docs
  * Add sphinx-inline-tab for better view

* revert not need change

* Correct python function indent

* Correct integration test
2022-04-15 15:50:52 +08:00
Jiajie Zhong 35578efd08
[python] Initiate document for pydolphinscheduler (#8005) 2022-01-26 12:37:57 +08:00
Devosend 7aeac75860
[Feature-7348] [Python] Add workflow as code task type mr (#8140) 2022-01-21 09:54:36 +08:00
Devosend 027af091c7
[Feature-7346] [Python]Add workflow as code task type spark (#7968)
* add spark task

* fix code format

* add parent class for flink and spark

* modify Engine docstring

* modify docstring of Engine
2022-01-15 23:24:55 +08:00
Jiajie Zhong 1417967d9e
[python] Task condition missing two downstream param (#7783)
* [python] Task condition missing two downstream param

We add two downstream tasks to set task condition
success and failed node

close: #7763

* Add getter and setter property condition_resulth in base task
2022-01-05 19:58:44 +08:00
Devosend 081adf4aaa
[Feature-7347] [Python]Add workflow as code task type flink (#7632)
* add Flink task for code as flow

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-01-04 09:45:54 +08:00
Jiajie Zhong cc8fbe3e14
[python] Fix task condition set wrong deps (#7650)
After #7505 merged. we could use condition task type
but our dependent set in the wrong direction, all
the condition operators should be upstream of the
current task instead of downstream

fix: #7649
2021-12-27 20:37:40 +08:00
Jiajie Zhong 946a0c7c57
[python] Add task switch (#7531)
* [python] Add task switch

close: #6928

* Fix code style
2021-12-22 11:46:34 +08:00
Jiajie Zhong e23a4848c0
[python] Add task condition (#7505)
* [python] Add task condition

close: #6927

* Add example and set downstream dep
2021-12-22 11:06:45 +08:00
Devosend 0c7aa4e2c5
[Feature-7349] [Python]Add workflow as code task type datax (#7437)
* support datax task by python
* add task datax example

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2021-12-17 11:12:48 +08:00
Jiajie Zhong 6f93ebf3ba
[python] Add task dependent (#7405)
fix: #6926
2021-12-16 09:58:50 +08:00
Jiajie Zhong 4988004150
[python] Fix task procedure define error (#7407)
Cause procedure parameter name need `method` instead of `sql`
we change it's name. And remove parameter `sql` from parent
class `Database`

closes: #7386
2021-12-15 10:19:32 +08:00
Jiajie Zhong 1948030151
[python] Add task base database and procedure (#7279)
We add a new task procedure, and add parent class database
for both sql task and procedure task

fix: #6929
2021-12-13 19:54:24 +08:00
Jiajie Zhong 2a7d6b468f
[python] Refactor get object define communicate to gateway (#7272)
* Change class Base `to_dict` to `get_define` for more clearer
* Remove class TaskParams and sub class make code easy to
  understand and avoid task implement cycle dependence
* Remove class ObjectJsonBase in Task to reduce complexity

fix: #7271
2021-12-08 22:17:13 +08:00
Devosend f480b8adf8
[Feature-6930] [Python]Add workflow as code task type sub_process (#7022)
* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task
2021-12-03 10:26:45 +08:00
Jiajie Zhong d93248acbc
[python] Add custom exception (#6935) 2021-11-23 16:59:38 +08:00
Jiajie Zhong 41e8836c91
[python] Add task sql (#6968)
* [python] Add task sql

* Add java gateway function doc
2021-11-23 16:58:00 +08:00
Jiajie Zhong 0dce68edd7
[python] Add task type python http (#6906)
* [python] Add task type python http

* Fix unittest error

* Fix UT error
2021-11-19 16:37:22 +08:00
Jiajie Zhong 32d4411469
[python] Add flake8 and black for code style and integrated to GA (#6679)
* [python] Add code style lint for GA

* Change github action name

* Auto change by black

* Fix flake8

* Fix broken link for pyds README.md

* Auto fix by black

* Separate GitHub workflows

* Add Black badge and CI locally in README.md
2021-11-05 15:14:28 +08:00
Jiajie Zhong 2e7036529c
Add CI test for python API (#6636)
* Add CI for python API

* Fix task assign without process definition

* Fix ci

* Add header

* Add build badge for pydolphinscheduler

* Add content to README.md

* Remove unnecessary blank line
2021-11-04 11:19:11 +08:00