Commit Graph

22 Commits (081adf4aaa7a11a4068acf123985fb5550138ec9)

Author SHA1 Message Date
Devosend 081adf4aaa
[Feature-7347] [Python]Add workflow as code task type flink (#7632)
* add Flink task for code as flow

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-01-04 09:45:54 +08:00
Jiajie Zhong cc8fbe3e14
[python] Fix task condition set wrong deps (#7650)
After #7505 merged. we could use condition task type
but our dependent set in the wrong direction, all
the condition operators should be upstream of the
current task instead of downstream

fix: #7649
2021-12-27 20:37:40 +08:00
Jiajie Zhong b1afd99c9e
[python] Fix task relation wrong hashable function (#7628)
* [python] Fix task relation wrong hashable function

Currently our :class:`TaskRelation` have wrong __hash__
function, would mistake a object as same object. It failed
our dependence and process definition. this patch fix it
by adding `_KEY_ATTR` to overwrite TaskRelation's __eq__
and correct __hash__ func

close: #7627

* Fix code format
2021-12-26 15:44:30 +08:00
Jiajie Zhong 946a0c7c57
[python] Add task switch (#7531)
* [python] Add task switch

close: #6928

* Fix code style
2021-12-22 11:46:34 +08:00
Jiajie Zhong e23a4848c0
[python] Add task condition (#7505)
* [python] Add task condition

close: #6927

* Add example and set downstream dep
2021-12-22 11:06:45 +08:00
Devosend 0c7aa4e2c5
[Feature-7349] [Python]Add workflow as code task type datax (#7437)
* support datax task by python
* add task datax example

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2021-12-17 11:12:48 +08:00
Jiajie Zhong 6f93ebf3ba
[python] Add task dependent (#7405)
fix: #6926
2021-12-16 09:58:50 +08:00
Jiajie Zhong 4988004150
[python] Fix task procedure define error (#7407)
Cause procedure parameter name need `method` instead of `sql`
we change it's name. And remove parameter `sql` from parent
class `Database`

closes: #7386
2021-12-15 10:19:32 +08:00
Jiajie Zhong 1948030151
[python] Add task base database and procedure (#7279)
We add a new task procedure, and add parent class database
for both sql task and procedure task

fix: #6929
2021-12-13 19:54:24 +08:00
Jiajie Zhong 2a7d6b468f
[python] Refactor get object define communicate to gateway (#7272)
* Change class Base `to_dict` to `get_define` for more clearer
* Remove class TaskParams and sub class make code easy to
  understand and avoid task implement cycle dependence
* Remove class ObjectJsonBase in Task to reduce complexity

fix: #7271
2021-12-08 22:17:13 +08:00
Devosend f480b8adf8
[Feature-6930] [Python]Add workflow as code task type sub_process (#7022)
* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task
2021-12-03 10:26:45 +08:00
Jiajie Zhong d93248acbc
[python] Add custom exception (#6935) 2021-11-23 16:59:38 +08:00
Jiajie Zhong 41e8836c91
[python] Add task sql (#6968)
* [python] Add task sql

* Add java gateway function doc
2021-11-23 16:58:00 +08:00
Jiajie Zhong 6e812be14d
[python] Add ut cover process definitions separate mode (#6908)
* [python] Add ut cover process definitions separate mode

* Remove unused print syntax

* Fix test error, cause raw_script not exists anymore
2021-11-19 17:24:19 +08:00
Jiajie Zhong 0dce68edd7
[python] Add task type python http (#6906)
* [python] Add task type python http

* Fix unittest error

* Fix UT error
2021-11-19 16:37:22 +08:00
Jiajie Zhong 7c5c7ec907
[ci][python] Add isort to sort out import (#6871)
* Add isort config file to fix conflict with black
* Add some doc about isort
2021-11-17 11:48:52 +08:00
Jiajie Zhong 54933b33e3
[ci][python] Add coverage check in CI (#6861)
* [ci] Add coverage check in CI

* Coverage add dependent

* Install pydolphinscheduler before run coverage

* Up test coverage to 87% and down threshold to 85%

* Fix code style

* Add doc about coverage
2021-11-17 09:46:40 +08:00
Jiajie Zhong e76cf77040
[python] Add parameter schedule for process definition (#6664)
* [python] Add parameter schedule for process definition

* Rebase and fix some code style

* May schedule work on both string and datetime

* Fix flaky test

* Add comment about freeze time

* Add edge test for schedule_json with None schedule

* Fix test function name

* Fix rebase error
2021-11-13 17:21:40 +08:00
Jiajie Zhong cd8205217a
[python] Fix process definition's user assigned not pass (#6753) 2021-11-13 16:47:08 +08:00
Jiajie Zhong 32d4411469
[python] Add flake8 and black for code style and integrated to GA (#6679)
* [python] Add code style lint for GA

* Change github action name

* Auto change by black

* Fix flake8

* Fix broken link for pyds README.md

* Auto fix by black

* Separate GitHub workflows

* Add Black badge and CI locally in README.md
2021-11-05 15:14:28 +08:00
Jiajie Zhong 1165afbdd1
[python] Add param workgroup to process definition (#6667) 2021-11-04 22:25:45 +08:00
Jiajie Zhong 2e7036529c
Add CI test for python API (#6636)
* Add CI for python API

* Fix task assign without process definition

* Fix ci

* Add header

* Add build badge for pydolphinscheduler

* Add content to README.md

* Remove unnecessary blank line
2021-11-04 11:19:11 +08:00