Commit Graph

5417 Commits (e76cf77040ae558461c54730c27eb0fbe28bc54f)

Author SHA1 Message Date
Jiajie Zhong e76cf77040
[python] Add parameter schedule for process definition (#6664)
* [python] Add parameter schedule for process definition

* Rebase and fix some code style

* May schedule work on both string and datetime

* Fix flaky test

* Add comment about freeze time

* Add edge test for schedule_json with None schedule

* Fix test function name

* Fix rebase error
2021-11-13 17:21:40 +08:00
Assert 089f73ebe4
[Fix-6707][common] Fix complement data error because of global variables (#6759)
* [Fix-6707][common] Fix complement data error because of global variables

* [Fix-6707][common] Fix complement data error because of global variables

* [Fix-6707][common] Fix complement data error because of global variables

* [Fix-6707][common] Fix complement data error because of global variables

Co-authored-by: shangeyao <sgy960921>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
2021-11-13 17:06:15 +08:00
Jiajie Zhong 0f92a1f541
[community] Add version selection for issue bug selection (#6793)
closes: #6620

Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: Jiwei Guo <technoboy@apache.org>
2021-11-13 16:47:42 +08:00
Jiajie Zhong 6045728428
[ci] Add checkstyle prohibit rule System.out.println (#6778)
* [ci] Add checkstyle prohibit rule System.out.println

* Correct RegexpSingleline place
2021-11-13 16:47:29 +08:00
Jiajie Zhong cd8205217a
[python] Fix process definition's user assigned not pass (#6753) 2021-11-13 16:47:08 +08:00
wangyizhi e2d516f279
Optimize DAG page opening speed (#6830) 2021-11-12 15:48:55 +08:00
Jiajie Zhong 5453f1e349
[ci] Remove PR requests strict rebase (#6831) 2021-11-12 15:48:34 +08:00
wind 5741c758b3
[DS-6771][MasterServer] fix failover worker interrupt (#6801)
Co-authored-by: caishunfeng <534328519@qq.com>
Co-authored-by: Kirs <acm_master@163.com>
Co-authored-by: OS <29528966+lenboo@users.noreply.github.com>
2021-11-12 14:19:57 +08:00
OS d168a3cdf7
[feature][Install] 6597 add worker group when upgrading from version 1.x to 2.0 (#6806)
* feature 6597 add worker group. Upgrade dolphinscheduler version 1. X to version 2.0

* feature 6597 add worker group. Upgrade dolphinscheduler version 1. X to version 2.0

* code style

* code style

* code style

* code style

* update postgre sql

Co-authored-by: JinYong Li <42576980+JinyLeeChina@users.noreply.github.com>
2021-11-12 12:03:23 +08:00
JinYong Li dfe4945b48
remove processDefinition resourceIds (#6813) 2021-11-12 11:39:16 +08:00
mask e36d18b588
[Fix-6792] resolve API and alert port conflicts (#6792) (#6815)
* [Fix-6792]Multiple ApplicationRunners on classpath make SpringApplication.run only run one (#6792)
2021-11-12 10:51:11 +08:00
JinYong Li da2a044947
[Feature][API] add processDefinitionSimpleList api (#6814)
* add processDefinitionSimpleList api

* add processDefinitionSimpleList api
2021-11-11 23:45:41 +08:00
罗铭涛 975131e913
[Improvement#6655] [UI] remove registry monitor (#6789)
* [DS-6655][fix] remove zkmonitor
- delete zkmonitor ui

* [DS-6655][fix] remove zkmonitor
- delete zkmonitor ui
- Modify the actions.js comma

* [DS-6655][fix] remove zkmonitor
- remove all backend API

Co-authored-by: Kirs <acm_master@163.com>
2021-11-11 19:59:10 +08:00
Jiajie Zhong c110488269
[ci] Remove GA path rule in build and unittest (#6797)
CI could not be run due to we add 9113817b to asf
configure and same file change will not trigger
build or unit test github actions run. This patch
try to skip github action path ignore rule temporarily

Maybe we should find more effective way to spearate py
, js, java CI later
2021-11-11 17:32:57 +08:00
zwZjut 088a5a4706
add default constructor (#6780)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-11-11 12:11:43 +08:00
kezhenxu94 9113817b78
Require build and unit tests before merging (#6777) 2021-11-11 08:45:35 +08:00
kezhenxu94 11844415b0
Refactor datasource and task plugins and simplify its usage (#6767) 2021-11-10 18:29:17 +08:00
Jiajie Zhong 98f466e09e
Make sql file change synchronize (#6465)
closes: #6431
2021-11-10 14:50:05 +08:00
zhang 28f871e58b
[Fix-6764]modifying prompt messages and modify dolphinscheduler_mysql.sql dolphinscheduler version (#6765)
* [FIX:201106]fix issue#6657

* replacement version number

* update resource create error messag

Co-authored-by: 张柏强 <zhangbaiqiang@lvshou.com>
Co-authored-by: crane <zhangbq3306@163.com>
2021-11-09 23:34:13 +08:00
mask 0aea5ae11a
[feature-6071][plugin] add datasource plugin. (#6071) (#6361)
* [feature-6071][plugin] add datasource plugin. (#6071)

* remove the dependency of datasource in common and task-plugin module.

* the new datasource-api module provides data source connection acquisition.

* add sqlserver data source plug-in.

* add ds license

* fix check style

* update spi pom

* frx conflict resolution

* fix check style

* fix datasource ut

* fix code check style

* fix code check style

* fix datasource ut

* add pg hive mysql oracle clickhouse db2  datasource

* fix datasource package

* update datasource pom

* update datasource pom

* update datasource pom

* update datasource pom

* update pom

* update pom dep

* update pom dep

* fix license

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* add oracle package

* modify comments

* conflict resolution

* rename log comment

* rename log comment
2021-11-09 20:20:07 +08:00
kezhenxu94 05bae0b226
Refactor alert plugins and simplify its usage (#6723) 2021-11-09 18:59:34 +08:00
kezhenxu94 02350ea37b
Fix datasource not work in tests and standalone server, and run all tests by default (#6743) 2021-11-09 16:57:26 +08:00
JinYong Li be28d2c63b
[Bug][SnowFlakeUtils] fix snowFlake bug (#6744)
* fix snowFlake bug

* fix hash machine
2021-11-09 10:59:16 +08:00
GaoTianDuo 484153ef76
[Feature][Install-sql] supplement Postgre procedure sql file #6716 (#6741)
* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、add dolphinscheduler_ddl_post.sql

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、modify  dolphinscheduler_ddl.sql

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、modify Code annotation

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、modify sql:ALTER TABLE "t_ds_process_definition" ADD CONSTRAINT "t_ds_process_definition_pkey" PRIMARY KEY ("id","code");

* [Feature][Install-sql] supplement Postgre procedure sql file #6716
1、delete code annotation
2021-11-09 10:00:08 +08:00
kezhenxu94 63602bf77c
Clean up the utility codes (#6732) 2021-11-08 23:22:07 +08:00
Wenjun Ruan 0f94577d26
Use nanoTime replace currentTimemill to avoid clock backwards (#6740) 2021-11-08 22:26:17 +08:00
wind b6824b4741
[DS-6737][MasterServer] fix event handle twice (#6738)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-08 16:54:58 +08:00
JinYong Li bdea8d6ae4
[Upgrade][Install] fix upgrade 2.0 bug (#6734)
* add convert dependent/conditions

* fix upgrade 2.0 bug

* fix upgrade 2.0 bug
2021-11-08 11:57:38 +08:00
OS 24bd8a16e1
[Bug][Master] The task of dependent should use taskCode (#6730)
* fix #6682 The task of dependent should use taskCode

* fix #6682 The task of dependent should use taskCode
2021-11-07 23:20:35 +08:00
Wenjun Ruan 0afb780718
[Improvement-5807][Dao] Use h2 in UT (#6500)
* Use H2 in UT

* change scope

* Change profile name to h2

* Add datasource-h2.properties, datasource-postgresql.properties

* Fix UT

* Resolve conflict

* Remove startDatabase
2021-11-07 23:19:19 +08:00
wangxj3 d27c6ea8f0
fix bug (#6728)
Co-authored-by: wangxj <wangxj31>
2021-11-07 18:01:40 +08:00
kezhenxu94 5400abc74f
Refactor registry plugin and simplify its usage (#6712) 2021-11-07 15:18:10 +08:00
Yao WANG 73f20b7553
[Fix-#6641]fix npe when the LowerWeightHostManager can not get the workNodeInfo (#6656)
* change the return type of getHostWeight

Co-authored-by: ywang46 <ywang46@paypal.com>
2021-11-07 11:40:54 +08:00
wangyizhi b226253eec
Replace taskName with taskCode (#6719) 2021-11-07 10:35:29 +08:00
小张 69f2d3b21b
[FIX:201106]fix issue#6657 (#6717)
Co-authored-by: 张柏强 <zhangbaiqiang@lvshou.com>
2021-11-06 19:27:00 +08:00
Yanbin Lin a728c85a92
[Fix-6649][UI] Get project according to code (#6649) (#6650)
Co-authored-by: linyanbin <linyanbin@lizhi.fm>
2021-11-06 10:12:01 +08:00
JinYong Li 1144360257
add convert dependent/conditions (#6710) 2021-11-05 23:02:46 +08:00
myangle1120 b8d0f06ad8
[Feature-#6268][UI]Serial execte proces (#6267)
* serial

* del ip
2021-11-05 21:36:44 +08:00
ReonYu d4d59b0627
[FIX 6701] [API] Fix paging error when worker group number more than 10 (#6704) 2021-11-05 20:23:20 +08:00
kezhenxu94 5e923971bf
Migrate http to https in maven address (#6698) 2021-11-05 20:17:27 +08:00
wind 72daba58cd
[DS-6694][Master] Reduce the refresh resource interval of LowerWeight (#6695)
* [DS-6694][Master] Reduce the refresh resource interval of LowerWeight

* add server node update event handle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-05 19:10:36 +08:00
Jiajie Zhong 5855c936a4
[community] Add stale bot close issue/pr days (#6676)
* Add label `Waiting for reply` to bug/feature label
avoid stale bot close

* Remove bot label-on-issue

We do not need this anymore, cause when issue is created
it would auto add label `waiting for reply` for now

* Change issue/pr stale and close days.

After this PR merged, issue would be close in 37 days
and PR will be close in 127 days.
2021-11-05 18:35:41 +08:00
Jiajie Zhong a269e3df76
Revert "Change the number of approving_review_count to 2 (#6456)" (#6706)
This reverts commit 4d69685b
2021-11-05 18:31:54 +08:00
wangxj3 dc8b87e473
[Feature-#6268][server-master] Serial execte process (#6185)
* add serial processInstance

* fix bug

* add test

* fix code style

* fix style code

* add sql

* fix sql error

* add api

* add test

* fix code style

* modify api

* delete column , fix mapper

* fix unimport

* fix test

* fix bug of missing param for Python

* fix code style

* fix test

* fix test

Co-authored-by: wangxj <wangxj31>
2021-11-05 17:25:45 +08:00
Jiajie Zhong 32d4411469
[python] Add flake8 and black for code style and integrated to GA (#6679)
* [python] Add code style lint for GA

* Change github action name

* Auto change by black

* Fix flake8

* Fix broken link for pyds README.md

* Auto fix by black

* Separate GitHub workflows

* Add Black badge and CI locally in README.md
2021-11-05 15:14:28 +08:00
xuhhui 2b90d90873
improve install_config.conf description (#6681)
* improve install_config.conf description

* Update install_config.conf

update accroding review ideas
2021-11-05 15:13:23 +08:00
wind 1ebab356f3
[DS-6660][Improvement] Change the registry default timeout (#6663)
* [DS-6660][Improvement] Change the registry default timeout

* [DS-6660][Improvement] Change the registry default timeout

* fix Double to Integer

Co-authored-by: caishunfeng <534328519@qq.com>
2021-11-04 22:31:13 +08:00
Jiajie Zhong 1165afbdd1
[python] Add param workgroup to process definition (#6667) 2021-11-04 22:25:45 +08:00
LiuBodong 71047db39e
[FIX][Standalone Server] check if maven settings.xml not exists (#6691) 2021-11-04 20:17:53 +08:00
JinYong Li 66fbcae9dd
[Upgrade][Install] add upgrade 2.0 code (#6672)
* Optimizing SQL scripts

* add upgrade 2.0 ddl

* add upgrade 2.0.0 code

* fix valid license header

* fix valid license header

* fix valid license header

* fix ut

* fix code style

* fix code style
2021-11-04 17:20:23 +08:00