Commit Graph

5542 Commits (f577b5bd30ea86df3cd07af04cef358db52d0f29)

Author SHA1 Message Date
zwZjut f577b5bd30
to #7228 (#7249)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-07 20:05:57 +08:00
Jiajie Zhong 29414a080f
Fix homepage user wrong statistics result for new user (#7242)
* Fix homepage user wrong statistics result for new user

For now, when we create a new user, it would see homepage
statistic info as admin user like, cause `projectCodeArray`
would not filter when we render SQL. So I add prefix conditions
to handle this situation.

Also, I do some minimal refactor here. including change function
name no quit fit

fix: #7182

* Fix UT
2021-12-07 18:50:06 +08:00
Zonglei Dong 0b9792db76
[Refactor]Rename class file, DatasourceUtil to DataSourceUtils. (#7181)
* Rename class file, DatasourceUtil  to  DataSourceUtils.
2021-12-07 18:49:05 +08:00
wind a80a7b3fcf
[Feature-6963][MasterServer] unified cache manager (#7187)
* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 15:55:22 +08:00
JinYong Li 544b015f76
[Bug] [API] task definition release/delete bug (#7232)
* fix worker group display

* fix workflow query bug

* fix workflow query bug

* fix task definition release/delete bug
2021-12-07 15:26:28 +08:00
JinYong Li 2792c35d68
[Bug] [API] queryProcessDefinitionByCode bug (#7221) 2021-12-07 14:41:26 +08:00
wind 9d65bf1452
fix postgre upgrade (#7230)
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-07 14:19:32 +08:00
Hua Jiang 4743e8f891
add two files (#7223) 2021-12-06 22:55:28 +08:00
wind ff9bc806ac
[Bug-7206] [MasterServer] fix process isntance always running when task timeout (#7207)
* fix timeout

* add task timeout map to avoid repeated timeout event

* split task check list for retry and timeout

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-06 18:20:56 +08:00
Hua Jiang 705ba74f52
[Fix-7195] The file path of 2.0.1-schema used a wrong separator. (#7200) 2021-12-06 17:29:11 +08:00
JinYong Li c7b06b0079
fix worker group display (#7208) 2021-12-06 15:44:28 +08:00
Kerwin 36ebfe0984
[Implement][server]TaskProcessor code optimization (#7146)
* TaskProcessor code optimization

* fix code style

* update Collectors.toMap
2021-12-06 14:15:09 +08:00
Kerwin 8f77fbbeb8
[Fix-7154][API] Fix MultipartFile resource [file] cannot be resolved to absolute file… (#7155)
* fix MultipartFile resource [file] cannot be resolved to absolute file path

* update testCopyFile.
2021-12-06 14:13:53 +08:00
ouyangyewei 3c8d1c6da9
to feature #7191 (#7194)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-06 07:17:33 +08:00
zwZjut 05aef279c6
[Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153 (#7192)
* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-05 23:57:46 +08:00
SbloodyS d83735ab51
[Fix-7037][dolphincheduler-server] process instance can not be killed when complement data (#7140)
* fix bug 7037

* update processComplementData

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-05 15:03:52 +08:00
ouyangyewei 3361d763c4
to feature #7180 (#7185)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-05 14:13:23 +08:00
Manhua 2dc09c627f
[Fix-5551][common] Fix ExecutionStatus for hive-on-Tez application (#6289)
* add ENDED state

* Update HadoopUtils.java
2021-12-05 14:03:29 +08:00
SbloodyS 8d39bf14b1
[Fix-7093][dolphincheduler-server] fix dependent node stuck in running (#7116)
* fix bug 7093

* remove redundant else

Co-authored-by: SbloodyS <sbloodys@qq.com>
2021-12-05 13:52:48 +08:00
songjianet eb07a2b1c9
[Fix][Warning instance manage] Expand the support of `i18n`. (#7184) 2021-12-05 11:23:50 +08:00
songjianet 2fa3157e6e
[Feature][Workflow] Main package. (#7177) 2021-12-04 21:54:59 +08:00
ouyangyewei 27ea43d5df
to feature #7110 (#7111)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 21:43:14 +08:00
Zonglei Dong fa8ccd3855
Refactor code, add class import, keep ‘logger’ variable names uniform. (#7164) 2021-12-04 21:18:43 +08:00
ouyangyewei db090685a8
to feature #7169 (#7170)
Co-authored-by: ouyangyewei <yewei.oyyw@alibaba-inc.com>
2021-12-04 19:24:09 +08:00
zwZjut af9ed35295
[Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)
* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @Param -> @Name

* to #6987: fix Sonar

* to #6987: fix Sonar

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-04 16:59:05 +08:00
BaoLiang 0e75bd8008
[BUG][SQL] fix the process list page load slowly. (#7166)
* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1
2021-12-04 15:53:18 +08:00
GaoTianDuo ed9fca6c97
[Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)
1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.
2021-12-04 13:13:30 +08:00
wind ba2b2a67c2
[Feature-6988][MasterServer] add cache manager for workflow (#7090)
* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-03 23:13:11 +08:00
zwZjut 1f184440ca
[UT#7157]Fix Ut Run Error (#7158)
* to #7157

* to #7157

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 22:01:26 +08:00
Hua Jiang 226bed1768
fix this issue (#7120) 2021-12-03 16:56:30 +08:00
zwZjut 2bf73603bb
[Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081 (#7082)
* to #7081

* fix #7081

* to #7081

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:56:08 +08:00
Kerwin 3de182dc08
Optimize the display of the pop-up window for creating worker groups (#7148) 2021-12-03 16:55:58 +08:00
zwZjut 8ebe060658
[Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project #7073 (#7080)
* to #7073

* to #7073

* to #7073

Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:55:20 +08:00
zwZjut d09d68e109
to #7065: fix ExecutorService and schedulerService (#7072)
Co-authored-by: honghuo.zw <honghuo.zw@alibaba-inc.com>
2021-12-03 16:54:39 +08:00
songjianet 99f6ab3847
[Feature][Workflow relationship] Format name. (#7143) 2021-12-03 14:43:45 +08:00
wangxj3 2b519f31e9
[Feature-#6422] task group queue (#6722)
* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>
2021-12-03 14:02:31 +08:00
Devosend f480b8adf8
[Feature-6930] [Python]Add workflow as code task type sub_process (#7022)
* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task
2021-12-03 10:26:45 +08:00
songjianet 3304edee47
[Feature][Workflow relationship] Format time. (#7123) 2021-12-03 10:02:33 +08:00
songjianet 791ed6de77
[Feature][Workflow relationship] Improve English support. (#7129) 2021-12-03 01:05:45 +08:00
Hua Jiang 6beae191e2
[Fix-7103] Fix the issue with the sqoop task missing the verification of props. (#7119)
* add the verification of parameters for the sqoop task
2021-12-02 19:33:51 +08:00
Kerwin 9fffbdb316
add TaskProcessorFactoryTest Ignore. (#7130) 2021-12-02 19:15:57 +08:00
zwZjut c42531e39c
to #7108 (#7109) 2021-12-02 16:34:38 +08:00
wangyizhi e0623973e0
[Fix-7083] [UI] The switch task is missing the else branch (#7084) 2021-12-02 14:35:36 +08:00
JinYong Li b265297fc8
[Improvement][API] fix 6772 new api bug (#7098)
* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle
2021-12-02 14:06:39 +08:00
Hua Jiang d7eb7453e5
[Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (#7107) 2021-12-01 21:03:49 +08:00
wangyizhi 173a385618
[Improvement-7092][UI] Optimize DAG to adapt to situations without locations (#7102) 2021-12-01 16:40:11 +08:00
songjianet b3efcdfeba
[Fix][Create Warning Instance Edit] Fix the problem of incomplete display of the edit form. (#7096) 2021-12-01 14:41:14 +08:00
张俊杰 463fc76bb7
[Fix-6809][dolphinscheduler-task-sql] Fix sql task can't send alert (#7089) 2021-12-01 14:08:40 +08:00
kezhenxu94 2ed3c0c3a7
Recover unit tests that have not been run for some time (#7054) 2021-12-01 13:55:26 +08:00
Sunny Lei 0dcff1425a
[DS-5229][feat] Implement server port custom config (#6947)
* [DS-5229][fix] server port custom config
This closes #5229

* [DS-5229][feat] Implement server port custom config
This closes #5229

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802 (#6862)

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link in readme_zh_cn.md

* [DS-6829][WorkerServer] skip create log dir and print log in dryRun model (#6852)

Co-authored-by: caishunfeng <534328519@qq.com>

* [DS-5229][feat] Implement server port custom config
Modify review suggestion
This closes #5229

Co-authored-by: GaoTianDuo <gaotianduo_yewu@cmss.chinamobile.com>
Co-authored-by: wind <caishunfeng@users.noreply.github.com>
Co-authored-by: caishunfeng <534328519@qq.com>
2021-12-01 00:21:03 +08:00