Commit Graph

128 Commits (dev)

Author SHA1 Message Date
xiangzihao d18cf184c6
[CI] Improve CI (#9436)
* test

* test

* test

* test

* test

* test

* test

* test

* test

* improve ci

* improve ci

* improve ci
2022-04-12 14:43:15 +08:00
kezhenxu94 14d71d1462
[UI] Migrate NPM to PNPM in CI builds (#9431) 2022-04-11 14:16:23 +08:00
Jiajie Zhong c828809b46
[doc] Change get help from dev mail list to slack (#9377)
* Change all get help from dev mailing list to slack, because
  we find out mailing list have many users ask for subscribe
  and they maybe subscribe by accident.
* remove join dev mailing list in faq.md because we already
  have it in https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html
2022-04-08 10:56:35 +08:00
xiangzihao 80ea8049e0
[CI] try to fix ci (#9366)
* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci

* try to fix ci
2022-04-06 21:57:52 +08:00
xiangzihao 40b73f7962
[Improve][CI] improve ci checking (#9325) 2022-04-06 09:33:41 +08:00
Jiajie Zhong 9f84dbbda0
[doc] Migrate dev doc from website repository (#9291)
* [doc] Migrate dev doc from website repository

* Correct release language

* Fix license issue

* Add all images from website

* Delete unused images

* Add CI

* Fix ci

* fix ci

* Remove no need file

* Add latest commit from website

* correct img_utils.py script

* Remove unused imgs
2022-04-02 10:43:34 +08:00
Jiajie Zhong 30746d9762
[python] Add integrated test to python gateway server (#8966)
* [python] Add integrated test to python gateway server

* Build java code and create standalone server image in GA
* Add component start docker in python
* Run example to make sure it work to it

close: #8035

* Fix build docker image working directory

* Fix working directory
2022-03-18 16:24:40 +08:00
xiangzihao eefe7896c8
[Feature-8850][E2E] Restore project center e2e test cases in ui-next (#8909) 2022-03-15 22:07:30 +08:00
kezhenxu94 14bb722f6f
[CI] Require E2E to passed before merging (#8891) 2022-03-15 14:35:21 +08:00
Jiajie Zhong b04a535744
[ci] Add timeout to each job avoiding zombie process (#8886) 2022-03-14 22:34:44 +08:00
kezhenxu94 d3cbf8eeaf
[UI-NEXT] Fix wrong packaging locations (#8851) 2022-03-14 15:26:15 +08:00
xiangzihao 2335453482
[Feature-8844][E2E] Restore datasource center e2e test cases in ui-next (#8845) 2022-03-13 12:33:19 +08:00
Jiajie Zhong 90f9aed11d
[python] Add tox for out of box test (#8837)
* [python] Add tox for out of box test

* Add `tox` for easy and complete test
* Correct package data attribute
* Add more dirs in flake8 ignore

* Change GA from raw to tox

* Fix env setter

* Temp skip sanity

* Rm not exists job

* Add code test

* fix

* rm all need

* fix

* fix

* fix

* fix

* fix

* fix

* Migrate pip upgrede to tox setting

* Change install_commands to commands

* fix config of install command

* Add GA env to do that

* Fix env error

* Finial commit

* remove duplicate

* Change doc

* Change dependence of ci

* Change dependence of ci
2022-03-12 21:27:36 +08:00
xiangzihao 49b1a073dd
[Feature-8823][E2E] Restore resource center e2e test cases in ui-next (#8840) 2022-03-12 20:49:46 +08:00
xiangzihao 20dd0c12ce
[E2E] Restore security center e2e test cases in ui-next (#8815) 2022-03-10 21:49:04 +08:00
kezhenxu94 d2a360480a
[CI][UI NEXT] Package new UI into release tar and remove old UI from it (#8670)
* [CI][UI NEXT] Package new UI into release tar and remove old UI from it

* Update e2e.yml
2022-03-07 11:20:24 +08:00
kezhenxu94 60a00490c6
[CI][UI Next][V1.0.0-Alpha] Add frontend UI to CI checks and fix the linting problems found (#8666) 2022-03-02 20:01:13 +08:00
wind fc9a31f813
[E2E] add switch task case (#8180)
* fix switch bug

* add switch e2e test

* add switch workflow

* pre task option

* selectPreTask id

* update

* switch task e2e

* naming optimization

Co-authored-by: caishunfeng <534328519@qq.com>
2022-02-10 10:00:50 +08:00
xiangzihao 43bd640aad
[E2E] Add hive datasource e2e test and fix s3 docker image latest tag (#8276) 2022-02-06 14:22:38 +08:00
QuakeWang 63f7efa993
[E2E] Add new test of token management (#8161) 2022-01-30 12:34:50 +08:00
xiangzihao 4b58df9aa8
[E2E] add function manage test case (#8250) 2022-01-30 09:01:19 +08:00
Jiajie Zhong f3d663a7ea
[python] Clean deps and prepare release (#8210)
* Change package name
* Migrate requirement*.txt to setup.py
* Add extra required for dev
* Add doc RELEASE and DEVELOP
* Correct description
2022-01-28 10:12:00 +08:00
xiangzihao d6690fa89c
[E2E] add udf manage test (#8203) 2022-01-27 16:17:32 +08:00
Jiajie Zhong 35578efd08
[python] Initiate document for pydolphinscheduler (#8005) 2022-01-26 12:37:57 +08:00
xiangzihao e7f480bdca
[E2E][CI] Add DataSource E2E test (#8085) 2022-01-19 17:29:02 +08:00
Jiajie Zhong a061cf136c
[e2e] Add new test of security environment (#8048)
* [e2e] Add new test of security environment

ref: #7281

* Fix frontend bug

* Fix order of UT

* Fix delete and edit bottom class

* Try to fix

* Try to fix
2022-01-15 09:13:04 +08:00
xiangzihao 1c0ad53a35
[CI][E2E] Optimize CI E2E test performance (#7974) 2022-01-14 10:31:37 +08:00
xiangzihao 5be8e0531c
[E2E] [Resource] add file manage case (#7906) 2022-01-12 14:29:03 +08:00
wind 3edc58cbdd
[E2E] Add queue e2e test (#7941) 2022-01-12 10:31:17 +08:00
janeHe13 0b38d0e826
[E2E]add worker group manage e2e case (#7879)
* add worker group manage e2e case

* add worker group manage e2e case

* Update dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/cases/WorkerGroupE2ETest.java

Co-authored-by: janeHe13 <438044805@qq.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-01-07 21:24:03 +08:00
Jiajie Zhong ab89e4335e
[ci] Remove workflow paths-ignore attribute (#7807)
It will cause our CI always waitting due to
we requests E2E contexts in .asf.yaml but
we set some ignore path on workflow e2e.
2022-01-05 14:24:12 +08:00
kezhenxu94 269a62418f
Add E2E case for user manage (#7652) 2021-12-28 14:02:06 +08:00
kezhenxu94 120c1755a1
Required E2E to pass before PR can be merged (#7612) 2021-12-24 17:02:42 +08:00
kezhenxu94 b350dc70c9
Build and publish snapshot Docker images for testing (#7454) 2021-12-17 00:26:09 +08:00
kezhenxu94 67cc260d52
Split the components into individual package, refactor configurations to use Spring properties (#7160)
* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.
2021-12-10 11:28:53 +08:00
kezhenxu94 37ba1eb5ad
Add E2E tests for some core features (#7025) 2021-11-30 13:07:55 +08:00
Jiajie Zhong 7c5c7ec907
[ci][python] Add isort to sort out import (#6871)
* Add isort config file to fix conflict with black
* Add some doc about isort
2021-11-17 11:48:52 +08:00
Jiajie Zhong 54933b33e3
[ci][python] Add coverage check in CI (#6861)
* [ci] Add coverage check in CI

* Coverage add dependent

* Install pydolphinscheduler before run coverage

* Up test coverage to 87% and down threshold to 85%

* Fix code style

* Add doc about coverage
2021-11-17 09:46:40 +08:00
Jiajie Zhong c110488269
[ci] Remove GA path rule in build and unittest (#6797)
CI could not be run due to we add 9113817b to asf
configure and same file change will not trigger
build or unit test github actions run. This patch
try to skip github action path ignore rule temporarily

Maybe we should find more effective way to spearate py
, js, java CI later
2021-11-11 17:32:57 +08:00
kezhenxu94 02350ea37b
Fix datasource not work in tests and standalone server, and run all tests by default (#6743) 2021-11-09 16:57:26 +08:00
Jiajie Zhong 5855c936a4
[community] Add stale bot close issue/pr days (#6676)
* Add label `Waiting for reply` to bug/feature label
avoid stale bot close

* Remove bot label-on-issue

We do not need this anymore, cause when issue is created
it would auto add label `waiting for reply` for now

* Change issue/pr stale and close days.

After this PR merged, issue would be close in 37 days
and PR will be close in 127 days.
2021-11-05 18:35:41 +08:00
Jiajie Zhong 32d4411469
[python] Add flake8 and black for code style and integrated to GA (#6679)
* [python] Add code style lint for GA

* Change github action name

* Auto change by black

* Fix flake8

* Fix broken link for pyds README.md

* Auto fix by black

* Separate GitHub workflows

* Add Black badge and CI locally in README.md
2021-11-05 15:14:28 +08:00
Jiajie Zhong 2e7036529c
Add CI test for python API (#6636)
* Add CI for python API

* Fix task assign without process definition

* Fix ci

* Add header

* Add build badge for pydolphinscheduler

* Add content to README.md

* Remove unnecessary blank line
2021-11-04 11:19:11 +08:00
Jiajie Zhong 26a84b30d8
Change issue bot mailing list from chinese to english (#6420) 2021-11-02 21:00:23 +08:00
Jiajie Zhong 03aaef0fe6
Add Github Actions stale bot (#6468) 2021-10-28 13:43:02 +08:00
kezhenxu94 16b94e207d
Update e2e.yml (#6574) 2021-10-21 15:11:50 +08:00
kezhenxu94 251255009a
Add end-to-end test framework and some basic cases (#6419) 2021-10-20 21:14:26 +08:00
kezhenxu94 cfb03ce8ee
Reorganize CI workflows to fasten the wasted time and resources (#6011) 2021-08-22 07:01:42 +08:00
深刻 0b4a20251f
ci: improve maven connection in CI builds (#5924)
issue: #5921
2021-08-01 22:18:32 +08:00
深刻 4c0993cdf4
[Fix-5781][UT] Fix test coverage in sonar (#5817)
* build(UT): make jacoco running in offline-instrumentation

issue: #5781

* build(UT): remove the jacoco agent dependency in microbench

issue: #5781
2021-07-26 22:04:36 +08:00