Unverified Commit a4d287d2 authored by Hugo Häggmark's avatar Hugo Häggmark Committed by GitHub
Browse files

Tests: Adds end-to-end tests skeleton and basic smoke test scenario (#16901)

* Chore: Adds neccessary packages

* Wip: Initial dummy test in place

* Feature: Downloads Chromium if needed

* Fix: Adds global config object

* Refactor: Adds basic e2eScenario

* Build: Adds end to end tests to config

* Build: Changes end to end job

* Build: Adds browsers to image

* Build: Adds failing test

* Refactor: Adds first e2e-test scenario

* Fix: Ignores test output in gitignore

* Refactor: Adds compare screenshots ability

* Refactor: Removes unnecessary code

* Build: Removes jest-puppeteer

* Fix: Replaces test snapshots

* Refactor: Creates output dir if missing

* Refactor: Changes aria-labels to be more consistent

* Docs: Adds section about end to end tests

* Fix: Fixes snapshots

* Docs: Adds information about ENV variables
parent ceb9f085
......@@ -69,6 +69,28 @@ jobs:
- run:
name: cache server tests
command: './scripts/circle-test-cache-servers.sh'
end-to-end-test:
docker:
- image: circleci/node:8-browsers
- image: grafana/grafana:master
steps:
- run: dockerize -wait tcp://127.0.0.1:3000 -timeout 120s
- checkout
- restore_cache:
key: dependency-cache-{{ checksum "yarn.lock" }}
- run:
name: yarn install
command: 'yarn install --pure-lockfile --no-progress'
no_output_timeout: 5m
- save_cache:
key: dependency-cache-{{ checksum "yarn.lock" }}
paths:
- node_modules
- run:
name: run end-to-end tests
command: 'env BASE_URL=http://127.0.0.1:3000 yarn e2e-tests'
no_output_timeout: 5m
codespell:
docker:
......
......@@ -84,3 +84,5 @@ debug.test
/packages/**/dist
/packages/**/compiled
/packages/**/.rpt2_cache
theOutput/
\ No newline at end of file
[Grafana](https://grafana.com) [![Circle CI](https://circleci.com/gh/grafana/grafana.svg?style=svg)](https://circleci.com/gh/grafana/grafana) [![Go Report Card](https://goreportcard.com/badge/github.com/grafana/grafana)](https://goreportcard.com/report/github.com/grafana/grafana) [![codecov](https://codecov.io/gh/grafana/grafana/branch/master/graph/badge.svg)](https://codecov.io/gh/grafana/grafana)
================
# [Grafana](https://grafana.com) [![Circle CI](https://circleci.com/gh/grafana/grafana.svg?style=svg)](https://circleci.com/gh/grafana/grafana) [![Go Report Card](https://goreportcard.com/badge/github.com/grafana/grafana)](https://goreportcard.com/report/github.com/grafana/grafana) [![codecov](https://codecov.io/gh/grafana/grafana/branch/master/graph/badge.svg)](https://codecov.io/gh/grafana/grafana)
[Website](https://grafana.com) |
[Twitter](https://twitter.com/grafana) |
[Community & Forum](https://community.grafana.com)
......@@ -12,12 +12,15 @@ Graphite, Elasticsearch, OpenTSDB, Prometheus and InfluxDB.
-->
## Installation
Head to [docs.grafana.org](http://docs.grafana.org/installation/) for documentation or [download](https://grafana.com/get) to get the latest release.
## Documentation & Support
Be sure to read the [getting started guide](http://docs.grafana.org/guides/gettingstarted/) and the other feature guides.
## Run from master
If you want to build a package yourself, or contribute - here is a guide for how to do that. You can always find
the latest master builds [here](https://grafana.com/grafana/download)
......@@ -48,7 +51,7 @@ go run build.go build
#### Frontend assets
*For this you need Node.js (LTS version).*
_For this you need Node.js (LTS version)._
```bash
yarn install --pure-lockfile
......@@ -80,7 +83,7 @@ yarn start:hot
env GRAFANA_THEME=light yarn start:hot
```
*Note: HMR for Angular is not supported. If you edit files in the Angular part of the app, the whole page will reload.*
_Note: HMR for Angular is not supported. If you edit files in the Angular part of the app, the whole page will reload._
Run tests and rebuild on source change:
......@@ -128,7 +131,9 @@ In your custom.ini uncomment (remove the leading `;`) sign. And set `app_mode =
### Running tests
#### Frontend
Execute all frontend tests
```bash
yarn test
```
......@@ -139,6 +144,7 @@ Writing & watching frontend tests
- Jest will run all test files that end with the name ".test.ts"
#### Backend
```bash
# Run Golang tests using sqlite3 as database (default)
go test ./pkg/...
......@@ -150,6 +156,26 @@ GRAFANA_TEST_DB=mysql go test ./pkg/...
GRAFANA_TEST_DB=postgres go test ./pkg/...
```
#### End-to-end
Execute all end-to-end tests
```bash
yarn e2e-tests
```
Execute all end-to-end tests using using a specific url
```bash
ENV BASE_URL=http://localhost:3333 yarn e2e-tests
```
Debugging all end-to-end tests (BROWSER=1 will start the browser and SLOWMO=1 will delay each puppeteer operation by 100ms)
```bash
ENV BROWSER=1 SLOWMO=1 yarn e2e-tests
```
### Datasource and dashboard provisioning
[Here](https://github.com/grafana/grafana/tree/master/devenv) you can find helpful scripts and docker-compose setup
......@@ -171,4 +197,3 @@ plugin development.
## License
Grafana is distributed under [Apache 2.0 License](https://github.com/grafana/grafana/blob/master/LICENSE).
require('module-alias/register');
module.exports = {
verbose: false,
transform: {
'^.+\\.(ts|tsx)$': 'ts-jest',
},
moduleDirectories: ['node_modules', 'public'],
roots: ['<rootDir>/public/e2e-test'],
testRegex: '(\\.|/)(test)\\.(jsx?|tsx?)$',
moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json'],
setupFiles: [],
globals: { 'ts-jest': { isolatedModules: true } },
setupFilesAfterEnv: ['expect-puppeteer', '<rootDir>/public/e2e-test/install/install.ts'],
};
......@@ -25,12 +25,16 @@
"@types/commander": "2.12.2",
"@types/d3": "4.13.1",
"@types/enzyme": "3.9.0",
"@types/expect-puppeteer": "3.3.1",
"@types/inquirer": "0.0.43",
"@types/jest": "24.0.11",
"@types/jquery": "1.10.35",
"@types/lodash": "4.14.123",
"@types/node": "11.13.4",
"@types/papaparse": "4.5.9",
"@types/pixelmatch": "4.0.0",
"@types/pngjs": "3.3.2",
"@types/puppeteer-core": "1.9.0",
"@types/react": "16.8.16",
"@types/react-dom": "16.8.4",
"@types/react-grid-layout": "0.16.7",
......@@ -55,6 +59,7 @@
"es6-promise": "3.3.1",
"es6-shim": "0.35.5",
"execa": "1.0.0",
"expect-puppeteer": "4.1.1",
"expect.js": "0.2.0",
"expose-loader": "0.7.5",
"file-loader": "3.0.1",
......@@ -85,6 +90,7 @@
"load-grunt-tasks": "3.5.2",
"mini-css-extract-plugin": "0.5.0",
"mocha": "4.1.0",
"module-alias": "2.2.0",
"monaco-editor": "0.15.6",
"ng-annotate-loader": "0.6.1",
"ng-annotate-webpack-plugin": "0.3.0",
......@@ -94,10 +100,13 @@
"optimize-css-assets-webpack-plugin": "5.0.1",
"ora": "3.2.0",
"phantomjs-prebuilt": "2.1.16",
"pixelmatch": "4.0.2",
"pngjs": "3.4.0",
"postcss-browser-reporter": "0.5.0",
"postcss-loader": "3.0.0",
"postcss-reporter": "6.0.1",
"prettier": "1.16.4",
"puppeteer-core": "1.15.0",
"react-hooks-testing-library": "0.3.7",
"react-hot-loader": "4.8.0",
"react-test-renderer": "16.8.4",
......@@ -140,6 +149,7 @@
"tslint": "tslint -c tslint.json --project tsconfig.json",
"typecheck": "tsc --noEmit",
"jest": "jest --notify --watch",
"e2e-tests": "jest --runInBand --config=jest.config.e2e.js",
"api-tests": "jest --notify --watch --config=tests/api/jest.js",
"storybook": "cd packages/grafana-ui && yarn storybook",
"storybook:build": "cd packages/grafana-ui && yarn storybook:build",
......@@ -242,5 +252,8 @@
"**/@types/*",
"**/@types/*/**"
]
},
"_moduleAliases": {
"puppeteer": "node_modules/puppeteer-core"
}
}
......@@ -20,7 +20,7 @@
<div class="search-section__header" ng-show="section.hideHeader"></div>
<div ng-if="section.expanded">
<a ng-repeat="item in section.items" class="search-item search-item--indent" ng-class="{'selected': item.selected}" ng-href="{{::item.url}}" >
<a ng-repeat="item in section.items" class="search-item search-item--indent" ng-class="{'selected': item.selected}" ng-href="{{::item.url}}" aria-label="{{::item.title}}">
<div ng-click="ctrl.toggleSelection(item, $event)" class="center-vh">
<gf-form-checkbox
ng-show="ctrl.editable"
......
......@@ -132,10 +132,15 @@ export class AddPanelWidget extends React.Component<Props, State> {
dashboard.removePanel(this.props.panel);
};
renderOptionLink = (icon, text, onClick) => {
renderOptionLink = (icon: string, text: string, onClick) => {
return (
<div>
<a href="#" onClick={onClick} className="add-panel-widget__link btn btn-inverse">
<a
href="#"
onClick={onClick}
className="add-panel-widget__link btn btn-inverse"
aria-label={`${text} CTA button`}
>
<div className="add-panel-widget__icon">
<i className={`gicon gicon-${icon}`} />
</div>
......
......@@ -35,6 +35,7 @@ exports[`Render should render component 1`] = `
>
<div>
<a
aria-label="Add Query CTA button"
className="add-panel-widget__link btn btn-inverse"
href="#"
onClick={[Function]}
......@@ -53,6 +54,7 @@ exports[`Render should render component 1`] = `
</div>
<div>
<a
aria-label="Choose Visualization CTA button"
className="add-panel-widget__link btn btn-inverse"
href="#"
onClick={[Function]}
......
......@@ -16,7 +16,11 @@ export const DashNavButton: FunctionComponent<Props> = ({ icon, tooltip, classSu
if (onClick) {
return (
<Tooltip content={tooltip}>
<button className={`btn navbar-button navbar-button--${classSuffix}`} onClick={onClick}>
<button
className={`btn navbar-button navbar-button--${classSuffix}`}
onClick={onClick}
aria-label={`${tooltip} navbar button`}
>
<i className={icon} />
</button>
</Tooltip>
......
......@@ -17,7 +17,7 @@ const template = `
<div class="p-t-2">
<div class="gf-form">
<label class="gf-form-label width-8">New name</label>
<input type="text" class="gf-form-input" ng-model="ctrl.clone.title" give-focus="true" required>
<input type="text" class="gf-form-input" ng-model="ctrl.clone.title" give-focus="true" required aria-label="Save dashboard title field">
</div>
<folder-picker initial-folder-id="ctrl.folderId"
on-change="ctrl.onFolderChange($folder)"
......@@ -34,7 +34,14 @@ const template = `
</div>
<div class="gf-form-button-row text-center">
<button type="submit" class="btn btn-primary" ng-click="ctrl.save()" ng-disabled="!ctrl.isValidFolderSelection">Save</button>
<button
type="submit"
class="btn btn-primary"
ng-click="ctrl.save()"
ng-disabled="!ctrl.isValidFolderSelection"
aria-label="Save dashboard button">
Save
</button>
<a class="btn-text" ng-click="ctrl.dismiss();">Cancel</a>
</div>
</form>
......
......@@ -92,7 +92,7 @@
</div>
</div>
<div class="gf-form" ng-show="modeSharePanel">
<a href="{{imageUrl}}" target="_blank"><i class="fa fa-camera"></i> Direct link rendered image</a>
<a href="{{imageUrl}}" target="_blank" aria-label="Link to rendered image"><i class="fa fa-camera"></i> Direct link rendered image</a>
</div>
</div>
</script>
......
......@@ -90,7 +90,12 @@ export class PanelHeader extends Component<Props, State> {
error={error}
/>
<div className={panelHeaderClass}>
<div className="panel-title-container" onClick={this.onMenuToggle} onMouseDown={this.onMouseDown}>
<div
className="panel-title-container"
onClick={this.onMenuToggle}
onMouseDown={this.onMouseDown}
aria-label="Panel Title"
>
<div className="panel-title">
<span className="icon-gf panel-alert-icon" />
<span className="panel-title-text">
......
......@@ -14,7 +14,9 @@ export const PanelHeaderMenuItem: FC<Props & PanelMenuItem> = props => {
<li className={isSubMenu ? 'dropdown-submenu' : null}>
<a onClick={props.onClick}>
{props.iconClassName && <i className={props.iconClassName} />}
<span className="dropdown-item-text">{props.text}</span>
<span className="dropdown-item-text" aria-label={`${props.text} panel menu item`}>
{props.text}
</span>
{props.shortcut && <span className="dropdown-menu-item-shortcut">{props.shortcut}</span>}
</a>
{props.children}
......
......@@ -145,7 +145,7 @@ function TabItem({ tab, activeTab, onClick }: TabItemParams) {
return (
<div className="panel-editor-tabs__item" onClick={() => onClick(tab)}>
<a className={tabClasses}>
<a className={tabClasses} aria-label={`${tab.text} tab button`}>
<Tooltip content={`${tab.text}`} placement="auto">
<i className={`gicon gicon-${tab.id}${activeTab === tab.id ? '-active' : ''}`} />
</Tooltip>
......
......@@ -54,6 +54,7 @@ class NewDataSourcePage extends PureComponent<Props> {
onClick={() => this.onDataSourceTypeClicked(plugin)}
className="add-data-source-grid-item"
key={`${plugin.id}-${index}`}
aria-label={`${plugin.name} datasource plugin`}
>
<img className="add-data-source-grid-item-logo" src={plugin.info.logos.small} />
<span className="add-data-source-grid-item-text">{plugin.name}</span>
......
......@@ -12,7 +12,13 @@ const ButtonRow: FC<Props> = ({ isReadOnly, onDelete, onSubmit, onTest }) => {
return (
<div className="gf-form-button-row">
{!isReadOnly && (
<button type="submit" className="btn btn-primary" disabled={isReadOnly} onClick={event => onSubmit(event)}>
<button
type="submit"
className="btn btn-primary"
disabled={isReadOnly}
onClick={event => onSubmit(event)}
aria-label="Save and Test button"
>
Save &amp; Test
</button>
)}
......
......@@ -212,7 +212,7 @@ export class DataSourceSettingsPage extends PureComponent<Props, State> {
<div className="gf-form-group">
{testingMessage && (
<div className={`alert-${testingStatus} alert`}>
<div className={`alert-${testingStatus} alert`} aria-label="Datasource settings page Alert">
<div className="alert-icon">
{testingStatus === 'error' ? (
<i className="fa fa-exclamation-triangle" />
......@@ -221,7 +221,9 @@ export class DataSourceSettingsPage extends PureComponent<Props, State> {
)}
</div>
<div className="alert-body">
<div className="alert-title">{testingMessage}</div>
<div className="alert-title" aria-label="Datasource settings page Alert message">
{testingMessage}
</div>
</div>
</div>
)}
......
......@@ -33,6 +33,7 @@ exports[`Render should render with buttons enabled 1`] = `
className="gf-form-button-row"
>
<button
aria-label="Save and Test button"
className="btn btn-primary"
disabled={false}
onClick={[Function]}
......
......@@ -17,7 +17,7 @@ const panelTemplate = `
<i class="fa fa-spinner fa-spin"></i>
</span>
<panel-header class="panel-title-container" panel-ctrl="ctrl"></panel-header>
<panel-header class="panel-title-container" panel-ctrl="ctrl" aria-label="Panel Title"></panel-header>
</div>
<div class="panel-content">
......
......@@ -34,7 +34,7 @@ function renderMenuItem(item, ctrl) {
}
html += `><i class="${item.icon}"></i>`;
html += `<span class="dropdown-item-text">${item.text}</span>`;
html += `<span class="dropdown-item-text" aria-label="${item.text} panel menu item">${item.text}</span>`;
if (item.shortcut) {
html += `<span class="dropdown-menu-item-shortcut">${item.shortcut}</span>`;
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment