From 0074fee909a625a23163266edfe424ed5f77ccdd Mon Sep 17 00:00:00 2001 From: Mart van Santen Date: Fri, 1 Apr 2022 17:25:39 +0800 Subject: [PATCH] Added documentation --- DEVELOPMENT.md | 293 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 293 insertions(+) create mode 100644 DEVELOPMENT.md diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 0000000..feedda6 --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,293 @@ +# Development + +The main role for this repo is provide Single-Sign-On. The architecture to make +this happen has a lot of moving components. A quick overview: + + - Hydra: Hydra is an Identity Provider, or IdP for short. It means connected + applications connect to Hydra to start a session with a user. Hydra provides + the application with the username and other roles/claims for the application. + This is done using the OIDC protocol. Hydra is developed by Ory and has + security as one of their top priorities. Also it is fully OpenSource. + + - Login application: If hydra hits a new session/user, it has to know if this + user has access. To do so, the user has to login. Hydra does not support + this, so it will redirect to a login application. This is developed by the + Stackspin team (Greenhost) and part of this repository. It is a Python Flask + application. + Because the security decisions made by kratos (see below), a lot of the + interaction is done in the web-browser, rather then server-side. + This means the login application has an UI component which relies heavily on + JavaScript. As this is a relatively small application, it is based on + traditional Bootstrap + Jquery. This elements the requirement for yet an + other build environment. + + - Kratos: This is Identity Manager and contains all the user profiles and + secrets (passwords). Kratos is designed to work mostly between UI (browser) + and kratos directly, over a public API endpoint without an extra server side + component/application. So authentication, form-validation etc, are all handled + by Kratos. Kratos only provides an API and not UI itself. + Kratos provides a Admin API, which is only used from the server-side flask + app to create/delete users. + + - MariaDB: All three components need to store data. This is done in a MariaDB + database server. There is once instance, with three databases. As all + databases are very small this will not lead to resource limitation problems. + +## Prerequisites + +The current login panel is not yet installed available in released versions +of Stackspin. However, this does not prevent us from developing already on the +login panel. Experience with `helm` and `kubernetes` is expected when you follow +this manual. + +On your provisioning machine, make sure to checkout: + +`git@open.greenhost.net:stackspin/dashboard-backend.git` + +Be sure to check out the latest main branch. Or select a more modern branch if you +want to test / install (optional) improvements of login panel. + +Once this is all fetched, installation can be done with the following steps: + +1. Create an overwrite ConfigMap file: + + For local development, we have to configure the endpoint of the application to + be pointing to our development system. In this example, we use `localhost` on + http. + + Because of CORS and strict configuration, all needs to end up on the same + system. With modern browser, it even have to run on the same port (at least with + firefox). As we want to mimic the real life setup as much as possible as, + we will do this by running a local proxy. In production this will be handled by + kubernetes ingress configuration. + + First we will tell kratos and hydra where to find the right endpoints. An + overview of all relevant end-points: + + The endpoints used by the browser are (public accessible) + + - `localhost/kratos` -> kratos public API + - `localhost/web` -> login flask app + + The endpoint used by the login app/API are: + - `localhost:8000` -> kratos Admin API (only local accessible) + - `localhost/kratos` -> kratos Public API + - `localhost:4445` -> hydra Admin API (only local accessible) + - `localhost:3306` -> MariaDB + + To reflect those public endpoints in your cluster, we have to override the + default URLs in the cluster. We do this with a ConfigMap. + + It is essential SMTP/e-mail is working during development, so an example + is included on how to override those if SMTP is not working on your + cluster. Otherwise those lines are irrelevant. + + Create a file with the following content: + +``` +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: stackspin-dashboard-override +data: + values.yaml: | + kratos: + kratos: + config: + courier: + smtp: + # Kratos enforces the use of STARTTLS. Be sure your SMTP provider + # supports that (if not, it is time to switch providers) + # + # Uncomment and correct below lines if e-mail is not working in your + # cluster + # connection_uri: smtp://user@password@smtp.example.com:25/ + # from_address: stackspin-admin@example.com + + # For development, we forward all to our local server (or your dev server + # if that is remote) + serve: + public: + base_url: http://localhost/kratos/ + + selfservice: + default_browser_return_url: http://localhost/web/login + + flows: + recovery: + ui_url: http://localhost/web/recovery + + login: + ui_url: http://localhost/web/login + + settings: + ui_url: http://localhost/web/settings + + registration: + ui_url: http://localhost/web/registration + + hydra: + hydra: + config: + urls: + # For development we redirect to localhost (or your dev server) + login: http://localhost/web/auth + consent: http://localhost/web/consent + logout: http://localhost/web/logout +``` + +2. Apply the ConfigMap to your cluster: + + ``` + kubectl apply -n stackspin -f stackspin-dashboard-override.yaml + ``` + +3. Tell flux to reconcile the configuration + + Normally flux will do this on some interval. We will tell flux to apply + the override immediately. + + ``` + flux reconcile kustomization core + flux reconcile helmrelease -n stackspin dashboard + ``` + +## Setting up the development environment + +1. Setup port redirects + +To be able to work on the Login panel, we have to configure our development +system to access all the remote services and endpoints. + +A helper script is available in this directory to setup and redirect the +relevant ports locally. It will open ports 8000, 8080, 4445, 5432 to get access +to all APIs: + +``` +cd project_root/login +./set-ssh-tunnel.sh "stackspin.example.com" +``` + +(the tunnel goes to the kubernetes node, so *not* to your provisioning machine, + it will uses SSH port forwarding to map ports, as a result you will also have + SSH session to your kubernetes node. Do not close this session, as closing the + session will close the forwarded ports as well) + +2. Configure a local proxy + +Because of strict CORS headers, we have to map the public kratos API and login +app which we will run locally, with a local proxy. + +This can be done with any proxy server, for example with NGINX. Be sure you have +NGINX installed and listening on port 80 locally (`sudo apt-get install nginx`) +should be enough. + +Now configure NGINX with this configuration in `/etc/nginx/sites-enabled/default` + +``` + +server { + listen 80 default_server; + listen [::]:80 default_server; + + root /var/www/html; + + index index.html; + + server_name _; + + # Flask app + location / { + proxy_pass http://127.0.0.1:5000/; + proxy_redirect default; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } + + # Kratos Public + location /kratos/ { + proxy_pass http://127.0.0.1:8080/; + proxy_redirect default; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +} +``` + +Reload your NGINX: + +``` +sudo systemctl reload nginx.service +``` + +3. Run FLASK app + +Now it is time to start the flask app. Please make sure you are using python 3 in your environment. And install the required dependencies: + +``` +cd projectroot/login +pip3 install -r requirements.txt +``` + +Then copy `source_env` to `source_env.local` and verify if you are happy with +the settings in the `source_env` file: + +``` +cat source_env.local + +export HYDRA_ADMIN_URL=http://localhost:4445 +export KRATOS_PUBLIC_URL=http://localhost/api +export KRATOS_ADMIN_URL=http://localhost:8000 +export PUBLIC_URL=http://localhost/login +export DATABASE_URL="mysql+pymysql://stackspin:stackspin@localhost/stackspin" +``` + +Normally you only need to change the database password if you did not use the +insecure default. + +Assuming you did not populate the database yet, run this to populate it: + +``` +. source_env.local +flask db upgrade +``` + +If that all looks fine, it is time to add you first user: + +``` +flask cli user create myemail@example.com +``` + +And now it is time to start the app: + +``` +./run.sh +``` + +If this starts smoothly, you should be ready to go. + +## Test your setup + +Hydra and kratos are now configured to redirect to localhost when they receive a +request. So to test the setup, you can go to one of your applications (for +example nextcloud), what we expect when you click the login button is the +following: + +- Nextcloud redirect to Hydra (on sso.example.com) +- Hydra does not have a session, so ask to authorize on: http://localhost/login/auth +- Kratos does not have a session, so the login panel will ask to login on: + http://localhost/login/login +- You do not have a password setup yet, so you click "recover account", which + should bring you to: http://localhost/login/recovery +- You enter your email address and request a reset token. Check you e-mail. The + email should contain a link to http://localhost/api/self-service/recovery/.. +- The link logs you in in kratos and ask you to setup a password. Complete this + step and you account is ready. + +We started the flow with trying to reach nextcloud. Because we +did a password recovery in between, this information is lost. If you go again to +nextcloud manually, you should now be logged in automatically. + +If you retry this, but now with a password (for example in a privacy window or +by removing you cookies), you should be redirected automatically after login. + +