Commit 5866455e authored by Carlos Torres's avatar Carlos Torres

Fixing the download urls.

parent 05242aa2
Pipeline #9485 passed with stage
in 1 minute and 4 seconds
......@@ -583,7 +583,7 @@ Sets a specific CustomerViewsetVersion as active viewset.
## Group ScPackVersion
This section groups opt\_pipeline Shimmercat pack versions.
### ScPackVersion Download [/presentation-api/v1/download/sc_pack/]
### ScPackVersion Download [/presentation-api/v1/upload/sc_pack_v2/
#### Retrieve ScPackVersion [GET]
Returns the latest ScPackVersion
......@@ -593,11 +593,20 @@ Returns the latest ScPackVersion
{}
+ Response 200 (application/json)
```
{
"version": "0.1.9",
"download_url": "https://accelerator.shimmercat.com/media/sc_pack_versions/sc_pack_test-0.1.9-py3-none-any.whl"
}
{
"bootstrapper_download_url": "https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-2020.1.bootstrap",
"versions": { "bootstrapper": "2020.1", "sc_logs_agent": "develop-rev-324", "shimmercat": "qs-rev-3623", "base_python": "2020.6", "usher3": "master-rev-116", "sc_pack": "v1533" },
"images": {
"celery": "aci.shimmercat.io/r/sc_pack:v1533",
"sc_logs_agent": "aci.shimmercat.io/r/sc_logs_agent:develop-rev-324",
"sc_pack": "aci.shimmercat.io/r/sc_pack:v1533",
"shimmercat": "aci.shimmercat.io/r/shimmercat:qs-rev-3623",
"usher3": "aci.shimmercat.io/r/usher3:master-rev-116"
}
}
```
### ScPackVersion Upload [/presentation-api/v1/upload/sc_pack/{version}/]
......@@ -704,8 +713,8 @@ The `email` field on the request data is optional, although we encourage you to
+ Response 201 (application/json)
{}
### StartAuthenticationTokenRecovery [POST /presentation-api/v1/tokens/start-recovery/]
Initiates token recovery procedure. After this request you should get an email with token recovery link.
......@@ -716,7 +725,7 @@ Initiates token recovery procedure. After this request you should get an email w
"username": "test_trial",
"email": "tests@shimmercat.com"
}
+ Response 200 (application/json)
{"status": "email sent"}
......@@ -1480,7 +1489,7 @@ Creates a new Grafana Slack Notification Channel.
"slack_channel_name": "channel-name",
"slack_hook_url": "http://example.com"
}
+ Response 201 (application/json)
+ Attributes (GrafanaSlackNotificationChannel)
......@@ -1504,7 +1513,7 @@ Updates a single Grafana Slack Notification Channel.
+ Parameters
+ channel_id: 1 (number)
+ Request (application/json)
{
......@@ -1523,7 +1532,7 @@ Deletes a Grafana Slack Notification Channel.
+ Parameters
+ channel_id: 1 (number)
+ Response 204
### GrafanaEmailNotificationChannel Collection [/presentation-api/v1/grafana-notification-channel/email/]
......@@ -1546,7 +1555,7 @@ Creates a new Grafana Email Notification Channel.
{
"grafana_uid": "new-uid"
}
+ Response 201 (application/json)
+ Attributes (GrafanaEmailNotificationChannel)
......
# Install `sc_pack`
To manage the edge servers running ShimmerCat Accelerator, we install a "pack" of programs called `sc_pack`. The installation consists of two steps: first you need to install `sc_pack` itself, and then you need to ask `sc_pack` to prepare the hierarchy of directories.
To manage the edge servers running ShimmerCat Accelerator, we install a "pack" of programs called `sc_pack`. The installation consists of three steps: first you need to install `sc_pack` itself, second you need to create deployment sites and domains to serve, and third you need to ask `sc_pack` to prepare the hierarchy of directories.
sc_pack comes with a self-sufficient installer that should hopefully work in any x64 Linux.
All the `sc_pack` services (shimmercat, sc_logsagent, redis, bots-blocking, usher(s), among others) run in a [`rkt` pod](https://coreos.com/rkt/). The advantage of this approach is that we can set individual resources quota to each service.
- One host (VPS or physical) contains several deployments.
- Each deployment runs in its own `rkt` pod.
- Each deployment/`rkt` pod is managed by a single host service, e.g. Systemd service.
The `rkt` pod uses its own filesystem, except for the directory `/srv/deployment` that is mapped to the outer host's deployment directory. This is exactly how Docker+Kubernetes does it, but without having to run Docker nor Kubernetes. Since the pod comes with its own filesystem, Python is installed directly on the image and not in a virtual environment. The `rkt` pod uses the same network namespace than the host. This means that we still need to assign ports in such a way that they don't collide with other ports in the host or other deployments.
## 1. Installation
First, you will need to install python3, python3-dev, virtualenv and git on the server. You can install them by executing:
The installer can be found at `https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-<sc_pack_version>.bootstrap`, versions will be something like: 2020.1 and we will increase the integer after the dot. For instance the first version can be found at:
[https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-2020.1.bootstrap](https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-2020.1.bootstrap). You can download the installer and run it in a VPS to get sc_pack installed.
$ sudo apt-get install python3 python3-dev virtualenv git
### Download sc_pack
We highly recommend that you use a Python 3 virtualenv. This virtualenv will contain an isolated copy of Python 3, `sc_pack`, supervisord and all the libraries needed for the tools and for the Celery worker. Since it's so tied to `sc_pack`, we recommend that you create the virtualenv as a subdirectory in the hierarchy we mentioned before, the so called `<venv>` directory. This can be done by executing:
```
wget https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-2020.1.bootstrap
```
### How to run the installer?
$ virtualenv /path/to/install_dir/venv -p python3
The .bootstrap files are like shell scripts, so just give execution permissions to the downloaded file:
Don't forget to replace `/path/to/install_dir/` with the actual path you use. Then you can activate it:
`$ chmod +x sc_pack-linux-amd64-2020.1.bootstrap`
$ source /path/to/install_dir/venv/bin/activate
and, run it:
You will need the `sc_pack` distributable to install. You can either find the download link by visiting the [API page](https://accelerator.shimmercat.com/presentation-api/v1/download/sc_pack/), or by directly using the below download command (requires [jq](https://stedolan.github.io/jq/)):
`$ ./sc_pack-linux-amd64-2020.1.bootstrap`
Now `sc_pack` should be installed. To verify that it is properly installed run:
```
curl -O $(curl -s https://accelerator.shimmercat.com/presentation-api/v1/download/sc_pack/?format=json | jq -r .download_url)
$ sc_pack -h
```
After you have downloaded `sc_pack` to your server, with the venv activated, do:
Note that after installation, the sc_pack command is available anywhere in the host but it still needs to be executed from a valid deployment directory most of the times.
$ pip install /path/to/<sc_pack-version>.whl
- - -
## 2. Create deployment sites and domains
A `DeploymentSite` represents a site where `sc_pack` has been deployed, and it is the root of the tree to configure everything else. In the commands below, make sure to replace `<your_authentication_token>` with your authentication token received in the first step of the getting started tutorial.
Now `sc_pack` should be installed. To verify that it is properly installed run:
<h4>Create a deployment site</h4>
To create a `DeploymentSite` use the below command:
```
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/deployment-site/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"description": "Deployment site on server test1"}'
```
$ sc_pack -h
You will get the response data:
## 2. Prepare the configuration file and the directory hierarchy
```
{
"id":<your_deployment_id>,
"customer": {
"id": <your_customer_id>,
"name":"ShimmerCat Accelerator Test"
},
"description":"Deployment site on server test1",
"name16":"<your_deployment_site_long_name>",
"secret16":"<your_deployment_site_long_secret>"
}
```
The programs need a directory as the top of the hierarchy, and some information for configuration. This information contains details about your setup, and secrets needed for the programs to send and receive data from our cloud service.
The returned values must be kept to be used later.
<h4>Create a domain for the deployment site</h4>
To create a domain for the deployment site, use the command below:
```
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/domain/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"name": "example-accelerator.shimmercat.com", "deployment_site_id": <your_deployment_id>}'
```
Note that you need to change `<your_deployment_id>` to make sure that the domain you are creating will be linked to the deployment site with this `id`. You will get the response:
```
{"id":<your_domain_id>,"name":"example-accelerator.shimmercat.com","sub_domains":[]}
```
If you would like to add more deployment sites to your domain, see more information on [Adding a new deployment for an existing domain](../how_to_guides/add-deployment.md). You are now ready to serve a website with ShimmerCat.
- - -
## 3. Directory hierarchy and configuration
The programs need a directory as the top of the hierarchy, and some information for configuration. This information contains details about your setup, and secrets needed for the programs to send and receive data from our cloud service.
### The configuration file
All configuration is structured in a YAML file, which has nesting levels. In the rest of this documentation, we sometimes refer to a nested field by using a `/`, for example `shimmercat/root-dir` refers to the field `root-dir` under the object `shimmercat`.
There is an example file inside the `sc_pack` redistributable, that can be obtained by executing:
$ sc_pack extract example.sc_pack.conf.yaml
Copy the content that appears in the terminal and create a file with extension `.yaml`, eg. `my_sc_pack_conf.yaml`.
In the configuration file, you need to edit the following:
- `install_dir`: the top directory for the hierarchy of files.
- `user`: the user that will be used to run all the programs in `sc_pack`.
This can be `root`, although is not needed nor recommended.
Instead, we recommend you create a special user for this.
Said user needs all access to the hierarchy of files, but to
nothing outside of it.
If you want, you can use the command
`adduser --system --disabled-login --no-create-home --group sc_pack`
to create a user called `sc_pack` and an associated group.
- `shimmercat/port`: the port where ShimmerCat will listen for web requests.
- `shimmercat/root-dir`: this setting is used to put an initial folder in
the `devlove.yaml` file for where to serve files from.
By default it's inside the hierarchy and called `www`, but you can also
write an absolute directory here. Just be sure the user you provided
before has read access to this directory.
- `shimmercat/bots-views-dir`: this directory contains ShimmerCat views for
the blocking bots feature. It is only required when enable_bots_blocking is True, and there is not a views-dir defined in devlove.yaml. If there is a views-dir in devlove.yaml, it is used by default.
- `logs_host_name`: a hostname to identify your logs. Any hostname will do,
so you can either write the output of the `hostname` program, or some other
label that you see fit.
This hostname will be used later in the dashboard with our service to identify
your deployment.
- `enable_bots_blocking`: <True/False> will enable the bots blocking mechanism.
- `humanity_validator_host`: host where the service to enable the humanity
validator runs. We need it because when an IP address has been automatically detected as a prospector bot, we ensure that it is actually a bot with a mechanism using [Google reCAPTCHA](https://www.google.com/recaptcha/intro/v3beta.html). That way a user can still access the website after defeating the CAPTCHA challenge, if an IP address has been wrongly classified as bot.
Note that this host should be set in the devlove.yaml consultants too.
- `humanity_validator_port`: port that will use the service to enable the humanity validator. Note that this host should also be set in the devlove.yaml.
- `google_recaptcha_site_key`: The Google reCAPTCHA mechanism is used to validate
human visitors vs bots. You can create the key through [Google reCAPTCHA admin](https://www.google.com/recaptcha/admin#list).
- `google_recaptcha_site_secret`: As mentioned before we use the Google reCAPTCHA mechanism to validate
human visitors vs bots. You can create the secret through [Google reCAPTCHA admin](https://www.google.com/recaptcha/admin#list).
- `transit_encryption_key`: a free-form password that sc_pack uses to encrypt files
to synchronized across deployments of the same deployment group.
Among other things, we use it to synchronize the certificates and their private
keys.
Make sure to set the same
`transit_encryption_key` to all the deployments you would like to
synchronize.
Also, keep this key safe, as access to it could allow a third party to gain
access to your certificates and their private keys.
The optimization cloud service does not store this key.
- `deployment_site_long_name`: the name16 provided by ShimmerCat cloud service
- `deployment_site_long_secret`: the secret16 provided by ShimmerCat cloud service
- `deployment_group`: an arbitrary name for a deployment group. A deployment group is
a set of deployments that share the same set of domains, and thus are expected to be
in sync regarding the `devlove.yaml`, the certificates and the views.
We suggest you strive to keep this name globally unique and hard to guess, as in
`www.example-supershoes.com-and-subdomains-4393pfnDf3A` .
All configuration is structured in a YAML file, called `sc_pack.conf.yaml`, which has nesting levels.
Below is an example of the configuration file, copy the content and create a file with extension `.yaml`, called `sc_pack.conf.yaml`. You can find more details about all the fields [here](/explanation/accelerator/install_sc_pack/#2-prepare-the-configuration-file-and-the-directory-hierarchy).
* We recommend creating a user especially to work with `sc_pack`, let's call it `shimmercat`. So run `$ sudo adduser shimmercat` . The user that runs the sc_pack should be on the sudo group, to do that just run: `$ sudo usermod -aG sudo shimmercat`
Remember to edit the fields according to your details:
```
amqp_server_url: amqp.staging.c.shimmercat.com
amqp_vhost: skyloft
api_access_token: <your_authentication_token> # Received in the first step of the getting started tutorial
deployment_site_long_name: <your_name16> # Update this value with the value of your deployment site's `name16` you will create on the next step
deployment_site_long_secret: <your_secret16> # Update this value with the value of your deployment site's `secret16` you will create on the next step
enable_bots_blocking: false
enable_images_optimization: false
enable_usher3: false
google_recaptcha_site_key: <your_google_recaptcha_site_key> # Only needed if enable_bots_blocking is true. If needed you should get it from https://www.google.com/recaptcha/admin/.
google_recaptcha_site_secret: <your_google_recaptcha_site_secret> # Only needed if enable_bots_blocking is True. If needed you should get it from https://www.google.com/recaptcha/admin/.
humanity_validator_host: localhost
humanity_validator_port: 8060 # The port where the service that send the Google
images_optimization_with_aws: false
improve_images_quality: false
install_dir: /path/to/sc_pack_files # Update this with the path you want the sc_pack files to be at
logs_host_name: localhost
transit_encryption_key: <your_transit_encryption_key> # User-defined variable. This will be the encryption key to encrypt and decrypt data sent from the edge servers to the Accelerator Platform and vice versa. Among other things, it is used to synchronize the certificates and their private keys. Make sure to set the same key to all the deployments you would like to synchronize. You choose the key yourself.
user: shimmercat
user_group: shimmercat
shimmercat: {bots_views_dir: views-dir, devlove_file_path: devlove.yaml, http2https_port: '8010',
listen_port: '4010', root_dir: www, scratch_dir_name: shimmercat-scratch-folder}
```
### Directory hierarchy
Once the configuration file is created, it's time to setup the directory hierarchy. If you have already a file called `sc_pack.conf.yaml` in the directory from where you are running that command, run:
Once the configuration file is created, it's time to setup the directory hierarchy, run:
$ sudo sc_pack create
Otherwise you should run:
`$ sc_pack create -f /path/to/sc_pack.conf.yaml` or if you are already on the path where the sc_pack.conf.yaml you can miss the `/path/to/sc_pack.conf.yaml` and just run the `$ sc_pack create`.
$ sudo sc_pack create -f configuration/file/path/sc_pack.conf.yaml
You will see now that the directories and files were created for the new deployment, take a look inside the `install_dir`. Each deployment has a hidden folder `.sc_pack-rkt` with some files. The files are:
You can get a copy of the config template content that sc_pack has by executing:
* `deployment-images.json`: the images that were used on this deployment. That's probably the most important file for you because in case you need a new version of sc_pack, celery, sc_logs-agent, or shimmercat this is the one you need to change.
* `detailed-versions.dhall`: the detailed versions and sha of the images that were installed on this deployment.
* `pod-manifest.json`: the manifest from where all the services were created for the deployment.
* `sc-<install_dir>.service`: a systemd service template ready to be installed at `/etc/systemd/system`.
$ sc_pack extract example.sc_pack.conf.yaml
You can then copy and paste that content on a file called `sc_pack.conf.yaml` and then run the `create` command without arguments. Subject to the same rule, it's possible to omit the `-f` in many `sc_pack` sub-commands.
E.g, if the current dir is `/srv/www.example.com` and the file `/srv/www.example.com/sc_pack.conf.yaml`, and you run
`sc_pack create`, the configuration file taken is `/srv/www.example.com/sc_pack.conf.yaml`.
All those files need a sudo user to be edited.
We recommend running the command above with root privileges so that `shimmercat` can be authorized to bind to port 443, without running ShimmerCat itself as root.
```
$ sudo chown -R shimmercat:shimmercat <install_dir>
```
Now change to user `shimmercat`
The `sc_pack ... create` command also creates an example `devlove.yaml`, which is ShimmerCat's configuration file with domains and such. The format of that file is documented [elsewhere](https://docs.accelerator.shimmercat.com/shimmercat_qs/configuration_reference/#the-devloveyaml-file), but this is a good moment to either edit it by hand or to request our cloud service to send you a version.
```
$ su shimmercat
```
```
$ cd <install_dir>
```
## 3. Deploy and setup `sc_pack`
The `sc_pack create` command also creates an example `devlove.yaml` file, which is ShimmerCat's configuration file with domains and such. You will edit this file in the coming steps, and for more info regarding the file you can [check here](/explanation/shimmercat_qs/configuration_reference).
You are now ready to setup and deploy your `sc_pack` configuration, see [using sc_pack](../sc_pack/using_sc_pack.md)
You can double check that the directories are properly created:
```
$ ls
$ celery redis shimmercat devlove.yaml sc_logs_agent shimmercat-scratch-folder views-dir # And some others
```
### Run the services
Do a symlink (as root) of `<install_dir>/.sc_pack-rkt/sc-<install_dir>.service` to `/etc/systemd/system`
```
$ sudo ln -s <install_dir>.sc_pack-rkt/sc-<install_dir>.service /etc/systemd/system
```
Reload the systemctl daemon:
```
$ sudo systemctl daemon-reload
```
Start the services:
```
$ sudo systemctl start sc-<install_dir>.service
```
Check the services statuses:
```
$ sudo systemctl status sc-<install_dir>.service
```
Inside every service directory there is a directory named `data`, where the logs for each service are stored. If there are errors, please check the logs for more details. For example, if you see the error `celery (exit status 1; not expected)`, you can check more details in the log file `<install_dir>/celery/data/logs/current.log`.
## 4. Deploy and setup `sc_pack`
You are now ready to setup and deploy your `sc_pack` configuration, see [using sc_pack](../sc_pack/using_sc_pack.md)
# Install and configure Django web application using uWSGI and ShimmerCat
# Install and configure Django web application using uWSGI and ShimmerCat
## 01) Requirements
### Create an authentication token
The token enables to use ShimmerCat’s cloud service for automatic website optimization. You can simply create a token by providing a username and an email address by using the below command.
The token enables to use ShimmerCat’s cloud service for automatic website optimization. You can simply create a token by providing a username and an email address by using the below command.
```
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/trial/ -H 'Content-type: application/json' -d '{"username": "<your_username>", "email": "<your_email>"}'
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/trial/ -H 'Content-type: application/json' -d '{"username": "<your_username>", "email": "<your_email>"}'
```
Note: Write down your username and authentication token as you will use it later.
Note: Write down your username and authentication token as you will use it later.
### Create user and group
It is recommended to create a user `shimmercat` with group `shimmercat` using sudo privileges. Create user shimmercat:
```
$ sudo adduser shimmercat
$ sudo adduser shimmercat
```
Give sudo privileges:
```
$ sudo usermod -aG sudo shimmercat
$ sudo usermod -aG sudo shimmercat
```
Next change into shimmercat user:
```
$ su shimmercat
$ su shimmercat
```
### Create python virtualenv
Sc_pack requires; python3, python3-dev, virtualenv and gcc to deploy. Use the below command to install the requirements.
```
$ sudo apt-get install python3 python3-dev virtualenv gcc
$ sudo apt-get install python3 python3-dev virtualenv gcc
```
To keep everything organized create a directory which will be your install_dir, You can use a meaning full name of your choice:
Next cd into the install_dir directory
```
$ mkdir /path/to/install_dir
$ cd /path/to/install_dir
```
We heartily recommend using a python virtual environment to keep all the project dependencies isolated. To create a virtualenv:
```
$ virtualenv /path/to/install_dir/venv -p python3
```
## 02) Download sc_pack.
Activate the virtualenv:
```
$ source /path/to/install_dir/venv/bin/activate
```
sc_pack comes with a self-sufficient installer that should hopefully work in any x64 Linux.
The installer can be found at https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-2020.1.bootstrap
Next cd into the install_dir directory
```
$ cd /path/to/install_dir
```
You can download the installer by running the following command:
## 02) Download sc_pack
It requires `jq` to download using the curl command
```
$ sudo apt-get install jq
$ wget https://storage.googleapis.com/downloads-shimmercat-com-561059/sc_pack/rkt/sc_pack-linux-amd64-2020.1.bootstrap
```
Next download the latest sc_pack by using:
## 03) Install sc_pack
Next cd into the directory with the downloaded package and install it using the below command:
```
$ curl -O $(curl -s https://accelerator.shimmercat.com/presentation-api/v1/download/sc_pack/?format=js on | jq -r .download_url)
$ chmod +x sc_pack-linux-amd64-2020.1.bootstrap
```
And:
## 03) Install sc_pack
Next cd into the directory with the downloaded package and install it using the below command:
```
$ pip install /path/to/install_dir/<sc_pack-version>.whl
$ ./sc_pack-linux-amd64-2020.1.bootstrap
```
To verify weather sc_pack is properly installed run:
```
$ sc_pack -h
$ sc_pack -h
```
## 04) Create deployment sites and domains
To create a Deployment Site use the below command:
```
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/deployment-site/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"description": "Deployment site on server test1"}'
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/deployment-site/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"description": "Deployment site on server test1"}'
```
You will get the response data:
```
{
"id":<your_deployment_id>, "customer": {
"id": <your_customer_id>, "name":"ShimmerCat Accelerator Test" },"description":"Deployment site on server test1", "name16":"<your_deployment_site_long_name>", "secret16":"<your_deployment_site_long_secret>" }Note: Note down the output in a text file since the “deployment_id, name16 and secret 16 will be used in later steps.
Create a domain for the deployment site Use the below command to create a deployment site by providing the domain name that will be used to serve the side along with the deployment site id which is obtained through the previous step as <your_deployment_id>
{
"id":<your_deployment_id>, "customer": {
"id": <your_customer_id>, "name":"ShimmerCat Accelerator Test" },"description":"Deployment site on server test1", "name16":"<your_deployment_site_long_name>", "secret16":"<your_deployment_site_long_secret>" }Note: Note down the output in a text file since the “deployment_id, name16 and secret 16 will be used in later steps.
Create a domain for the deployment site Use the below command to create a deployment site by providing the domain name that will be used to serve the side along with the deployment site id which is obtained through the previous step as <your_deployment_id>
$ curl -X POST https://accelerator.shimmercat.com/presentation-api/v1/domain/ -H 'Authorization: Token <your_authentication_token>' -H 'Content-type: application/json' -d '{"name": "example-accelerator.shimmercat.com", "deployment_site_id": <your_deployment_id>}'
```
To add a new deployment for an existing domain refer: https://docs.accelerator.shimmercat.com/how_to_guides/add-deployment/
To add a new deployment for an existing domain refer: https://docs.accelerator.shimmercat.com/how_to_guides/add-deployment/
## 05) Create directory hierarchy and configuration
The configuration file All the configurations are structured in YAML file, extract an example file by executing the below command:
```
$ sc_pack extract example.sc_pack.conf.yaml
```
Next copy the content that appears in the terminal to a file named `sc_pack.conf.yaml`. Now edit the fields according to your details and change the user and user_group to shimmercat. For more details about the fields please refer to https://docs.accelerator.shimmercat.com/explanation/accelerator/install_sc_pack/#2-prepare-the-configuration-file-and-the-directory-hierarchy
All configuration is structured in a YAML file, called `sc_pack.conf.yaml`, which has nesting levels.
Below is an example of the configuration file, copy the content and create a file with extension `.yaml`, called `sc_pack.conf.yaml`. You can find more details about all the fields [here](/explanation/accelerator/install_sc_pack/#2-prepare-the-configuration-file-and-the-directory-hierarchy).
Remember to edit the fields according to your details:
```
$ vim sc_pack.conf.yaml
amqp_server_url: amqp.staging.c.shimmercat.com
amqp_vhost: skyloft
api_access_token: <your_authentication_token> # Received in the first step of the getting started tutorial
deployment_site_long_name: <your_name16> # Update this value with the value of your deployment site's `name16` you will create on the next step
deployment_site_long_secret: <your_secret16> # Update this value with the value of your deployment site's `secret16` you will create on the next step
enable_bots_blocking: false
enable_images_optimization: false
enable_usher3: false
google_recaptcha_site_key: <your_google_recaptcha_site_key> # Only needed if enable_bots_blocking is true. If needed you should get it from https://www.google.com/recaptcha/admin/.
google_recaptcha_site_secret: <your_google_recaptcha_site_secret> # Only needed if enable_bots_blocking is True. If needed you should get it from https://www.google.com/recaptcha/admin/.
humanity_validator_host: localhost
humanity_validator_port: 8060 # The port where the service that send the Google
images_optimization_with_aws: false
improve_images_quality: false
install_dir: /path/to/sc_pack_files # Update this with the path you want the sc_pack files to be at
logs_host_name: localhost
transit_encryption_key: <your_transit_encryption_key> # User-defined variable. This will be the encryption key to encrypt and decrypt data sent from the edge servers to the Accelerator Platform and vice versa. Among other things, it is used to synchronize the certificates and their private keys. Make sure to set the same key to all the deployments you would like to synchronize. You choose the key yourself.
user: shimmercat
user_group: shimmercat
shimmercat: {bots_views_dir: views-dir, devlove_file_path: devlove.yaml, http2https_port: '8010',
listen_port: '4010', root_dir: www, scratch_dir_name: shimmercat-scratch-folder}
```
### Directory hierarchy
Next create the deployment environment by executing the command:
```
$ sc_pack create -f configuration/file/path/sc_pack.conf.yaml
```
Now check whether the directories are properly created in the present working directory by using:
```
$ ls
```
Once the configuration file is created, it's time to setup the directory hierarchy, run:
The directories and files will be celery, redis, shimmercat, devlove.yaml sc_logs_agent, shimmercat-scratch-folder, views-dir and some others.
`$ sc_pack create -f /path/to/sc_pack.conf.yaml` or if you are already on the path where the sc_pack.conf.yaml you can miss the `/path/to/sc_pack.conf.yaml` and just run the `$ sc_pack create`.
And now you can start the accelerator:
```
$ sc_pack supervisord
```
It will give a log putput similar to:
```
2019-05-18 14:52:00,385 INFO success: redis entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-05-18 14:52:00,385 INFO success: shimmercat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-05-18 14:52:00,385 INFO success: kv_store entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-05-18 14:52:00,385 INFO success: humanity_validator entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
```
You will see now that the directories and files were created for the new deployment, take a look inside the `install_dir`. Each deployment has a hidden folder `.sc_pack-rkt` with some files. The files are:
* `deployment-images.json`: the images that were used on this deployment. That's probably the most important file for you because in case you need a new version of sc_pack, celery, sc_logs-agent, or shimmercat this is the one you need to change.
* `detailed-versions.dhall`: the detailed versions and sha of the images that were installed on this deployment.
* `pod-manifest.json`: the manifest from where all the services were created for the deployment.
* `sc-<install_dir>.service`: a systemd service template ready to be installed at `/etc/systemd/system`.
Now, halt sc_pack by pressing CTRL+C, and then open a new systemd service file.
## 06) Create sc_pack service as a systemd service
Create a simple alias to your domain name.
All those files need a sudo user to be edited.
```
$ MY_SITE_ALIAS=www_mysite_com
$ sudo chown -R shimmercat:shimmercat <install_dir>
```
Now change to user `shimmercat`
To obtain the content to the systemd service file:
```
$ sc_pack extract -f configuration/file/path/sc_pack.conf.yaml \ --options user=shimmercat \ sc_pack.service.simple
$ su shimmercat
```
Copy the result of the above command and paste it in the file as shown below and next save the file and exit.
```
$ sudo vim /etc/systemd/system/${MY_SITE_ALIAS}.service
$ cd <install_dir>
```
To use the newly created service, reload systemd:
The `sc_pack create` command also creates an example `devlove.yaml` file, which is ShimmerCat's configuration file with domains and such. You will edit this file in the coming steps, and for more info regarding the file you can [check here](/explanation/shimmercat_qs/configuration_reference).
You can double check that the directories are properly created:
```
$ systemctl daemon-reload
$ ls
$ celery redis shimmercat devlove.yaml sc_logs_agent shimmercat-scratch-folder views-dir # And some others
```
You can now use sc_pack at the boot by enabling the service using the following command:
### Run the services
Do a symlink (as root) of `<install_dir>/.sc_pack-rkt/sc-<install_dir>.service` to `/etc/systemd/system`
```
$ systemctl enable ${MY_SITE_ALIAS}.service
$ sudo ln -s <install_dir>.sc_pack-rkt/sc-<install_dir>.service /etc/systemd/system
```
You can now start sc_pack services by using the following command:
Reload the systemctl daemon:
```
$ systemctl start ${MY_SITE_ALIAS}.service
$ sudo systemctl daemon-reload
```
To make sure sc_pack services are running, check the service’s status:
Start the services:
```
$ sc_pack ctl status all
$ sudo systemctl start sc-<install_dir>.service
```
Note: You can use the below commands to start, stop or restart the sc_pack services:
Check the services statuses:
```
sc_pack ctl -f configuration/file/path/sc_pack.conf.yaml \ start|restart|stop \ shimmercat|redis|sc_logs_agent|all
$ sudo systemctl status sc-<install_dir>.service
```
Inside every service directory there is a directory named `data`, where the logs for each service are stored. If there are errors, please check the logs for more details. For example, if you see the error `celery (exit status 1; not expected)`, you can check more details in the log file `<install_dir>/celery/data/logs/current.log`.
## 07) Setup Django with uWSGI
### Install and configure Django
If you are creating a new django application refer this link; https://docs.accelerator.shimmercat.com/tutorials/django/
If you are creating a new django application refer this link; https://docs.accelerator.shimmercat.com/tutorials/django/
Or if you are using an exsisting django application which is in a git repository follow the steps below; Use the same python virtualenv and directory used to install and configure the sc_pack.
......@@ -194,7 +207,7 @@ $ pip install django-cors-headers