Merge pull request Azure#6196 from sassoftware/sas-develop

Update to fix errors caused by newly published sas orchestration tool

sas-viya/README.md

@@ -32,6 +32,9 @@ For assistance with SAS software, contact  [SAS Technical Support](https://suppo
     1. [Set Up SAS Data Agent](#DataAgent)
 1. [Usage](#Usage)
 1. [Troubleshooting](#Tshoot)
+    1. [Reviewing the Logs](#ReviewLog)
+    1. [Restarting the SAS Services](#RestartServices)
+    1. [Uncommon Errors that Require a Redeploy](#UncommonErrors)
 1. [Appendix A: Configuring the Identities Service](#AddA)
     1. [Verify Security Settings](#AddAVerify)
     1. [Create a Service Account](#AddACreateServiceAccount)
@@ -347,7 +350,9 @@ If your deployment fails:
 * In general, issues that occur in the primary deployment but do not spring from a sub-deployment are platform issues such as not being able to get enough resources in a timely manner. In these cases, you must redeploy.  The primary deployment is often called "azure-deploy" and the sub-deployments usually start with "AnsiblePhase..".
 
 * If the error comes from a sub-deployment (ex. “AnsiblePhase5PreViyaInstall”), review the logs.
+* With some older licenses, some services might not be fully started after a full deployment. If you receive a connection error when connecting to SASHome or SASDrive, then you must restart the services as described [here](#RestartServices).
 
+<a name="ReviewLog"></a>
 ### Review the Log Files
 Ansible is the primary virtual machine used for the install and the majority of the deployment log files reside on the Ansible virtual machine.  
 #### Ansible Server Log Files:
@@ -368,7 +373,25 @@ The /tmp directory is the primary deployment log directory.
 *	/var/log/sas: SAS logs. If there is a  startup issue after install, this will be the location of the information.
 *	/tmp	
     * prerequisites.log: log generated by the first setup script on controller VM that primes it for Ansible to run against it and mounts the /mnt/viyashare
+    
+<a name="RestartServices"></a>
+### Restarting the SAS Services
+While all the services can be started on each box independently, the Viya-Ark toolkit provides an efficient way to restart all the services across all the boxes from the ansible controller.
+
+#### Checking the status of the services through Viya-Ark
+Viya-Ark can check the status of the services by issuing the following commands as the vmuser on the ansible controller:
+```
+cd /sas/install/setup/orchestration/sas_viya_playbook/
+ansible-playbook viya-ark/playbooks/viya-mmsu/viya-services-status.yml
+```
 
+#### Restarting the services through Viya-Ark
+Viya-Ark can restart all of the services by issuing the following commands as the vmuser on the ansible controller:
+```
+cd /sas/install/setup/orchestration/sas_viya_playbook/
+ansible-playbook viya-ark/playbooks/viya-mmsu/viya-services-restart.yml -e enable_stray_cleanup=true
+```
+<a name="UncommonErrors"></a>
 ### Uncommon Errors that Require a Redeploy
 If you encounter the following errors, remove the deployment and redeploy.
 #### SSH Error: data could not be sent to remote host

sas-viya/azuredeploy.parameters.json

@@ -3,7 +3,7 @@
   "contentVersion": "1.0.0.0",
   "parameters": {
     "DeploymentDataLocation" : {
-      "value": "https://azureviya.blob.core.windows.net/mercury-deployment-data/ThisEntireLicenseUriShouldBeReplacedWithAUriToYourOwn.zip?sp=r&st=2019-04-25T02:37:44Z&se=2019-04-27T01:37:44Z&spr=https&sv=2018-03-28&sig=4ByEWfUIJtom9BQhzY%2BO%2FYLDf3MuV%2FeEsQOhSv4BzWo%3D&sr=b"
+      "value": "https://azureviya.blob.core.windows.net/mercury-deployment-data/ThisEntireLicenseUriShouldBeReplacedWithAUriToYourOwn.zip?sp=r&st=2019-05-24T00:32:46Z&se=2019-05-24T03:32:46Z&spr=https&sv=2018-03-28&sig=iQc3fWHaR%2BpvKdaKpJVRnaB6Knh0gDlYQrDE6I3QRRU%3D&sr=b"
     },
     "SSHPublicKey": {
       "value": "GEN-SSH-PUB-KEY"

sas-viya/metadata.json

@@ -5,5 +5,6 @@
   "description": "The SAS Viya Quickstart Template for Azure deploys these products on the cloud: SAS Visual Analytics 8.3.1 on Linux. SAS Visual Statistics 8.3.1 on Linux and SAS Visual Data Mining and Machine Learning 8.3.1 on Linux. This Quickstart is a reference architecture for users who want to deploy the SAS Viya platform, using microservices and other cloud-friendly technologies. By deploying the SAS platform on Azure, you get SAS analytics, data visualization, and machine learning capabilities in an Azure-validated environment. SAS Viya is a cloud-enabled, in-memory analytics engine. It uses elastic, scalable, and fault-tolerant processing to address complex analytical challenges. SAS Viya provides faster processing for analytics by using a standardized code base that supports programming in SAS, Python, R, Java, and Lua. It also supports cloud, on-premises, or hybrid environments and deploys seamlessly to any infrastructure or application ecosystem.",
   "summary": "This Quickstart is a reference architecture for users who want to deploy the SAS Viya platform, using microservices and other cloud-friendly technologies.",
   "githubUsername": "sassoftware",
-  "dateUpdated": "2019-04-24"
-}
+  "dateUpdated": "2019-05-23"
+}
+

sas-viya/playbooks/post.deployment.yml

@@ -93,13 +93,13 @@
       - /var/log/messages
 #      - /sas/install/setup/orchestration/sas_viya_playbook/roles/consul/files/sitedefault.yml
 
-- hosts:
-    - all
-  become: true
-  tasks:
-
-  - name: run async yum update
-    command: >-
-      yum update -y --exclude WALinuxAgent
-    async: 2400
-    poll: 0
+#- hosts:
+#    - all
+#  become: true
+#  tasks:
+#
+#  - name: run async yum update
+#    command: >-
+#      yum update -y --exclude WALinuxAgent
+#    async: 2400
+#    poll: 0

sas-viya/playbooks/roles/_setup_sas_orchestration_inventory_config/3.3/tasks/update.config.yml

@@ -98,6 +98,8 @@
       dest: "{{ ORCHESTRATION_VARS }}"
       regexp:  '#3: ORACLE_HOME=Oracle home directory.*'
       replace: '3: PGCLIENTENCODING=UTF-8'
+    run_once: true
+    delegate_to: localhost
 
   - name: set JAVA_HOME
     replace:

sas-viya/playbooks/roles/predeploy_sas_orchestration_virk/defaults/main.yml

@@ -1,8 +1,8 @@
 ---
 # defaults file for getLocalVIRK
 
-VIRK_CLONE_DIRECTORY: "/exports/bastion/setup/virk"
-VIRK_ORIGIN_CONNECTOR: "https://github.com/sassoftware/virk.git"
+VIRK_CLONE_DIRECTORY: "/exports/bastion/setup/viya-ark"
+VIRK_ORIGIN_CONNECTOR: "https://github.com/sassoftware/viya-ark.git"
 VIRK_PLAYBOOK: ""
 ORCHESTRATION_DIRECTORY: "/exports/bastion/setup/orchestration"
 SSL_WORKING_FOLDER: "/exports/bastion/setup/ssl"
@@ -18,17 +18,16 @@ LICENSE_FILE: ""
 #SSL_CLIENT_KEY_TARGET_FOLDER: "/etc/pki/tls/private"
 SAS:
   "3.3":
-    VIRK_COMMIT: "e210c8d"
+    VIRK_COMMIT: "9efbe65ed9a7cefdbf9db981b7974eee4f3b3933"
     ORCHESTRATION_URL: "https://support.sas.com/installation/viya/sas-orchestration-cli/lax/sas-orchestration.tgz"
     MIRROR_MANAGER_URL: ""
     ORCHESTRATION_MIRROR_MANAGER: ""
     USE_MIRROR: False
 
 
   "3.4":
-    VIRK_COMMIT: "fec76e556"
+    VIRK_COMMIT: "d43bedba1095792bf604de12cfed6cb9118e8ae5"
     ORCHESTRATION_URL: "https://support.sas.com/installation/viya/34/sas-orchestration-cli/lax/sas-orchestration-linux.tgz"
     MIRROR_MANAGER_URL: "https://support.sas.com/installation/viya/34/sas-mirror-manager/lax/mirrormgr-linux.tgz"
     ORCHESTRATION_MIRROR_MANAGER: "{{ ORCHESTRATION_DIRECTORY }}/mirrormgr"
     USE_MIRROR: True
-

sas-viya/playbooks/roles/predeploy_sas_orchestration_virk/tasks/main.yml

@@ -362,7 +362,7 @@
 
 - name: Build Orchestration block for sas
   shell: >-
-    {{ ORCHESTRATION_DIRECTORY }}/sas-orchestration --verbose build --input {{ LICENSE_FILE }} --output "{{ ORCHESTRATION_DIRECTORY }}/SAS_Viya_playbook.tgz" {% if USE_MIRROR %} --repository-warehouse "file://{{ LOCAL_DIRECTORY_MIRROR }}" {% endif %}
+    {{ ORCHESTRATION_DIRECTORY }}/sas-orchestration --verbose build --input {{ LICENSE_FILE }} --output "{{ ORCHESTRATION_DIRECTORY }}/SAS_Viya_playbook.tgz" --deployment-type default {% if USE_MIRROR %} --repository-warehouse "file://{{ LOCAL_DIRECTORY_MIRROR }}" {% endif %}
   args:
     chdir: "{{ ORCHESTRATION_DIRECTORY }}"
     creates: "{{ ORCHESTRATION_DIRECTORY }}/SAS_Viya_playbook.tgz"

sas-viya/scripts/ansiblecontroller_prereqs.sh

@@ -28,7 +28,7 @@ if ! type -p ansible;  then
    # install Ansible
     curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
     sudo python get-pip.py
-    pip install 'ansible==2.4.0'
+    pip install 'ansible==2.7.10'
 fi
 
 # remove the requiretty from the sudoers file. Per bug https://bugzilla.redhat.com/show_bug.cgi?id=1020147 this is unnecessary and has been removed on future releases of redhat, 

sas-viya/scripts/ansiblecontroller_startup.sh

@@ -44,7 +44,7 @@ export CAS_SIZING_FILE="\${DIRECTORY_LICENSE_FILE}/cas_size.txt"
 export FILE_SSL_JSON_FILE="\${DIRECTORY_SSL_JSON_FILE}/loadbalancer.pfx.json"
 export FILE_CA_B64_FILE="\${DIRECTORY_SSL_JSON_FILE}/sas_certificate_all.crt.b64.txt"
 export ORCHESTRATION_DIRECTORY="\${SAS_INSTALL_SRC_DIRECTORY}/setup/orchestration"
-export VIRK_CLONE_DIRECTORY="\${ORCHESTRATION_DIRECTORY}/sas_viya_playbook/virk"
+export VIRK_CLONE_DIRECTORY="\${ORCHESTRATION_DIRECTORY}/sas_viya_playbook/viya-ark"
 export CODE_DIRECTORY="\${SAS_INSTALL_SRC_DIRECTORY}/code"
 export ANSIBLE_SSH_RETRIES=10
 EOF