installing-hck-ci-from-scratch.txt

Installing hck-ci from scratch
------------------------------

Install the server
--------------------
Start by installing a [Fedora server](https://getfedora.org) edition and following checklist for new server in installing-hck-ci-from-scratch.txt

Build QEMU
----------
Clone and bulid qemu from: https://github.com/qemu/qemu

```
mkdir build
cd build
../configure
make
```

Install Ruby
------------
Check that you have ruby installed on the host. Ruby installation instructions (better use RVM): https://rvm.io
We tested on 2.4.3 and we assume it can work on later versions as well.

Deploy AutoHCK
--------------
1. Get a copy of AutoHCK release package: https://github.com/HCK-CI/AutoHCK/releases
2. Extract the archive, and read the package release notes.
4. Edit optional configuration according to the release notes.
3. Run `./install.sh` in the release package

Configure Jenkins
-----------------
1. Start and enable libvirt:

   * Add your user to the libvirt group: usermod --append --groups libvirt `whoami`

   * Start and enable libvirt service: sudo systemctl enable libvirtd --now

2. Start and enable docker service: sudo systemctl enable docker --now

3. Add permission to the docker socket: sudo chmod 666 /var/run/docker.sock

4. create a dockerfile to install libvirt client on jenkins image (vi jenkins.dockerfile):

   vi jenkins.dockerfile

   ```
   FROM jenkins/jenkins:lts
   USER root
   RUN apt-get update && apt-get install -y libvirt-clients libvirt-daemon-system
   USER jenkins
   ```

5. Build the docker image: docker build . -t hck-ci/jenkins -f jenkins.dockerfile

6. Run docker container (detached with auto restart): 
   docker run -d --restart unless-stopped -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home hck-ci/jenkins
   
7. Find the container id: docker ps

8. Open a shell inside the container with: docker exec -i -t <container_id> /bin/bash

9. Create a new private key inside the container: ssh-keygen -t rsa

10. Copy the public key to the host: ssh-copy-id -i ~/.ssh/id_rsa user@host
    * If you can't ssh the host detach from container and run:
    cat /var/lib/docker/volumes/jenkins_home/_data/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    
11. Make sure you can SSH from container to host  and detach from container (ctrl-d):
    ssh -l <host_username> <host> -i /var/jenkins_home/.ssh/id_rsa

12. Open a browser with server address and allocated port eg http://myserverip:8080,

13. Use the initial password located at (from host): /var/lib/docker/volumes/jenkins_home/_data/secrets/initialAdminPassword to setup Jenkins.

14. Install following plugins ("Manage" -> "Manage plugins" -> "Available") And restart Jenkins when done :
    * Libvirt Slaves plugin
    * Copy artifacts
    * Parameterized Trigger
    * Github Pull Request Builder
    * Rebuilder

15. Make sure the machine has enough free space allocated: (This is needed on automatically provisioned machines):
    # check free space and mappings
    df -h

    # Extending can be done on fedora LVM partitioning with:
    lvextend /dev/fedora/root -l+100%FREE
    xfs_growfs /

16. Copy your Windows drivers Build VM imagse to libvirt images dir located at: var/lib/libvirt/images
    cp win8.1_x64.qcow2 /var/lib/libvirt/images

17. Defined the VM with visrsh xml file:

    Inside the XML file set the vnc port and use address 0.0.0.0 to make it available outside the machine:

    <graphics type='vnc' port='0' autoport='yes' address='0.0.0.0'>
          <listen type='address' address='0.0.0.0'/>
    </graphics>

    run: virsh define win8.1.xml

18. Add new Slave to Jenkins as a libvirt node (The will be the driver build vm slave):

    Click on "Manage Jenkins" from side pannel -> "Configure System" -> Go to the bottom of the page and click on "Add a new cloud"

    Hypervisor Type - should be QEMU
    Hypervisor Host - your hostname
    Username - HCK CI user name on the host

    Click on "Advanced button" to configure connection with ssh key ->
    Put "1" in " Concurrent Slaves Capacity" field ->
    Click on "Add" button ->
    Choose "Jenkins" from drop down list ->
    In "Jenkins Credentials Provider: Jenkins" dialog ->
    Switch kind to "SSH Username with private key" ->
    Type the host Username->
    Click on "Enter directly" and paste the private key you created in step 10 ->
    Put your host user name in "Username" field ->
    Click "Add" button ->
    Check that in "Native Credentials" drop down list there is your recently added credentials ->
    Click on "Test Connection" -> You should see something like "OK: qemu+ssh://username@host.com:22/system?no_tty=1, version=2010001" ->
    Click "Save" button

19. Click on "Build Executor Status" from left pannel ->
    Click on "New node from left pannel ->
    Give the node a name: build_vm and select "Slave virtual computer..." and click "OK" ->
    Fill the form as followed:
    * Virtual Machine: --leave empty --
    # of executors: 1
    * Remote FS root: c:\
    * Usage: Only build jobs with label expressions matching this node
    * Secondary launch method: Launch agent via Java Web Start
    As of Jenkins version 2.176.3, there is option enabled when "Launch agent by connecting it to the master" which is same as "launch agent via java web start"
    * Availability - "Take this agent online when in demend, and offline when idle", set delays to 1
    -> save changes and go back to configure slave ("Build executer status" -> Configure) ->
    Now your driver build vm will be available under "Virtual Machine".
    Save changes again.

20. Connect to the build_vm with VNC client: <ipaddress>:< 5900 + the_defined_port >

21. Inside the slave VM download the slave agent: ```http://myserverip:8080/computer/build_vm/slave-agent.jnlp``` 
    (change to the name used when configuring slave instead of build_vm)
    * Make sure you can access the jenkins dasboard from hostname, if you can't you must change it in the jenkins configuration to the real ip before you download

22. Edit the jnlp file with text editor, remove both XML tags: "workdir" and "internaldir" and save the file.

    * Configure Java security settings. Run Java configuration.

    Go to "Advanced" tab:

    * JNLP File/MIME Association - Always allow

    * Secure Execution Environment - Uncheck Show sandbox warning banner, uncheck warn if site certificate does not match hostname, uncheck show site certificate from server even if it is valid

    * Mixed code - Disable verification

    * Perform certificate revocation checks on - Do not check

    * Advanced Security Settings - Uncheck enable blacklist revocation check

    Go to "Security" tab:
    * Add the site to Security tab > edit site list > add button > type URL or IP.

    * Run the file as administrator with javaws slave-agen.jnlp

    * Install as a service: "File" > "Install as a service"

    In case service already exists, stop and delete it first:

    sc stop jenkinsslave-c__
    sc delete jenkinsslave-c__
    
23. Now turn of vm and make sure you can run it and launch agent directly from Jenkins:
    Jenkins -> Build Excutor Status -> build_vm -> launch agent

24. Now we will add another slave for running AutoHCK tests

    "Build Executor Status" ->
    "New node" -> Check "Permanent Agent" and use "Node name" - "linux_host" ->Click "OK" ->
    Select from "Usage" check list "Only build jobs with label expressions matching this node" ->
    Change "# of executors" to "2" ->
    Select from "Launch method" check list "Launch slave agents via SSH" and select credentials (should be already configured) -> "Click save"

25. Copy jobs all jobs from daynix/hck-ci-jenkins-config to /var/lib/docker/volumes/jenkins_home/_data/jobs/
    * You will first have to match external jobs fils with your own AutoHCK path and slaves names.
    * Open a shell inside the container again and make sure the jobs file are owend by Jenkins user
    cd /var/jenkins_home/jobs
    chown -R jenkins:jenkins *
    * Load new jobs on jenkins with: Jenkins -> Manage Jenkins -> Reload Configuration from Disk

GitHub Pull Request Builder
-----------------------------
Install 'GitHub Pull Request Builder' (Manage Jenkins -> Manage Plugins -> Available -> check "GitHub Pull Request Builder" and click on "Install without restart" and after installation is finished, check "restart") in Jenkins and configure it as followed:

Setup Pull request trigger configuration
* Go to 'Manage Jenkins' on the left menu and then 'Configure System'
* Scroll down to 'GitHub Pull Request Builder'
* Add a new credentials, use a token created at github or create a new one at: https://github.com/settings/tokens, add a description to the credential to identify it as the github access token.
* Select it from the credential list and apply changes.

Setup Pull request trigger for project

* Click in the build job name on the main page of Jenkins (VirtIO)

* Click in "Configure" in the left panel

* Click on the  'Source Code Management' tab

* On the advanced section change 'Refspec' to "+refs/pull/*:refs/remotes/origin/pr/*"

* Below that change the 'Branch Specifier' to "${sha1}"

* Click on the "Build triggers" tab

* Check 'GitHub Pull Request Builder'

* Click advanced, then mark the checkbox on 'Build every pull request automatically without asking (Dangerous!).'

* Below that on 'Trigger Setup' add 'Update commit status during build' and 'Build Status Messages' to make a custom text to show on github when updating.

Storage related devices tests
-----------------------------
Some devices has tests that requires to create additional partitions:
https://docs.microsoft.com/en-us/windows-hardware/test/hlk/testref/file-system-testing-prerequisites

* Extract the filesystem_tests_image.qcow2.zip file into the VirtHCK images folder to make VirtHCK load these partitions automatically.
* You can create it your own by following this: https://github.com/daynix/VirtHCK/tree/master/guest_tools/StorageAutoPartition
* This might not always work since the drive letter assignment is saved within windows files, in such cases you will need to assign the letters manually before testing such devices.