Ssh Library Robot Framework

(Part 2) With this tutorial you will get a Windows VM in which End2End tests are executed with Robot Framework in about 1h. With the help of the module “RobotMK” the test results of Checkmk are monitored. More… (Direct link to part 1 of the tutorial)

Selenium test

So far our test does nothing special, except for a few Sleeps. In the repository we cloned in part 1, I have another robot file that uses the Selenium library and will be our first real web test. Please open in VSC the file selenium_test/selenium_test.robot. This simple web test checks if a web page contains a table with book prices. Now please also start this test with the green arrow in the upper left corner and check if the result looks like mine: Now I come back to the special role of Python bitness. Free of charge: RobotMK whitepaper This tutorial is also available as PDF with the following advantages:

30 pages of pro tips u. Bonus material
professional book layout with link navigation
eye- and printer-friendly layout

The complete implementation path to the first Robot Framework client incl. Checkmk integration! If instead of the above result you get the error message SessionNotCreatedException: Message: Unable to find a matching set of capabilities instead of the above error message, it is almost certainly because the bitness of the downloaded webdriver does not match that of the browser. Please check first if your browser, e.g. Firefox is 64 bit: go to “Help” -> “About Firefox”. There should be something like “79.0 (64-bit)”. long explanationIf you have installed the 32-bit version instead of Python-64Bit, then the webdrivermanager probably downloaded a 32-bit driver, because at the beginning of the download it checks via an internal Python function, how large the integer number space is that can be represented with this Python interpreter. For a 32-bit Python, this is the number 2147483647. Consequently, a 32-bit web driver lands on your system. If it is started now, it looks for the installation location of Firefox in registry paths, which only exist on 32-bit Windows. short solutionEither you install the 64-bit version of Python (as recommended) and call the webdrivermanager exactly as you did before. This time the internal check of the number space will come up with a 64-bit system and also install the appropriate driver. Or you can say webdrivermanager with the special option --bitnessthat it should fetch the 64-bit version: C:\webdrivermanager --linkpath c:\webdriver --bitness 64 firefox ::: At the latest now the start of the Selenium test should work!

Installing the Selenium test

Please create a new directory below C:\ProgramData\checkmk\agent\ create a new directory robot (Attention: not “robotmk”, because here are tests of “Robot Framework”, which – primarily at least – have nothing to do with RobotMK at all). Robot-Dir: This directory is by default the place where the RobotMK agent plugin will look for Robot test definitions. DeepL Access: I recommend you to use the agent-path in Windows Explorer to the left in the DeepL access list. You will see that this is very handy because you will be working here every now and then. Now copy the folder selenium_test folder from the repository into the robot dir:

Configuration of Checkmk

Installation of the RobotMK MKP

.mkp (MK Package) are package files used to collect a set of files belonging to a project/extension. RobotMK is also installed using this proven concept. Download the RobotMK package from and install it in Checkmk via WATO -> Extension Packages -> Upload Package. If everything worked RobotMK will show up in the list of installed packages: The right column “CONTENTS” in the screenshot above gives a taste of what we are about to use:

2 agents:
- Agent Plugin: runs robot tests on the hosts.
- Bakery Plugin: creates agent packages in the Bakery with dynamically configured RobotMK control files for the agent plugin
1 Check plugin: the plugin examines on checkmk side the XML output generated by Robot as a result.
1 GUI Extension: Python definitions for the following WATO pages:
- RobotMK Discovery Level
- RobotMK Service Configuration
- RobotMK Bakery

Checkmk Bakery

Bakery-Rule: Assigning the test execution

We are ready: we can now start configuring the test execution. The Checkmk Bakery does us a great service here: via WATO rule we simply specify which tests should run on which host and the Bakery creates ready-made installation packages from this. One after the other:

Open WATO -> Monitoring Agents -> Rules
Create a new Bakery Rule RobotMK (Linux, Windows):

This rule should only match to the test host, i.e. take care of this with an appropriate folder or host assignment:

If we don’t set anything else in this rule, the RobotMK defaults apply: the RobotMK plugin would run on wind10wp and every 15 minutes all suites in the Robot folder. This is certainly not as desired, which is why we make further settings: :::

Execution should be every 5 minutes
In the robot-dir only the suite selenium_test (you can add more suites by clicking the “Add test suite” button).

Bakery rule: allowed file types

The second important rule you have to set up is called Limit script types to execute. Apply it to all hosts where Robot tests will be executed, because without it the Python-based RobotMK plugin will never work!

“Baking” new installation files

Now also the button “Agents” in the menu bar above is colored orange: it shows us that new agents have to be baked. In the following, the button “Bake Agents” starts the creation of all agent installers. Whereby this is not quite correct: the Bakery is so smart and detects changes to the ruleset. Only those agents are rebuilt for which changes are pending.

Installing the new agent

Now install the freshly built version of the agent on the Windows host. Make sure that the file C:\ProgramData\checkmk\agent\bakery\check_mk.bakery.yml looks something like this:

global: enabled: true execute: exe bat vbs ps1 py port: 6556 plugins: enabled: true execution: - pattern: $CUSTOM_PLUGINS_PATH$\robotmk.py cache_age: 900 async: yes timeout: 840

You should also see that the directory C:\ProgramData\checkmk\agent\plugins now also contains the robotmk.py-plugin.

Checkmk inventory

It may take some time for the CMK agent to run the robot test. A manual inventory run shows the newly detected service: “Fix all missing/vanished” button adds the new service to the set of monitored items.

Detail view of a robot test

Now open the detail view of the new robot service: Here you can see that the service output is not just a single line, but a hierarchical representation of the XML result that the robot test has stored in the file system of the Windows host. From the abbreviations at the beginning of the line you can see what type of partial result it is. These can be:

Suites: [S]
Tests: [T]
Keywords [K]

Suite naming: You may wonder why there is a second suite with the same name below the “Selenium Test” suite. The first suite name is generated from the name of the directory located in the robot dir on the Windows client. The second suite name results from the robot file selenium_test.robotwhich is also interpreted as a suite. By definition, a suite for Robot is a collection of individual tests (as in the case of the Robot file) and/or additional directories (such as the directory selenium_test).

Thresholds and performance data

Checkmk would not be a monitoring system if it could not also alert! In the case of Robot Framework not only failures are alerted (=when the complete service becomes WARNING or CRITICAL), but with the help of my development RobotMK also exceedings of predefined thresholds. For the test case “table technical literature” a maximum runtime of 9 seconds shall now apply as an example:

If the test takes longer than 12 seconds, this shall be considered CRITICAL.
Run times between 9 and 12 seconds as WARNING

Switch to WATO and open the view “Host and service parameters”: Search for “robot” and create a new RobotMK rule for “discovered services”: Within the rule there is a checkbox “Runtime thresholds“. Here you can set regex search patterns for suites, tests and keywords, which apply the respective WARN/CRIT thresholds very efficiently to all matching subnodes of the robot result. Info: This is where a major advantage of RobotMK becomes apparent: the test result arrives “neutrally” at the checkmk server, where it is evaluated. Since no adjustment to the robot code is required, RobotMK can theoretically also be used to monitor results of robot tests that are part of CI/CD pipelines, for example. Now activate the checkbox for “Test thresholds” and enter patterns and thresholds as shown in the screenshot below: Performance data: You can also apply the same pattern in the “Perfdata creation > Test perfdata” item. With this regex you make sure that all tests starting with “table” get a checkmk performance graph:

Summary

You can celebrate this moment! Because with the help of this tutorial you have taken the “fast lane” to get into professional End2End monitoring with Checkmk. Of course, this is just the beginning – the Robot Selenium library alone would be worth a book of its own. And the Robot ecosystem has the right library for every use case:

Web-based tests
REST API calls (very interesting e.g. to check third party systems during a test)
Access to email inboxes
Image pattern based testing (Sikuli, ImageHorizon library)
Use of AutoIT keywords
SSH access
DataDriver: dynamic parameterization with test data
Database access
and many more.

This diversity makes Robot a Swiss army knife for testing mission-critical applications. Have fun writing your Robot tests! Simon P.S.: If you think of any useful features for RobotMK, feel free to create an issue on GitHub. P.P.S.: Do you know my offer for a free consultation? Enter your text here… Ssh Library Robot Framework.