|
ReleaseNotes26
Release notes for Robot Framework 2.6 series
Robot Framework 2.6Robot Framework 2.6 is a new major release with loads of bigger and smaller enhancements and bug fixes. It was released after several preview releases on Tuesday 26th July 2011. Questions and comments related to the release can be sent to the mailing lists and possible bugs submitted to the issue tracker. Downloads and installationInstallers and other packages are available on the download page. You can follow the normal installation instructions when installing RF 2.6 releases. If you have an earlier Robot version installed it should be safe to install over it. Removing the old version first is always safer, though, and should always be done if installation fails somehow. Compatibility with RIDE and MabotRobot Framework 2.6 should be fully compatible with the earlier RIDE releases but unfortunately it is not compatible with the current Mabot release. Compatibility with outputs generated with RF 2.5Robot Framework 2.6 can read and process outputs generated by 2.5.x. releases. This allows testing the enhanced log and report (see below) also without executing tests using 2.6. All you need to do is get the output file (such as ouput.xml) and generate logs and reports based on it with rebot. Most important enhancements and new featuresBetter log file performanceThe biggest enhancement in this release is making the performance of the log files better when the number of test cases and test data increases ( issue 360 ). This enhancement is implemented so that when you open the log file it is not loaded into the browsers memory fully but generated dynamically using Javascript (and wonderful jQuery) as you drill down to more details. This both makes the log files' size on a disk much smaller (in one case a log file that used to be 120MB is now 9MB) and also reduces browsers' memory consumption dramatically (the earlier log file performed flawlessly on a browser). Creating the log files dynamically allows also interesting enhancements in the future. Possible ideas include searching test cases and setting log level. In RF 2.6 the log files should look pretty much like the ones we all have got used to in previous releases, though. Transparent log splittingA common problem when the number of test cases increases is that log files gets too big. They do not open nicely into browsers anymore and if they are on a remote machine downloading them takes too much time. In RF 2.5 and earlier this problem was solved with a functionality to split both log files and output files (--splitoutputs option) from a predefined level. This functionality was removed early during 2.6 development because it never worked that well and it was incompatible with the new log generation system discussed above. Although the new logs are order of magnitude smaller than earlier, some log splitting functionality was still needed with very large test suites. The new approach introduced in RF 2.6 ( issue 898 ) is splitting the log into smaller files that can be loaded dynamically when needed. This approach has two huge benefits:
The main drawback is that the overall size taken by the log file increases. In practice this should not matter too much because not all the data needs to be loaded into a browser, disk space is pretty cheap, and the logs in 2.6 are otherwise so small compared to the old logs. Another drawback is that copying log files or otherwise manipulating them after the execution gets a bit more complicated when a log file has been splitted because you have lot of small files. Technically data from each test and suite setup/teardown is written into a separate Javascript file and the number of files grows when the number of tests grows. The files have name in format 'log-42.js' where 'log' changes depending on the actual name of the log file and '42' is a growing index. The new mechanism is transparent for users viewing the log file but it still needs to be enabled with --splitlog option when tests are run or results post-processed with rebot. Enhanced and better performing reportIn RF 2.6 also report files are generated dynamically and are thus smaller and have better performance than earlier. In addition to that, reports have also got some new functionality. The smaller size (in one case from 9MB to 0.4MB) and faster opening time are due to reports not showing test cases automatically but instead having an easy way to select what you actually want to see. Old test details by suite and test details by tag tables have been removed and replaced with dynamically generated tables that show the same information ( issue 849 ). In addition to these views being accessible from the statistics table like the removed tables in earlier releases, you also have drop-down menus where you can select which suite or tag to show. A totally new functionality in reports is having a view where all tests or all critical tests can be seen ( issue 863 ). Tests in these views are sorted so that failed tests are first (similarly as when viewing tests by tag) so it is very easy to see all failed or all failed critical tests in one place. These views are also accessible from the statistics table. The screenshot below is from the User Guide that also contains a live version of the same report.
Keyword teardown functionalityRobot Framework has always had test and suite level setups and teardowns, but nowadays also individual user keywords can have their own teardowns ( issue 711 ). The mechanism of this new feature is very simple, the keyword teardown is executed both when the keyword passes and when it fails. Keyword teardowns allow, for example, splitting a large test teardown into more fine-grained teardowns that are executed closer to the activity that needs to be cleaned up. Another good use case is with test templates (a new feature in RF 2.5) where the system should be in a known state after every repeated step. Using keyword teardown and test templates is illustrated by the example below. It shows a different approach to create invalid login cases from the SeleniumLibrary demo: ***Settings***
Test Setup Open login page
Test Template Login with invalid credentials should fail
Test Teardown Close browser
Resource common_resource.txt
***Test Cases***
Invalid login
${VALID USER} invalid
invalid ${VALID PASSWORD}
invalid invalid
${VALID USER} ${EMPTY}
${EMPTY} ${VALID PASSWORD}
${EMPTY} ${EMPTY}
***Keywords***
Login with invalid credentials should fail
[Arguments] ${username} ${password}
Input user name ${username}
Input password ${password}
Submit credentials
Welcome page should be open
[Teardown] Navigate to login pageNotice that keyword teardown functionality is currently not supported by RIDE. New logging APIs and other logging enhancementsRobot Framework 2.6 has its own Python API for logging (robot.api.logger module) ( issue 339 ) and it also supports Python's standard logging module ( issue 455 ). These APIs make both make logging cleaner than using the standard output and also provide nice functionality such as accurate timestamps. They are explained more thoroughly, and with examples, in the User Guide. Another logging related enhancement is that libraries can embed accurate timestamp into the messages logged through stdout/stderr ( issue 456 ). Python based libraries are probably better of using the new logging APIs, but this enhancement adds timestamp support also for Java based libraries as well as for libraries that utilize the remote interface. Finally, messages logged during library initialization are nowadays written into the syslog ( issue 892 ). Support for custom regexps with embedded argument syntaxThe embedded argument syntax nowadays supports using a custom regular expression to match the used arguments ( issue 854 ). This is especially useful to prevent name collisions with keywords having similar names. Backwards incompatible changesViewing logs and reports requires a modern browserLogs and reports nowadays use Javascript and CSS styles that only work well in modern browsers. Firefox 3.5, IE 8, or equivalent is required, but newer browsers are recommended. Most notable IE 6 and IE 7 are not supported at all. Newer browsers, such as Firefox 4 and Google Chrome 12, have blazingly fast Javascript engines and they can easily view even very large log files. Upgrading your browser may thus be a good idea even if you can open logs and reports with your old browser. Viewing in Excel is not supported anymore. You can still copy statistics and other results from logs and reports to Excel, though. If your company policy prevents you from upgrading IE or using alternative browsers, you can always keep on using older Robot Framework versions. IronPython is currently not supportedUnfortunately Robot Framework 2.6 does not work with IronPython. You can use Robot Framework 2.5.7 with IronPython until we manage work around the incompatibilities listed in issue 917 . Splitting outputs is not supported anymoreThe earlier explained big log file performance enhancements unfortunately completely broke the old output splitting functionality ( issue 861 ). Fixing splitting so that it would work like it worked earlier would have been a really big task. Outputs have been split mainly to make log files smaller by splitting them into pieces that should not grow too big. Because the old approach never worked too well (see e.g. issue 386 and issue 862 ), a decision was made to rather implement new log splitting approach. In RF 2.6 using --splitoutputs option gives you a warning and in RF 2.7 the option will be removed altogether. New --splitlog option should be used instead. Summary reports were removedAs explained above, reports in Robot Framework 2.6 do not have tests open when you open them and in practice they look a lot like the old summary reports. Because reports are generated dynamically, their size is also very small. Because summary reports did not provide any value anymore they were removed ( issue 864 ). testdoc.py tool does not work currentlyUnfortunately testdoc.py tool is not compatible with RF 2.6 ( issue 908 ). Hopefully this useful tool can be fixed already in 2.6.1. Many internal APIs have changedThere are lot of internal changes in a big release like this and some of the changes may affect libraries or tools that are dependent on the internal APIs. The most important changes are listed in issue 907 . Deprecated keywords and arguments were removedFollowing earlier deprecated keywords or arguments have been removed:
Other possible backwards incompatible changes
Deprecated featuresThe only deprecated feature in RF 2.6 is that old screenshot taking keywords have been deprecated ( issue 733 ). AcknowledgementsVery big thanks to the JSXGraph project for creating and releasing the JSXCompressor module. One reason new logs and reports are so small is that all larger strings are compressed using the zlib algorithm and then uncompressed on-demand in the browser using this great module. Special thanks to Alfred Wassermann for making JSXCompressor available also under the Apache 2 license so that it was easier for us to include it. Big thanks also to all jQuery developers for making Javascript development a joy. Thanks to Imran for providing a great patch to issue 897 and for actively testing the preview releases. Thanks also to all others who have tested alphas, betas, and release candidates! Full list of fixes and enhancements in 2.6
Altogether 57 issues. Robot Framework 2.6.1Robot Framework 2.6.1 is a micro release that fixes the library documentation tool which inserted too many whitespace characters in Robot Framework 2.6.0. This doesn't affect normal Robot functionality at all. Only the packages with libdoc tool or the generated documentation are released. Packages are available on the download page.
Altogether 1 issues. Robot Framework 2.6.2Robot Framework 2.6.2 is a minor release which fixes a couple of high impact bugs and adds some nice usability enhancements. It was released on Friday 30th September 2011. It is now possible to log safely from listener methods via the logging API ( issue 548 , issue 953 ) and this is documented also in the user guide. A serious bug with using Python logging module from thread ( issue 950 ) was fixed. We also enhanced the generated HTML files by adding sortable columns to report ( issue 969 ), favicons ( issue 970 ), and by ensuring that IE users are not forced to IE 7 compatibility mode ( issue 957 ). All fixes and enhancements are listed in the table below:
Altogether 21 issues. Robot Framework 2.6.3Robot Framework 2.6.3 mainly fixes support running Robot Framework on .NET with IronPython ( issue 917 ) but it also contains some other fixes and makes report and log generation slightly faster ( issue 975 ). It was releases on Friday 7th October 2011.
Altogether 6 issues. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
► Sign in to add a comment