From The Linux Foundation

LSB Testing

Even though Linux is developed in a highly decentralized manner, in order to be attractive to the Independent Software Vendors (ISV) community, Linux must provide the same long-term compatibility guarantees and comprehensive compatibility testing as proprietary platforms such as Microsoft Windows. That's why the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences are jointly putting huge resources for developing new tools and technologies to break through Linux Standard Base (LSB) testing challenges. These tools and technologies are collectively known as LSB Testing Framework, which includes the following components:

• Test Execution Frameworks to ease test execution and results analysis:
  • Application Testkit (ATK) Manager - a web-based tool for checking applications for LSB compliance and analyzing external interfaces required by applications to run correctly.
  • Distribution Testkit (DTK) Manager - a web-based tool for testing Linux distributions - selecting which tests to run, execute them and finally analyze colorful HTML reports.
• Test Development Frameworks to ease development of new tests in different value & cost segments:
  • AZOV Shallow Test Development Framework - allows semi-automatic construction of shallow tests by using information about interfaces from the central LSB Database (low value & low cost).
  • T2C Normal Test Development Framework - allows automated development of medium quality tests (medium value & medium cost).
  • UniTESK Deep Test Development Framework - allows automated development of sophisticated deep tests for critical interfaces (high value & high cost).

Tests produced under T2C and UniTESK frameworks include explicit linkage of test failures to specific requirements of the source standard. So when a test fails it says which specific assertion from the text of the standard is violated and provides details about specific inconsistencies. This gives unprecedented comfort for test users.

LSB Navigator

The Testing Framework is leveraged by LSB Navigator - a web-portal for LSB community, which interlinks the various moving parts that make up the Linux platform to an unprecedented degree. This provides upstream package developers and downstream distribution vendors with a powerful set of tools for coordinating their work and improving the quality of the platform, as well as giving ISVs a more effective way to provide feedback to both parties.

One of the most challenging aspects of maintaining a standard for the Linux community is tracking all the moving parts. A typical Linux distribution is made up of hundreds of independent “upstream” open source components, each of which evolves at its own pace. The LSB tracks the independently evolving Linux distributions to guarantee cross-distribution portability, which adds a third dimension to an already complex task. This coordination is crucial to ensure the Linux platform’s long term success and LSB Navigator is to play an important role here.

LSB Infrastructure Program

LSB Testing Framework and LSB Navigator are developed under LSB Infrastructure Program jointly run by the Linux Foundation and the Institute for System Programming of the Russian Academy of Sciences (ISP RAS). This multi-million dollar cooperation takes advantage of hundreds of man years of existing ISP RAS developments and plans for dozens of man years to be added. The Russian Academy of Sciences has a wealth of knowledge and experience in developing testing frameworks and informational systems, including work with HP, Intel, Microsoft, Nortel and others, and the development of the Open Linux Verification Project at the Linux Verification Center (http://linuxtesting.org).

The Testing Framewok and LSB Navigator are just selected parts of the LSB Infrastructure Program. The full program includes the following areas:

• Development of new Infrastructure Software Systems:
  • Central LSB Database & Scripts
  • LSB Navigator
  • LSB Certification System
• Development of new Testing Frameworks and Tests:
  • Test Execution Frameworks (ATK and DTK Managers).
  • 3 Test Development Frameworks for 3 grades of value&cost – shallow, normal, deep.
  • New Tests for LSB Interfaces (with corresponding quality depending on the interface "importance").
• LSB Analytical and Promotional Tasks

All results of LSB Infrastructure program are free for anyone to use under the Gnu Public License (GPL).