Back to all discussions

  •   almost 11 years ago

Definition of Compatibility

The "Criteria for Evaluation of Open Source Compatibility" update contains the following paragraphs:

-----

This condition is verified by OSEHRA utilizing a set of contestant-supplied automated software scripts that are run as “tests” against both the unmodified version of open source VistA, and against the combination of the new component integrated with open source VistA. The new component will be called “compatible with open source VistA” if the set of tests produces the same results before and after the integration.

For details on the Use Cases that contestants will have to demonstrate via automated scripts, please refer to the document “TCMS – Step 1 Test Cases”

------

From these paragraphs, it looks like we are supposed to design automated scripts that showcase and test the new functionality of our scheduling module, but the tests must produce the same results when run against unmodified VistA as when run against VistA with our new module.

The contest requirements seem to indicate that the test scripts should definitely be testing the new functionality we create in our submission to the contest, so by definition they require functionality that is not present in unmodified vista. In that case, I would expect our test scripts to fail (that is, produce different results) against unmodified vista; unfortunately, from the definition of compatibility above, it appears that if our automated tests for our new module do not run successfully on unmodified vista, we will fail the contest.

Can we get some clarification on this issue?

Thanks!

  • 9 comments

  •   •   almost 11 years ago

    We have similar questions. Several specific discussion items include:
    1. What documentation is available to fully define the current VistA scheduling functionality that must be verified for each Use Case? Is all required functionality currently available in VistA?

    2. Is the data provided sufficient for entry into VistA? For example, the data specified in Tables 1 and 2 of Use Case 1 do not appear to be consistent with the data required to enter a new entity/facility in VistA (see Hospital Location/Institution File).

    It is not clear how any test scripts could successfully run against the 3 VistA instances in their current configuration and the "new" scheduling application unless the new submission is modified VistA.

  • Manager   •   almost 11 years ago

    1. The test scripts required in Step 1 need not exercise any of the new capabilities required by the contest. They are meant only to ensure that the contestant's submission continues to provide basic capabilities already present in VistA.
    2. A secondary purpose of the Step 1 testing is to ensure that software submitted by a contestant will not adversely affect any of the capabilities already present in VistA. This is necessary so that other systems that rely on VistA will continue to work alongside the new capabilities implemented by a contestant, and will still have access to the data they need. This would be essential for a successful, phased migration from the existing system to any new system in the future.
    3. The contestant must provide scripts that demonstrate that the integration of their solution into VistA satisfies the eight use cases listed in the contest documents for Step 1 testing.
    4. It must be possible to run the Step 1 test scripts repeatedly with identical results, if the system is first restored to the state it was in on the day of its submission to the VA.
    5. A contestant's submission may provide new data and new places to store data, but it must also store in VistA's database any information that is already being stored there.

  •   •   almost 11 years ago

    Please provide further clarification regarding response 5. above. If the proposed system delivers all required scheduling capability and retains data in a separate database, to what extent must VistA continue to maintain "any information that is already being stored there." Storing data in multiple places creates issues for integrity and consistency. For example, appointment type is used throughout the scheduling process; do other VistA applications also use the appointment type file?

    Is a reference document available to determine what data must be retained in VistA to ensure that "other systems that rely on VistA" will have access to the appropriate scheduling data? Is an ICD available for VistA scheduling to define interface requirements between VistA and external systems?

  •   •   almost 11 years ago

    Per the VistA Open Source compatibility definition, "all the functionalities of VistA must continue to operate as they did before the new component was integrated." Does this mean that the current VistA scheduling software must remain operational "as is" in addition to any new scheduling functionality submitted for evaluation? If an entirely new scheduling application is provided, does the VA anticipate both VistA and "new" scheduling capabilities to be available concurrently? Is it acceptable to disable the VistA scheduling menu while while maintaining the underlying table structure/data for access to scheduling data by other VistA applications?

  • Manager   •   almost 11 years ago

    Yes, the current VistA scheduling software must remain operational “as is” in addition to any new scheduling functionality submitted for evaluation.
    Yes, VA anticipate both VistA and “new” scheduling capabilities to be available concurrently.
    No, it is not acceptable to disable VistA scheduling menu while maintaining underlying table structure/data for access to scheduling data by other VistA applications.

  • Manager   •   over 10 years ago

    Question: Please provide further clarification regarding response 5...

    Storing the same data at multiple places in a database or among multiple databases is indeed bad practice, and we recommend against it to the extent possible. However, it is necessary that packages outside of the scheduling application continue to have access to up-to-date and changing data. We can appreciate the difficulties of this position, but removing data from the database raises issues of backup integrity and silent failures of scheduling-dependent code. As to your example, indeed the appointment-type file is used by several VistA applications including the Scheduling package. For details see OSEHRA VistA Cross Reference documentation:http://code.osehra.org/dox/Global_XlNEKDQwOS4x.html

    Question: Is a reference document available...

    The VistA system is a widely deployed, long-lived system, and we are not aware of any comprehensive set of readily available documents that can guarantee successfully capturing all aspects of Interface Control. In fact, VA does not use ICDs (Interface Control Documents) that define an interface, but instead uses ICRs (Interface Control Registrations) which are formal and informal agreements among the package maintainers to coordinate changes to the API. These have not been fully released and may not be comprehensive. For details see ALL Active IA (Interface Agreement) Description Nov 2010, http://downloads.va.gov/files/FOIA/Software/Integration_Agreements/ALL%20ACTIVE%20IA%20DESCRIPTIONS%20NOV%202010.txt
    OSEHRA does provide an analysis of the interactions among the FOIA packages at http://code.osehra.org/dox/ that can be used to view the interactions observable from within the code, but this is necessarily limited to a single instance of VistA. We recommend maintaining the API of the current scheduling package as it currently stands to avoid breaking VistA instances running at the varied sites.

  • Manager   •   over 10 years ago

    Question: Per the VistA Open Source compatibility definition...

    The VistA system is widely deployed, and it is unlikely that any scheduling upgrade will be implemented at all sites simultaneously. Disabling the current scheduling menu (with the exception of menus for reporting functions) for sites where the new software has been deployed, while maintaining programmatic access to the scheduling data, is perfectly acceptable, provided it does not require a global roll-out of the new scheduling capability to all VA sites in a single step.

  •   •   over 10 years ago

    In the response above, you state "We recommend maintaining the API of the current scheduling package as it currently stands to avoid breaking VistA instances running at the varied sites." Please explain. How do you define the 'API of the current scheduling package'? Where will we find the supporting API documentation?

  • Manager   •   over 10 years ago

    As noted in some of the previous answers, as is typical for a large, living, and long lived code base, the legacy code from the VA does not have a reliable, formal Interface Control Document (ICD) defining the API. The most rigorous way to ensure maintaining the current API would be to write a comprehensive set of unit and regression tests that fully capture the state of the current system and to then ensure that these tests continue to pass after your code changes are in place; however, that is a daunting effort and is beyond the scope of what we think the contestants will be able to achieve.

    More realistically, we expect that your changes will maintain the current calling sequence for all existing modules and entry points, and will continue to keep the current M database files synchronized with whatever data it they currently hold; i.e. if scheduling data are captured by the interface and are maintained in an M file prior to the addition of your code, then the same M file will continue to hold that data after the addition of your new scheduling code. We recognize that this is not an easy barrier, but part of the purpose of this contest is to reduce the acquisition risk associated with exactly these types of complications. As you research this during your development, you may find the OSEHRA "VistA Cross Reference Documentation" at http://code.osehra.org/dox/ and VistA Document Library (VDL) for Scheduling http://www.va.gov/vdl/application.asp?appid=100 to be useful.

Comments are closed.